I will look into this.

Can you install numpy on your machine & try again? Telemetry has a numpy version of the implementation that's way way faster.

Deleted: test_png.py 554 bytes

test_png.py 554 bytes View Download

With the python png:
python test_png.py 
Base64 decoding took: 0 ms
PNG decoding took: 3336 ms


With numpy on my machine:
python test_png.py 
Base64 decoding took: 1 ms
PNG decoding took: 54 ms
[[[ 61 201 117]
  [ 61 201 117]
  [ 61 201 117]
  ..., 
  [255 255 255]
  [255 255 255]
  [255 255 255]]

 [[ 61 201 117]
  [ 61 201 117]
  [ 61 201 117]
  ..., 
  [255 255 255]
  [255 255 255]
  [255 255 255]]

 [[ 61 201 117]
  [ 61 201 117]
  [ 61 201 117]
  ..., 
  [255 255 255]
  [255 255 255]
  [255 255 255]]

 ..., 
 [[  0   0   0]
  [  0   0   0]
  [ 41  41  41]
  ..., 
  [ 41  41  41]
  [  0   0   0]
  [  0   0   0]]

 [[  0   0   0]
  [  0   0   0]
  [  0   0   0]
  ..., 
  [  0   0   0]
  [  0   0   0]
  [  0   0   0]]

 [[  0   0   0]
  [  0   0   0]
  [  0   0   0]
  ..., 
  [  0   0   0]
  [  0   0   0]
  [  0   0   0]]]

I also run ./run_gpu_test.py screenshot_sync --browser=system --story-filter=ScreenshotSync.WithDivs and confirm that if I disable numpy on my machine, it takes 5 times longer to run.

I can add some warning to telemetry to let people know to install numpy to speed things up.

Deprioritize this as the instruction to improve decoding speed is shown.

Err, sorry, the python module you want to install is 'cv2'

(CC'ing maruel@ and vadimsh@ for question about how to get these packages on the Swarming bots)

Thanks for the analysis. The bots need to have these packages installed. How can we go about doing this? Is it necessary to "pip install" the packages or can the binaries be bundled into Telemetry's isolates?

Also, on my MacBook Retina I pip installed numpy and cv2 and the revised test_png.py is still using image_util_bitmap_impl.

>>> print numpy.version.version
1.8.0rc1
>>> from distutils import version
>>> version.StrictVersion('1.8.0rc1')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/version.py", line 40, in __init__
    self.parse(vstring)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/version.py", line 107, in parse
    raise ValueError, "invalid version number '%s'" % vstring
ValueError: invalid version number '1.8.0rc1'

This will need to be fixed. It'll also be necessary to figure out how to assert we're going down the fast path.

Otherwise, we could work on optimizing the pure Python PNG decoder.

Raising back to P1. Fixing this is not as simple as adding a warning.

Note: "pip install cv2" on Mac OS X does not install Python's OpenCV bindings. It installs something else which can't be imported. http://stackoverflow.com/questions/34853220/cannot-import-cv2-in-python-in-osx indicates that it actually installs https://pypi.python.org/pypi/cv2/1.0 .

After installing Homebrew and OpenCV via it in http://www.mobileway.net/2015/02/14/install-opencv-for-python-on-mac-os-x/ , and manually setting PYTHONPATH to:

/usr/local/Cellar/opencv/2.4.12_2/lib/python2.7/site-packages/:/usr/local/Cellar/numpy/1.11.0/lib/python2.7/site-packages

"import numpy" and "import cv2" both work, and it's possible to call numpy.asarray and cv2.imdecode (per code in src/third_party/catapult/telemetry/telemetry/internal/image_processing/image_util_numpy_impl.py). However, test_png.py still takes the slow path.

If we can create a hermetic copy of numpy and cv2 and bundle them along with Telemetry, that would be an acceptable solution.

After Googling around how to remove the bad version of numpy (need disabling System Integrity Protection) & reinstall it on my Mac,  I can run the fast path on my Mac:
python test_png.py
Base64 decoding took: 0 ms
PNG decoding took: 35 ms
...
The version of numpy & cv2 I am using are:
>> import numpy
>>> numpy.__version__
'1.11.0'
>>> import cv2
>>> cv2.__version__
'2.4.12'

I will try to see if we can bundle the numpy & cv2 binary.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2462b5ca0d3392655b7ac4304e6481cb3bd8a600

commit 2462b5ca0d3392655b7ac4304e6481cb3bd8a600
Author: catapult-deps-roller <catapult-deps-roller@chromium.org>
Date: Fri Apr 22 17:46:39 2016

Roll src/third_party/catapult/ a1d076c6f..db9bc94f5 (3 commits).

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/a1d076c6fbbd..db9bc94f5f92

$ git log a1d076c6f..db9bc94f5 --date=short --no-merges --format='%ad %ae %s'

BUG= 605739 

TBR=catapult-sheriff@chromium.org

Review URL: https://codereview.chromium.org/1914553002

Cr-Commit-Position: refs/heads/master@{#389155}

[modify] https://crrev.com/2462b5ca0d3392655b7ac4304e6481cb3bd8a600/DEPS

I found the way to bundle numpy & cv2 as standalone binaries. I just sent the zip files of those to Ken & verify he can run the fast path to debug  Issue 605699 . 

We can now deprioritize this bug to P2. 

Probably the next step for this bug is package those zips of numpy & cv2 to telemetry's binary manager.

Thanks Ned for your help. Please do work on getting these binaries into cloud storage. After that, let's work on bundling them into the Telemetry testing isolates for the bots.

Thinking about this more, this is actually blocked on issue 591173. We need a script that can download all telemetry's dependency & that step should be executed before we run telemetry.

The reason is that in this case, the dependency files are python libs & will be loaded by telemetry during import time. So either the binary downloading should happen during import time (which seems super awful), or it should happen before someone invokes a telemetry script at all.

Yes, that would be super awful. Please use a script to download them.

I'm sorry for dropping this on the floor. I can confirm that with this diff (also attached):

diff --git a/telemetry/telemetry/__init__.py b/telemetry/telemetry/__init__.py
index 220f76d..8c20366 100644
--- a/telemetry/telemetry/__init__.py
+++ b/telemetry/telemetry/__init__.py
@@ -47,9 +47,11 @@ _AddDirToPythonPath(util.GetCatapultThirdPartyDir(), 'typ')
 
 # Add Telemetry third party dependencies into our path.
 _AddDirToPythonPath(util.GetTelemetryThirdPartyDir(), 'altgraph')
+_AddDirToPythonPath(util.GetTelemetryThirdPartyDir(), 'cv2')
 _AddDirToPythonPath(util.GetTelemetryThirdPartyDir(), 'mock')
 _AddDirToPythonPath(util.GetTelemetryThirdPartyDir(), 'modulegraph')
 _AddDirToPythonPath(util.GetTelemetryThirdPartyDir(), 'mox3')
+_AddDirToPythonPath(util.GetTelemetryThirdPartyDir(), 'numpy')
 _AddDirToPythonPath(util.GetTelemetryThirdPartyDir(), 'pexpect')
 _AddDirToPythonPath(util.GetTelemetryThirdPartyDir(), 'png')
 _AddDirToPythonPath(util.GetTelemetryThirdPartyDir(), 'pyfakefs')

that the hermetic copies of numpy and cv2 that Ned prepared work on Mac OS X.

Could we please figure out how to proceed so that these can be bundled? From my standpoint it would be fine if they were checked in, even if it meant that the search path had to be extended to only conditionally include them on Mac OS X.

I'm raising this to P1 as it's blocking investigation of bugs in the screenshot code path in Telemetry ( Issue 599776 ,  Issue 614394 ).

Deleted: diff.txt 907 bytes

diff.txt 907 bytes View Download

Ken: I have hard time getting consensus from other catapult owners to check in those big files to catapult github repo. An option for us to moving forward on this is check those into gpu/third_party/ & you add the python path in gpu test scripts with a TODO that we will do the proper way later. 

Ping. Should this be Pri-1? If so please assign a milestone.

We've reached consensus to do this through Telemetry's binary dependency manager and DEPS hooks during gclient sync. Duplicating into that bug.