Issue metadata
Sign in to add a comment
|
8.1% regression in smoothness.gpu_rasterization.tough_filters_cases at 458189:458250 |
||||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Mar 21 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8984485213283017472
,
Mar 21 2017
=== Auto-CCing suspected CL author zmo@chromium.org === Hi zmo@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : zmo Commit : 9b5537f92d5910a563811ce14208d7cb8b45b377 Date : Mon Mar 20 23:01:56 2017 Subject: Expose GL_VENDOR/GL_RENDERER strings to internal clients of GPU command buffer. Bisect Details Configuration: android_nexus5_perf_bisect Benchmark : smoothness.gpu_rasterization.tough_filters_cases Metric : frame_times/http___web.archive.org_web_20150502135732_http___ie.microsoft.com_testdrive_Performance_Pirates_Default.html Change : 8.48% | 40.4461386579 -> 43.8741455086 Revision Result N chromium@458188 40.4461 +- 0.45366 6 good chromium@458219 40.6049 +- 0.495397 6 good chromium@458220 43.8002 +- 0.434328 6 bad <-- chromium@458221 43.8722 +- 0.287438 6 bad chromium@458223 43.9151 +- 0.662745 6 bad chromium@458227 43.8717 +- 0.608588 6 bad chromium@458235 43.9402 +- 0.170957 6 bad chromium@458250 43.8741 +- 0.544586 6 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...web.archive.org.web.20150502135732.http...ie.microsoft.com.testdrive.Performance.Pirates.Default.html smoothness.gpu_rasterization.tough_filters_cases Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8984485213283017472 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=6177132989906944 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Mar 21 2017
Bian, can you take a look at this? Assuming skia going down a different path than before, which you said it shouldn't?
,
Mar 21 2017
Yep, I will investigate.
,
Mar 22 2017
I'm having trouble running the perf test. I run tools/perf/run_benchmark -v --browser=android-chromium --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...web.archive.org.web.20150502135732.http...ie.microsoft.com.testdrive.Performance.Pirates.Default.html smoothness.gpu_rasterization.tough_filters_cases The web.archive.org page never finishes loading on the device. The tail of the output looks like: INFO:devil.utils.cmd_helper:[host]> /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb -s 094cda84 shell '( dumpsys SurfaceFlinger --latency SurfaceView );echo %$?' WARNING:root:Exception was raised in the with statement block, the end of interaction record is not marked. INFO:root:*************** BROWSER STANDARD OUTPUT *************** INFO:root:Cannot get standard output on Android INFO:root:*********** END OF BROWSER STANDARD OUTPUT ************ INFO:root:********************* BROWSER LOG ********************* INFO:root:No log file INFO:root:***************** END OF BROWSER LOG ****************** INFO:devil.utils.cmd_helper:[host]> /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb -s 094cda84 shell '( /system/bin/screencap -p /data/local/tmp/temp_file-eec5d74ca14ce.png );echo %$?' INFO:devil.utils.cmd_helper:[host]> /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb -s 094cda84 shell '( dumpsys SurfaceFlinger --latency SurfaceView );echo %$?' INFO:devil.utils.cmd_helper:[host]> /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb -s 094cda84 pull /data/local/tmp/temp_file-eec5d74ca14ce.png /tmp/tmpu8KF9U.png INFO:devil.utils.cmd_helper:[host]> /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb -s 094cda84 shell 'rm -f /data/local/tmp/temp_file-eec5d74ca14ce.png' Traceback (most recent call last): File "/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 88, in _RunStoryAndProcessErrorIfNeeded state.RunStory(results) File "/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 299, in RunStory self._current_page.Run(self) File "/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/telemetry/telemetry/page/__init__.py", line 112, in Run self.RunPageInteractions(action_runner) File "/usr/local/google/home/bsalomon/src/clank/src/tools/perf/page_sets/tough_filters_cases.py", line 27, in RunPageInteractions 'document.getElementById("benchmarkButtonText").click()') File "/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function return func(*args, **kwargs) File "/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py", line 236, in EvaluateJavaScript return self._tab.EvaluateJavaScript(*args, **kwargs) File "/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function return func(*args, **kwargs) File "/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 167, in EvaluateJavaScript return self._inspector_backend.EvaluateJavaScript(*args, **kwargs) File "/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function return func(*args, **kwargs) File "/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 37, in inner return func(inspector_backend, *args, **kwargs) File "/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 244, in EvaluateJavaScript return self._runtime.Evaluate(expression, context_id, timeout) File "/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_runtime.py", line 54, in Evaluate description=details.get('exception', {}).get('description')) EvaluateException: UncaughtTypeError: TypeError: Cannot read property 'click' of null at <anonymous>:1:47 INFO:root:Chrome version: 3047 INFO:devil.utils.cmd_helper:[host]> /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb -s 094cda84 shell '( rm -f /data/local/chrome-trace-config.json );echo %$?' INFO:devil.utils.cmd_helper:[host]> /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb -s 094cda84 shell '( dumpsys SurfaceFlinger --latency SurfaceView );echo %$?' INFO:devil.utils.cmd_helper:[host]> /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb -s 094cda84 shell '( dumpsys SurfaceFlinger --latency SurfaceView );echo %$?' INFO:devil.utils.cmd_helper:[host]> /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb -s 094cda84 shell '( ps | grep -F surfaceflinger; echo "PIPESTATUS: ${PIPESTATUS[@]}" );echo %$?' INFO:devil.utils.cmd_helper:[host]> /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb -s 094cda84 shell '( log -p i -t Telemetry '"'"'END http://web.archive.org/web/20150502135732/http://ie.microsoft.com/testdrive/Performance/Pirates/Default.html'"'"' );echo %$?' INFO:devil.utils.cmd_helper:[host]> /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb -s 094cda84 shell '( ps | grep -F org.chromium.chrome; echo "PIPESTATUS: ${PIPESTATUS[@]}" );echo %$?' INFO:devil.utils.cmd_helper:[host]> /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb -s 094cda84 shell '( ls /root );echo %$?' INFO:devil.utils.cmd_helper:[host]> /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb -s 094cda84 shell '( cd /sys/devices/system/cpu && for CPU in cpu0 cpu1 cpu2 cpu3; do test -e "${CPU}/cpufreq/scaling_available_governors" && grep -q interactive ${CPU}/cpufreq/scaling_available_governors && test -e "$CPU/cpufreq/scaling_governor" && echo interactive > "$CPU/cpufreq/scaling_governor"; echo -n "%~%$?%~%"; done );echo %$?' INFO:devil.android.perf.perf_control:Successfully set cpufreq/scaling_governor to 'interactive' on: cpu0 cpu1 INFO:devil.utils.cmd_helper:[host]> /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb -s 094cda84 shell '( start mpdecision );echo %$?' INFO:devil.utils.cmd_helper:[host]> /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb -s 094cda84 shell '( p=org.chromium.chrome;if [[ "$(ps)" = *$p* ]]; then am force-stop $p; fi );echo %$?' INFO:devil.utils.cmd_helper:[host]> /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb -s 094cda84 forward --remove tcp:43368 INFO:devil.utils.cmd_helper:[host]> /usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/devil/bin/deps/linux2/x86_64/forwarder_host --adb=/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb --serial-id=094cda84 --unmap 35841 ERROR:devil.android.forwarder:`/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/devil/bin/deps/linux2/x86_64/forwarder_host --adb=/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb --serial-id=094cda84 --unmap 35841` exited with 1: [0322/115842:ERROR:host_forwarder_main.cc(477)] ERROR: could not unmap port. ERROR: Existing controllers: [ FAILED ] http://web.archive.org/web/20150502135732/http://ie.microsoft.com/testdrive/Performance/Pirates/Default.html (45405 ms) [ PASSED ] 0 tests. [ FAILED ] 1 test, listed below: [ FAILED ] http://web.archive.org/web/20150502135732/http://ie.microsoft.com/testdrive/Performance/Pirates/Default.html 1 FAILED TEST WARNING:root:Failed to extract previous results from HTML output View result at file:///usr/local/google/home/bsalomon/src/clank/src/tools/perf/results.html INFO:devil.utils.cmd_helper:[host]> /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb -s 094cda84 shell '( dumpsys battery );echo %$?' INFO:root:Charging status checked at exit.
,
Mar 22 2017
Ken, do you have any tips for how to run the perf tests? I've encountered this same issue with run_benchmark when working on other perf regressions over the past N > 6 months and have never figured it out. In the past I've always found some other way to reproduce the problem. In this case I've tried just running the Pirate Marrrk benchmark from archive.org manually but I'm not seeing a consistent difference in the benchmark score. This probably just means the benchmark is finishing in about the same time but with fewer frames rendered with the change.
,
Mar 22 2017
I had been using a N7v2 thinking it would be close enough. I got my hands on a N5 and now on the device it never attempts to load the page. It stays on about:blank. The N7 I was using was running Android 5.1.1. The N5 is running 6.0.1.
,
Mar 22 2017
Sorry, I'm not an expert on the run_benchmark harness. (To be honest, the last time I investigated a perf regression I also wasn't able to reproduce the problem locally.) Annie or Ned: can you help Brian figure out why this benchmark doesn't run correctly locally? Thanks.
,
Mar 22 2017
Also CC'ing nduca for help.
,
Mar 22 2017
Can you try to follow go/telemetry-device-setup to see if that fix the issues you are having?
,
Mar 23 2017
I followed the instructions, including the provisioning script, but I get the same behavior. The site never fully loads when run run_benchmark. I attached a log of run_benchmark.
,
Mar 23 2017
Are you running this on Mac or Linux workstation as the host? Note that we only support Linux as the host.
,
Mar 23 2017
Hmhh, actually from the log, it seems like the step of installing the fake certificate was failing:
Traceback (most recent call last):
File "/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/telemetry/telemetry/internal/platform/network_controller_backend.py", line 141, in _InstallTestCa
self._platform_backend.InstallTestCa(self._wpr_ca_cert_path)
File "/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/telemetry/telemetry/internal/platform/android_platform_backend.py", line 558, in InstallTestCa
self._device_cert_util.install_cert(overwrite_cert=True)
File "/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/telemetry/third_party/web-page-replay/adb_install_cert.py", line 191, in install_cert
if self._is_cert_installed():
File "/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/telemetry/third_party/web-page-replay/adb_install_cert.py", line 151, in _is_cert_installed
return (self._adb_su_shell('ls', self.android_cacerts_path).strip() ==
File "/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/telemetry/third_party/web-page-replay/adb_install_cert.py", line 98, in _adb_su_shell
build_version_sdk = int(self._get_property('ro.build.version.sdk'))
ValueError: invalid literal for int() with base 10: "adb server version (32) doesn't match this client (36); killing...\n* daemon started successfully *\n22"
,
Mar 23 2017
In this case, because certificate installation failed, Telemetry has to launch the browser with "--ignore-certificate-error". @bsalomon: when you run the test locally, did you see the butter bar that warn about certificate error? If not, then this is probably another case of Telemetry failed to pass flags to Chrome Android.
,
Mar 24 2017
I have been using a Goobuntu workstation. I did not see a butter bar about a certificate error.
,
Mar 24 2017
,
Mar 24 2017
Actually, the problem seems to be the forwarder using a different version of adb than the rest of Telemetry. Most adb commands appear to be using: /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb But the line when setting the forwarder shows: INFO:devil.android.forwarder:Forwarding using commands: [[u'--adb=/usr/local/google/home/bsalomon/src/clank/src/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb', '--serial-id=018e0bb120c8b857', '--map', '0', '54044']] +jbudorick
,
Mar 24 2017
bsalomon: do you have an ANDROID_SDK_ROOT environment variable set? devil jumps through a few hoops to find the adb path to use for AdbWrapper: https://codesearch.chromium.org/chromium/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py?rcl=0d592b9399db95b8e8b935d45ad238a14aa4a12d&l=58 It looks for: - a configured adb that exists locally - platform-tools/adb within a configured android_sdk that exists locally - a configured adb that it has to download In your case, it appears to be using the second. Unfortunately, the forwarder *just* looks for a configured adb (https://codesearch.chromium.org/chromium/src/third_party/catapult/devil/devil/android/forwarder.py?rcl=0d592b9399db95b8e8b935d45ad238a14aa4a12d&l=133) rather than deferring to AdbWrapper.GetAdbPath. That's a bug; I'll send out a CL in a bit.
,
Mar 24 2017
I do. I have ANDROID_SDK_ROOT=/usr/local/google/home/bsalomon/android-sdk-linux and the adb there is on my path: bsalomon:~/src/clank/src ▶ which adb /usr/local/google/home/bsalomon/android-sdk-linux/platform-tools/adb
,
Mar 24 2017
yep, as expected. https://codereview.chromium.org/2775793002/ should make the forwarder use the same version of adb as everything else.
,
Mar 24 2017
I remove the environment var and took adb off my path and now the benchmark runs correctly. Thanks for your help with this! And I'm glad to hear that it'll be fixed.
,
Mar 24 2017
,
Mar 24 2017
not sure where bugdroid is, but that fix rolled into chromium in https://chromium.googlesource.com/chromium/src/+/7522fb00d86cf9436d2af703fe17a0c1471b706a
,
Mar 25 2017
The following revision refers to this bug: https://skia.googlesource.com/skia/+/1b52df33927c00d9c5aee63f7239dd77bce15e03 commit 1b52df33927c00d9c5aee63f7239dd77bce15e03 Author: Brian Salomon <bsalomon@google.com> Date: Sat Mar 25 01:12:53 2017 Remove draw-as-clear workaround for Adreno 3xx in Chrome. This caused a performance regression after Skia started seeing the real GL strings. BUG= chromium:703774 Change-Id: Ib0878062670b5ea51a005718a64a58d89fbaa42c Reviewed-on: https://skia-review.googlesource.com/10165 Reviewed-by: Brian Salomon <bsalomon@google.com> Commit-Queue: Brian Salomon <bsalomon@google.com> [modify] https://crrev.com/1b52df33927c00d9c5aee63f7239dd77bce15e03/src/gpu/gl/GrGLCaps.cpp
,
Mar 25 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/9f26840d0d0cb457306ee4e589011bb8ad97436e commit 9f26840d0d0cb457306ee4e589011bb8ad97436e Author: skia-deps-roller@chromium.org <skia-deps-roller@chromium.org> Date: Sat Mar 25 09:41:12 2017 Roll src/third_party/skia/ dd1b4e94c..1b52df339 (1 commit) https://skia.googlesource.com/skia.git/+log/dd1b4e94c429..1b52df33927c $ git log dd1b4e94c..1b52df339 --date=short --no-merges --format='%ad %ae %s' 2017-03-24 bsalomon Remove draw-as-clear workaround for Adreno 3xx in Chrome. Created with: roll-dep src/third_party/skia BUG= 703774 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, see: http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_trusty_blink_rel TBR=fmalita@chromium.org Change-Id: Id21c276163440a86050730e413391d6d81f8180c Reviewed-on: https://chromium-review.googlesource.com/459090 Reviewed-by: Skia Deps Roller <skia-deps-roller@chromium.org> Commit-Queue: Skia Deps Roller <skia-deps-roller@chromium.org> Cr-Commit-Position: refs/heads/master@{#459648} [modify] https://crrev.com/9f26840d0d0cb457306ee4e589011bb8ad97436e/DEPS
,
Mar 25 2017
There was an unrelated performance improvement and then after the Skia roll there was a additional improvement commensurate with the original regression.
,
Mar 25 2017
,
Mar 28 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b9ee54948a13bebf39f76a3cac615aa986beb3c4 commit b9ee54948a13bebf39f76a3cac615aa986beb3c4 Author: skia-deps-roller@chromium.org <skia-deps-roller@chromium.org> Date: Tue Mar 28 01:54:32 2017 Roll src/third_party/skia/ 61e990297..2084ffa9a (43 commits) https://skia.googlesource.com/skia.git/+log/61e990297a5c..2084ffa9a3fa $ git log 61e990297..2084ffa9a --date=short --no-merges --format='%ad %ae %s' 2017-03-27 bsalomon Revert "Fix SkImage_Gpu::getROPixels" 2017-03-23 benjaminwagner Omit ImageStorageLoad on IntelBayTrail Ubuntu 2017-03-27 egdaniel Minor fix to VkWrapTest 2017-03-27 mtklein ok: add basic unit test support 2017-03-27 csmartdalton Add a simple tool for isolating a path from an SKP 2017-03-27 mtklein quiet some annoying logging in unit tests 2017-03-27 jvanverth Add height function to shadow tessellators. 2017-03-27 herb Change SkMemory to the more accurately named SkMalloc. 2017-03-27 bsalomon Split GrPipelineInput into separate color and coverage types, the latter of which is just an enum. 2017-03-27 mtklein ok: alias 565 -> sw:ct=565 2017-03-27 robertphillips Remove GrFragmentProcessor-derived class' GrTexture-based ctors 2017-03-27 kjlubick Update ATT Galaxy S7 to Android N 2017-03-27 robertphillips Add new proxy-based DetermineDomainMode w/ test 2017-03-26 mtklein ok: handle concurrent crashes better 2017-03-27 bsalomon Fully initialize array of previous edges in GrGLConvexPolyEffect to please MSAN 2017-03-24 iannucci Manually roll recipes (trivial) 2017-03-24 iannucci Manually roll recipes 2017-03-24 iannucci Manually roll recipes (trivial) 2017-03-26 mtklein focus hidden-visibility flags 2017-03-24 iannucci Manually roll recipes 2017-03-24 iannucci Manually roll recipes (trivial) 2017-03-26 update-skps Update SKP version 2017-03-25 mtklein ok: refactor Src/Dst interactions 2017-03-25 mtklein ok: basic Vias 2017-03-25 mtklein Spin off non-core parts of ok into their own files. 2017-03-25 reed Revert "Revert "simplify api to bitmapcache"" 2017-03-24 bsalomon Remove draw-as-clear workaround for Adreno 3xx in Chrome. 2017-03-24 fmalita Revert "simplify api to bitmapcache" 2017-03-24 mtklein ok: introduce the idea of Dsts 2017-03-24 reed simplify api to bitmapcache 2017-03-24 benjaminwagner Omit tests causing IntelIris540 Vulkan to crash/fail. 2017-03-24 jvanverth Try a different calculation for computing convexity. 2017-03-24 brianosman Revert "Remove texture sampling from GrConfigConversionEffect" 2017-03-24 reed quick-fix to surface dox 2017-03-24 brianosman Fix SkImage_Gpu::getROPixels 2017-03-24 msarett Delete SkColorSpace::ColorSpaceFlags 2017-03-24 bsalomon Add function to perform color fragment processor elimination while performing FP analysis. 2017-03-24 egdaniel Update Linux Intel Release driver with patch to fix Vulkan 2017-03-24 bsalomon Run angle in 8 sample msaa on desktop 2017-03-24 jvanverth Disable convex check for now. 2017-03-24 bsalomon Revert "Revert "Create new inset algorithm for spot shadows"" 2017-03-24 bsalomon Revert "Create new inset algorithm for spot shadows" 2017-03-23 mtklein ok: fix an assert from unbalanced save/restores Created with: roll-dep src/third_party/skia BUG= 703774 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, see: http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_trusty_blink_rel TBR=stephana@chromium.org Change-Id: If7df898d4d3cb03d6d4511276a395d0e59677881 Reviewed-on: https://chromium-review.googlesource.com/461227 Reviewed-by: Skia Deps Roller <skia-deps-roller@chromium.org> Commit-Queue: Skia Deps Roller <skia-deps-roller@chromium.org> Cr-Commit-Position: refs/heads/master@{#459979} [modify] https://crrev.com/b9ee54948a13bebf39f76a3cac615aa986beb3c4/DEPS
,
Jan 16
,
Jan 16
|
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by tdres...@chromium.org
, Mar 21 2017