Issue metadata
Sign in to add a comment
|
Fix and reenable smoothnes.top_25_smooth on Win Perf bots |
||||||||||||||||||||||
Issue description
I'm still poking through revision ranges to see what changed. It may be a WPR failure with expired date/time?
Gmail page failing with this error:
C:\b\c\b\Win_10_Perf__3_\src\third_party\catapult\telemetry\telemetry\internal\image_processing\_bitmap.py:152: UserWarning: Using pure python png decoder, which could be very slow. To speed up, consider installing numpy & cv2 (OpenCV).
'Using pure python png decoder, which could be very slow. To speed up, '
Traceback (most recent call last):
File "C:\b\c\b\Win_10_Perf__3_\src\third_party\catapult\telemetry\telemetry\internal\story_runner.py", line 86, in _RunStoryAndProcessErrorIfNeeded
state.RunStory(results)
File "C:\b\c\b\Win_10_Perf__3_\src\third_party\catapult\common\py_trace_event\py_trace_event\trace_event_impl\decorators.py", line 52, in traced_function
return func(*args, **kwargs)
File "C:\b\c\b\Win_10_Perf__3_\src\third_party\catapult\telemetry\telemetry\page\shared_page_state.py", line 301, in RunStory
self._current_page.Run(self)
File "C:\b\c\b\Win_10_Perf__3_\src\third_party\catapult\telemetry\telemetry\page\__init__.py", line 99, in Run
self.RunPageInteractions(action_runner)
File "C:\b\c\b\Win_10_Perf__3_\src\tools\perf\page_sets\top_25_smooth.py", line 69, in RunPageInteractions
'window.__scrollableElementForTelemetry != null')
File "C:\b\c\b\Win_10_Perf__3_\src\third_party\catapult\telemetry\telemetry\internal\actions\action_runner.py", line 220, in WaitForJavaScriptCondition
self._tab.WaitForJavaScriptExpression(condition, timeout_in_seconds)
File "C:\b\c\b\Win_10_Perf__3_\src\third_party\catapult\telemetry\telemetry\internal\browser\web_contents.py", line 136, in WaitForJavaScriptExpression
e.message + '\n' + debug_message)
TimeoutException: Timed out while waiting 60s for IsJavaScriptExpressionTrue.
Console output:
INFO:root:Chrome version: 2868
[ FAILED ] https://mail.google.com/mail/ (79140 ms)
https://build.chromium.org/p/chromium.perf/builders/Win%2010%20Perf%20%283%29/builds/4062/steps/smoothness.top_25_smooth/logs/stdio
Screenshot: https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/profiler-file-id_1-2016-09-21_22-09-3576161.png
Link to failing step log:
https://build.chromium.org/p/chromium.perf/builders/Win%2010%20Perf%20%283%29/builds/4062/steps/smoothness.top_25_smooth/logs/stdio
https://build.chromium.org/p/chromium.perf/builders/Win%2010%20Perf%20%283%29?numbuilds=200
https://build.chromium.org/p/chromium.perf/builders/Win%2010%20Perf%20%283%29/builds/4062
https://build.chromium.org/p/chromium.perf/builders/Win%208%20Perf%20%283%29
If the test is disabled, please downgrade to Pri-2.
,
Sep 27 2016
I started digging into this. Here are all the charts that are no longer getting data: https://chromeperf.appspot.com/report?sid=f71a05120543ccf666b46aec319e0f63062727717a073ac6bdae838f72e97e11 They all stop at different times, revisions: chromium-rel-win7-gpu-nvidia: 2016-09-21T23:08:48, 420107-420168 chromium-rel-win7-gpu-ati: 2016-09-21T23:11:37, 420107-420168 chromium-rel-win7-x64-dual: 2016-09-25T16:28:58, 420860-420862 chromium-rel-win10: 2016-09-26T11:03:25, 420876-420883 chromium-rel-win7-gpu-intel: 2016-09-26T16:51:46, 420888-420922 chromium-rel-win7-dual: 2016-09-27T15:14:51, 421180-421229 I'm not sure if there are different failures here. I'm going to try kicking off return code bisects on some of the different breakages.
,
Sep 27 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9000329613919973056
,
Sep 27 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9000329574983967504
,
Sep 27 2016
Ned, any ideas what could be going on here? The revision ranges and timings don't overlap.
,
Sep 27 2016
No idea. I think we need to run this locally to debug.
,
Sep 27 2016
===== BISECT JOB RESULTS ===== Status: failed ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@420107 16.733 0.0 1 good Bisect job ran on: winx64ati_perf_bisect Bug ID: 650762 Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.top_25_smooth Test Metric: mean_frame_time/https___mail.google.com_mail_ Relative Change: None Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64ati_perf_bisect/builds/1575 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000329574983967504 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=6451222150643712 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Sep 28 2016
===== BISECT JOB RESULTS ===== Status: completed ===== TESTED REVISIONS ===== Revision Exit Code Std Dev N Good? chromium@420107 1 N/A 20 good chromium@420169 1 N/A 20 bad Bisect job ran on: winx64nvidia_perf_bisect Bug ID: 650762 Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.top_25_smooth Test Metric: mean_frame_time/https___mail.google.com_mail_ Relative Change: 0.00% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64nvidia_perf_bisect/builds/1880 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000329613919973056 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=6383418844119040 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Sep 28 2016
I accidentally did a regression bisect instead of a return-code bisect in #7, but it looks like the test is flaky: for r 420107 it passed once then failed 4 times. https://build.chromium.org/p/tryserver.chromium.perf/builders/winx64ati_perf_bisect/builds/1575
,
Sep 28 2016
Ned, I want to debug this on a windows laptop on canary. What's the fastest way to get telemetry and run the test (and possibly re-record the page)? Is there still a zip file of telemetry ToT or do I need to sync all of Chrome source?
,
Sep 28 2016
cc vmiura, test owner. it looks like I won't be able to debug in a short time frame, so disabling this test on windows.
,
Sep 28 2016
Victor, would you prefer to disable the benchmark on windows, or the gmail page on all platforms? (those are the knobs we have for failing tests)
,
Sep 28 2016
Annie: to quickly grab a copy of "telemetry + perf benchmark + chromium build", you can pick any recent swarming task of telemetry_perf_unittest, then fetch the isolate. For example, here I pick one from chromium_rel_ng build: https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/301371 Click on any shard in telemetry_perf_unittest. Here I pick shard 0: https://chromium-swarm.appspot.com/user/task/318af5b940b27110 Then there is the instruction in the swarming page for downloading the isolate: "Download inputs files into directory foo: python isolateserver.py download -I https://isolateserver.appspot.com --namespace default-gzip -s a6e26ba2135597c3da8a33937995ae51b9364f4a --target foo" The isolateserver.py script can be found in https://github.com/luci/luci-py/blob/master/client/isolateserver.py
,
Sep 28 2016
If you disable on gmail, please cc the chromium perf-sheriff list since it will likely change the performance characteristics of the benchmark.
,
Sep 28 2016
,
Sep 28 2016
Issue 650066 has been merged into this issue.
,
Sep 28 2016
Issue 650038 has been merged into this issue.
,
Sep 28 2016
Issue 650039 has been merged into this issue.
,
Sep 28 2016
Issue 649978 has been merged into this issue.
,
Oct 3 2016
,
Oct 3 2016
Just noticed this is still a problem for many Windows bots. Reopen now.
,
Oct 7 2016
smoothness.top_25_smooth.reference is still failing. Should we disable it on reference build until it is fixed?
,
Oct 7 2016
Disable CL is up: https://codereview.chromium.org/2402693002/
,
Oct 8 2016
,
Nov 3 2016
,
Nov 17 2016
Issue 652731 has been merged into this issue.
,
Aug 4 2017
|
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by sullivan@chromium.org
, Sep 27 2016