Review rasterize_and_record_micro.top_25 failures on wide range of builders |
||||||||||||||||
Issue descriptionrasterize_and_record_micro.top_25 failing on 6 builders Builders failed on: - Android Nexus6 Perf: https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus6%20Perf - Mac 10.11 Perf: https://build.chromium.org/p/chromium.perf/builders/Mac%2010.11%20Perf - Mac 10.12 Perf: https://build.chromium.org/p/chromium.perf/builders/Mac%2010.12%20Perf - Mac Air 10.11 Perf: https://build.chromium.org/p/chromium.perf/builders/Mac%20Air%2010.11%20Perf - Mac Retina Perf: https://build.chromium.org/p/chromium.perf/builders/Mac%20Retina%20Perf - Win 10 High-DPI Perf: https://build.chromium.org/p/chromium.perf/builders/Win%2010%20High-DPI%20Perf
,
Aug 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2b6d05405d1327415a01af0e952ea11d9bbee834 commit 2b6d05405d1327415a01af0e952ea11d9bbee834 Author: Simon <simonhatch@chromium.org> Date: Wed Aug 16 20:44:17 2017 Disable stories on rasterize_and_record_micro.top_25 on Mac. TBR=rnephew@chromium.org Bug: 756117 Change-Id: I1bf110d2d961e94ead0e48f2c834fb6c3e40ab8f Reviewed-on: https://chromium-review.googlesource.com/617585 Reviewed-by: Simon Hatch <simonhatch@chromium.org> Reviewed-by: rnephew <rnephew@chromium.org> Commit-Queue: Simon Hatch <simonhatch@chromium.org> Cr-Commit-Position: refs/heads/master@{#494935} [modify] https://crrev.com/2b6d05405d1327415a01af0e952ea11d9bbee834/tools/perf/benchmarks/rasterize_and_record_micro.py
,
Aug 16 2017
Picking up various likely related rasterize_and_record_micro benchmark failure issues for bulk investigation soon.
,
Aug 21 2017
Standardising priority and component for this batch of rasterize_and_record_micro bugs filed to track intended-to-be-temporary disabling.
,
Aug 21 2017
Issue 747021 has been merged into this issue. Issue 752569 has been merged into this issue. Issue 753161 has been merged into this issue. Issue 755577 has been merged into this issue.
,
Aug 21 2017
,
Aug 25 2017
Issue 758753 has been merged into this issue.
,
Aug 25 2017
,
Sep 1 2017
,
Sep 1 2017
,
Sep 1 2017
,
Sep 1 2017
Running benchmark locally on Linux yesterday-ToT 17 stories pass, 8 fail. I've created blocking bugs on this issue for the failures. [ PASSED ] 17 tests. [ FAILED ] 8 tests, listed below: [ FAILED ] https://www.google.com/calendar/ [ FAILED ] https://www.google.com/search?q=cats&tbm=isch [ FAILED ] Docs (1 open document tab) [ FAILED ] https://plus.google.com/110031535020051778989/posts [ FAILED ] http://www.youtube.com [ FAILED ] Blogger [ FAILED ] Wordpress [ FAILED ] http://answers.yahoo.com 8 FAILED TESTS The perf waterfall is currently filled with red and purple: https://build.chromium.org/p/chromium.perf/waterfall?show_events=true&failures_only=true&reload=120 making it hard to survey the current state of where the benchmark is running successfully vs not. I have a few local platforms I can try but the full set in the waterfall is quite long. In meantime an obvious candidate to explore more closely is the Google login issue 747021 .
,
Sep 1 2017
,
Sep 1 2017
,
Sep 2 2017
,
Sep 2 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e093016b4d1d7291575365277c3105ae165fe75e commit e093016b4d1d7291575365277c3105ae165fe75e Author: Walter Korman <wkorman@chromium.org> Date: Sat Sep 02 00:50:28 2017 Update Top Pages story login logic for Google and Facebook. Replace deprecated broken login logic for Google stories with the new supported login helpers, following: https://chromium-review.googlesource.com/c/chromium/src/+/546615 Switch Facebook story to remove login. We have three Facebook test accounts. The first two both hit a wall at login asking for birthdate confirmation. The last one is the "recommended, approved" Facebook test account, but it also is known to be unable to view actual page content for some not-planned-to-be-changed reason. Since the whole point of the existing FB story was to view Barack Obama's FB page content, my design decision is to cease logging in and retrieve that page content though now we are no longer testing the normal set of FB logged-in-user functionality. Between dreams and reality, can there be a middle ground? Only time will tell. Bug: 756117 , 747021 Change-Id: Ia75e3fe1d5d89dac28ceb17cf9d4b495fa8ba70c Reviewed-on: https://chromium-review.googlesource.com/648252 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Walter Korman <wkorman@chromium.org> Cr-Commit-Position: refs/heads/master@{#499357} [modify] https://crrev.com/e093016b4d1d7291575365277c3105ae165fe75e/tools/perf/page_sets/top_pages.py
,
Sep 2 2017
,
Sep 2 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/71d0b6e55036e66e74dde48b1c7d5364eb1e5b1d commit 71d0b6e55036e66e74dde48b1c7d5364eb1e5b1d Author: Walter Korman <wkorman@chromium.org> Date: Sat Sep 02 15:02:07 2017 New web page recordings for various Top 25 pages. Re-record Top 25 page set to incorporate updated versions of broken pages, in particular: Google sites that were failing login and/or hitting DCHECK: - Calendar (recording here is still broken, see http://crbug.com/761572 for more) - Image Search - Docs - Plus - YouTube - Gmail Facebook Barack Obama page was failing login. Fixing to use new login method produces a page with no content since the test account can't view page content for some known and expected reason. So I changed Facebook story to act as logged-out which can at least see the content though it's perhaps less representative of a real user. Blogger and WordPress story recordings are still broken here, see http://crbug.com/761440 . http://games.yahoo.com looks like it's recording a redirect to what looks like plain Yahoo News or home page. Harmless but perhaps redundant. Other changes to follow shortly to fix login logic for Google and Facebook sites and DCHECK issues properly, but having updated recordings in meantime seems worthwhile despite hackery required to accomplish. Steps taken to re-record: - comment out DCHECK in LayoutTableBoxComponent.cpp:45 http://crbug.com/761434 - comment out DisableStory for mail.google.com and CNN in tools/perf/benchmarks/rasterize_and_record_micro.py - comment out DCHECK in rasterize_and_record_benchmark.cc:99 http://crbug.com/761510 % tools/perf/record_wpr --story top25_page_set \ rasterize_and_record_micro_top25 % upload_to_google_storage.py --bucket=chrome-partner-telemetry \ tools/perf/page_sets/data/top_25_009.wprgo Bug: 756117 , 757601 Change-Id: I67f6226173bccdea8cdb2a377d023d1dc18438d6 Reviewed-on: https://chromium-review.googlesource.com/648349 Commit-Queue: Walter Korman <wkorman@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#499396} [modify] https://crrev.com/71d0b6e55036e66e74dde48b1c7d5364eb1e5b1d/tools/perf/page_sets/data/top_25.json [add] https://crrev.com/71d0b6e55036e66e74dde48b1c7d5364eb1e5b1d/tools/perf/page_sets/data/top_25_009.wprgo.sha1
,
Sep 5 2017
,
Sep 7 2017
Looks like something is wrong with the page set? There's a list of failures starting around the time it got updated, with WebPageReplay stating it can't gunzip the files:
(INFO) 2017-09-07 08:15:30,811 cmd_helper._ValidateAndLogCommand:160 [host]> /b/swarming/w/ir/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s ZX1G22L7VG shell '( /system/bin/screencap -p /data/local/tmp/temp_file-914c480972aa7.png );echo %$?'
(INFO) 2017-09-07 08:15:38,402 cmd_helper._ValidateAndLogCommand:160 [host]> /b/swarming/w/ir/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s ZX1G22L7VG pull /data/local/tmp/temp_file-914c480972aa7.png /b/swarming/w/it0Ryrbg/tmpra0aDY.png
(INFO) 2017-09-07 08:15:39,267 cmd_helper._ValidateAndLogCommand:160 [host]> /b/swarming/w/ir/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s ZX1G22L7VG shell 'rm -f /data/local/tmp/temp_file-914c480972aa7.png'
Traceback (most recent call last):
File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 98, in _RunStoryAndProcessErrorIfNeeded
state.WillRunStory(story)
File "/b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
return func(*args, **kwargs)
File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 241, in WillRunStory
archive_path, page.make_javascript_deterministic)
File "/b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
return func(*args, **kwargs)
File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/core/network_controller.py", line 42, in StartReplay
make_javascript_deterministic)
File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/platform/network_controller_backend.py", line 193, in StartReplay
local_ports = self._StartReplayServer()
File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/platform/network_controller_backend.py", line 227, in _StartReplayServer
return self._wpr_server.StartServer()
File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/util/webpagereplay_go_server.py", line 283, in StartServer
''.join(self._LogLines()))
ReplayNotStartedError: Web Page Replay failed to start. Log output:
2017/09/07 08:14:56 Loading cert from /b/swarming/w/ir/third_party/catapult/web_page_replay_go/wpr_cert.pem
2017/09/07 08:14:56 Loading key from /b/swarming/w/ir/third_party/catapult/web_page_replay_go/wpr_key.pem
2017/09/07 08:14:56 Loading script from /b/swarming/w/ir/third_party/catapult/web_page_replay_go/deterministic.js
Error openining archive file: gunzip failed: gzip: invalid header
,
Sep 7 2017
Walter is migrating these pages to static HTML files, so I think we can mark this obsolete.
,
Sep 7 2017
Is there a tracking bug for that, in the meantime? A current mass of failures linked to a WontFix bug looks odd.
,
Sep 7 2017
issue 762230 He just landed the CL that switch the pages over in https://chromium-review.googlesource.com/c/chromium/src/+/653999
,
Sep 14 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/82102d061fcc284e520803bad8670ed5ba939990 commit 82102d061fcc284e520803bad8670ed5ba939990 Author: Charlie Andrews <charliea@chromium.org> Date: Thu Sep 14 23:19:27 2017 Update top_25 WPR recordings I'm fairly sure that they should have been updated in http://crrev.com/c/648252, but it was overlooked. Bug: 756117 , 747021 , 765383 Change-Id: I4ebb7a5cbaf9fa52c99ba9f05d990ca639f3d3e7 Reviewed-on: https://chromium-review.googlesource.com/667250 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Charlie Andrews <charliea@chromium.org> Cr-Commit-Position: refs/heads/master@{#502092} [modify] https://crrev.com/82102d061fcc284e520803bad8670ed5ba939990/tools/perf/page_sets/data/top_25.json [add] https://crrev.com/82102d061fcc284e520803bad8670ed5ba939990/tools/perf/page_sets/data/top_25_011.wprgo.sha1 |
||||||||||||||||
►
Sign in to add a comment |
||||||||||||||||
Comment 1 by simonhatch@chromium.org
, Aug 16 2017Not sure if the android/win ones are related, but SoM just groups them all together. Mac seems to be failing with this in the logs: (INFO) 2017-08-16 04:53:22,516 browser.DumpStateUponFailure:388 *************** BROWSER STANDARD OUTPUT *************** (INFO) 2017-08-16 04:53:22,517 browser.DumpStateUponFailure:390 DevTools listening on ws://127.0.0.1:52681/devtools/browser/7cb501df-75a8-463f-a342-8175396991dc 2017-08-16 04:51:01.339 Google Chrome[1105:10541] Errors logged by ksadmin: KSKeyedPersistentStore store directory does not exist. [com.google.UpdateEngine.CommonErrorDomain:501 - '/Library/Google/GoogleSoftwareUpdate/TicketStore' - 'KSKeyedPersistentStore.m:372'] KSPersistentTicketStore failed to load tickets. (productID: com.google.Chrome) [com.google.UpdateEngine.CoreErrorDomain:1051 - '/Library/Google/GoogleSoftwareUpdate/TicketStore/Keystone.ticketstore'] (KSKeyedPersistentStore store directory does not exist. - '/Library/Google/GoogleSoftwareUpdate/TicketStore' [com.google.UpdateEngine.CommonErrorDomain:501]) ksadmin cannot access the ticket store:<KSUpdateError:0x1019052a0 domain="com.google.UpdateEngine.CoreErrorDomain" code=1051 userInfo={ function = "-[KSProductKeyedStore(ProtectedMethods) errorForStoreError:productID:message:timeoutMessage:]"; date = 2017-08-16 11:51:01 +0000; productids = {( "com.google.Chrome" )}; filename = "KSProductKeyedStore.m"; line = 91; NSFilePath = "/Library/Google/GoogleSoftwareUpdate/TicketStore/Keystone.ticketstore"; NSUnderlyingError = <KSError:0x101903fd0 domain="com.google.UpdateEngine.CommonErrorDomain" code=501 userInfo={ date = 2017-08-16 11:51:01 +0000; line = 372; filename = "KSKeyedPersistentStore.m"; function = "-[KSKeyedPersistentStore(PrivateMethods) validateStorePath]"; NSFilePath = "/Library/Google/GoogleSoftwareUpdate/TicketStore"; NSLocalizedDescription = "KSKeyedPersistentStore store directory does not exist."; } >; NSLocalizedDescription = "KSPersistentTicketStore failed to load tickets."; } > [1109:775:0816/045105.335564:ERROR:gpu_command_buffer_stub.cc(950)] Got WaitForGetOffset command while currently waiting for offset. (INFO) 2017-08-16 04:53:22,517 browser.DumpStateUponFailure:393 *********** END OF BROWSER STANDARD OUTPUT ************ (INFO) 2017-08-16 04:53:22,517 browser.DumpStateUponFailure:395 ********************* BROWSER LOG ********************* (INFO) 2017-08-16 04:53:22,517 browser.DumpStateUponFailure:397 No log file (INFO) 2017-08-16 04:53:22,517 browser.DumpStateUponFailure:400 ***************** END OF BROWSER LOG ****************** Traceback (most recent call last): File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 105, in _RunStoryAndProcessErrorIfNeeded state.RunStory(results) File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 326, in RunStory self._current_page, self._current_tab, results) File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/tools/perf/measurements/rasterize_and_record_micro.py", line 63, in ValidateAndMeasurePage 'window.benchmark_results.done', timeout=self._timeout) File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 192, in WaitForJavaScriptCondition return self._inspector_backend.WaitForJavaScriptCondition(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 287, in WaitForJavaScriptCondition e.message + '\n' + debug_message) TimeoutException: Timed out while waiting 120s for IsJavaScriptExpressionTrue. Console output: (warning) http://l.yimg.com/zz/combo?yui:3.5.1/build/yui/yui-min.js:12: yui: NOT loaded: media-orientation (warning) http://l.yimg.com/zz/combo?yui:3.5.1/build/yui/yui-min.js:12: yui: NOT loaded: media-orientation (warning) http://l.yimg.com/zz/combo?yui:3.5.1/build/yui/yui-min.js:12: yui: NOT loaded: media-orientation (info) http://l.yimg.com/zz/combo?yui:3.5.1/build/yui/yui-min.js:12: SocialChromeBase: scbase:fb_status_not_authorized fired (info) http://l.yimg.com/zz/combo?yui:3.5.1/build/yui/yui-min.js:12: SocialChromeLogin: switched logout state (info) http://l.yimg.com/zz/combo?yui:3.5.1/build/yui/yui-min.js:12: SocialChromeFriendsList: switched logout state (warning) http://l.yimg.com/zz/combo?yui:3.5.1/build/yui/yui-min.js:12: yui: NOT loaded: media-location-widget (warning) http://l.yimg.com/zz/combo?yui:3.5.1/build/yui/yui-min.js:12: yui: NOT loaded: media-location-widget (INFO) 2017-08-16 04:53:22,626 browser.Close:268 Closing browser (pid=1105) ... (INFO) 2017-08-16 04:53:22,744 browser.Close:283 Browser is closed. [ FAILED ] http://news.yahoo.com (141851 ms)