Flakiness with scrolling benchmarks in telemetry. |
||||||||
Issue descriptionSometimes scrolling benchmarks time out waiting for _IsScrollingDone. See this log: https://build.chromium.org/p/tryserver.chromium.perf/builders/mac_retina_perf_cq/builds/222/steps/v8.top10_mobile_memory/logs/stdio
,
Jun 9 2016
I've just come across this on an Android perf bot: https://build.chromium.org/p/chromium.perf/builders/Android%20Galaxy%20S5%20Perf%20%283%29/builds/2282/steps/top_10_mobile_memory_ignition/logs/stdio The error message suggests that it's indeed due to redirection: EvaluateException: TypeError: window.__ScrollAction_SupportedByBrowser is not a function at <anonymous>:1:8 We've already across similar issues in the past ( issue 540022 and issue 586156), which were fixed by changing google.com to google.co.uk in some page sets (https://codereview.chromium.org/1702463002 by skyostil). I suppose we could do the same here, but I think we'll need to change google.com to google.ca this time.
,
Jun 9 2016
The core fix here is we should switch from the chrome.gpuBenchmarking.smoothScrollBy to devtool APi for scrolling :-(
,
Jun 9 2016
Applying the redirect in the page should fix the problem for now, but I agree that longer term these injected scroll shenanigans need to be replaced with devtools commands.
,
Jun 21 2016
I suspect that the same issue occurs on Windows perf bots: Win 7 Intel GPU Perf (2): https://build.chromium.org/p/chromium.perf/builders/Win%207%20Intel%20GPU%20Perf%20%282%29/builds/4424/steps/top_10_mobile_memory/logs/stdio Win 10 Perf (2): https://build.chromium.org/p/chromium.perf/builders/Win%2010%20Perf%20%282%29/builds/4427/steps/top_10_mobile_memory/logs/stdio Win 10 Perf (5): https://build.chromium.org/p/chromium.perf/builders/Win%2010%20Perf%20%285%29/builds/3835/steps/top_10_mobile_memory_ignition/logs/stdio Mythri: Could you please look into this?
,
Jun 21 2016
,
Jun 23 2016
Friendly sheriff ping (According to the Perf Bot Sheriffing guide, Pri-1 build bot bugs should be pinged daily. Please close the issue if it's already been fixed.)
,
Jun 24 2016
Issue 622777 has been merged into this issue.
,
Jun 24 2016
,
Jun 24 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d0c95636111161400a576472427de8eeb2330d43 commit d0c95636111161400a576472427de8eeb2330d43 Author: mythria <mythria@chromium.org> Date: Fri Jun 24 14:58:11 2016 [perf] Change google.com to google.ca in top_10_mobile page set. In the top10_mobile page set google.com gets redirected to google.ca. This sometimes causes problems in telemetry with scrolling. As a short term fix, replace google.com with google.ca in the page set. BUG= chromium:617914 LOG=N CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq Review-Url: https://codereview.chromium.org/2092443002 Cr-Commit-Position: refs/heads/master@{#401863} [modify] https://crrev.com/d0c95636111161400a576472427de8eeb2330d43/tools/perf/page_sets/data/memory_top_10_mobile.json [modify] https://crrev.com/d0c95636111161400a576472427de8eeb2330d43/tools/perf/page_sets/data/top_10_mobile.json [modify] https://crrev.com/d0c95636111161400a576472427de8eeb2330d43/tools/perf/page_sets/text_selection_sites.py [modify] https://crrev.com/d0c95636111161400a576472427de8eeb2330d43/tools/perf/page_sets/top_10_mobile.py
,
Jun 29 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8e45aa437f44a72f971913dd1a3ceb20a6f672ad commit 8e45aa437f44a72f971913dd1a3ceb20a6f672ad Author: perezju <perezju@chromium.org> Date: Wed Jun 29 17:56:01 2016 Revert of [perf] Change google.com to google.ca in top_10_mobile page set. (patchset #2 id:20001 of https://codereview.chromium.org/2092443002/ ) Reason for revert: Caused artificial memory drop in memory.top_10_mobile benchmark. Sorry if this breaks or flakes other benchmarks. We can re-land (with re-recorded pages) when the dust settles after health plan for this milestone is completed. Original issue's description: > [perf] Change google.com to google.ca in top_10_mobile page set. > > In the top10_mobile page set google.com gets redirected to google.ca. > This sometimes causes problems in telemetry with scrolling. As a short term > fix, replace google.com with google.ca in the page set. > > BUG= chromium:617914 > LOG=N > CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq > > Committed: https://crrev.com/d0c95636111161400a576472427de8eeb2330d43 > Cr-Commit-Position: refs/heads/master@{#401863} TBR=petrcermak@chromium.org,nednguyen@google.com,mythria@chromium.org # Not skipping CQ checks because original CL landed more than 1 days ago. BUG= chromium:617914 Review-Url: https://codereview.chromium.org/2112583002 Cr-Commit-Position: refs/heads/master@{#402865} [modify] https://crrev.com/8e45aa437f44a72f971913dd1a3ceb20a6f672ad/tools/perf/page_sets/data/memory_top_10_mobile.json [modify] https://crrev.com/8e45aa437f44a72f971913dd1a3ceb20a6f672ad/tools/perf/page_sets/data/top_10_mobile.json [modify] https://crrev.com/8e45aa437f44a72f971913dd1a3ceb20a6f672ad/tools/perf/page_sets/text_selection_sites.py [modify] https://crrev.com/8e45aa437f44a72f971913dd1a3ceb20a6f672ad/tools/perf/page_sets/top_10_mobile.py
,
Jul 13 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c5acd3851a2f34de15dd89119f0be48c22a9aa72 commit c5acd3851a2f34de15dd89119f0be48c22a9aa72 Author: mythria <mythria@chromium.org> Date: Wed Jul 13 09:02:45 2016 [perf] Change google.com to google.co.uk in top_10_mobile_memory and dual_browser page sets. In the top10_mobile page set google.com gets redirected to google.ca. This sometimes causes problems in telemetry with scrolling. As a short term fix, replace google.com with google.co.uk in the page set. We cannot change it to google.ca because memory_top_10_mobile uses the same page set but with a different wpr that has a recording of google.co.uk. Updated dual_browser page set to be consistent with top_10_mobile page set. Also recorded new wprs for memory_top_10_mobile, dual_browser and top_10_mobile_memory. BUG= chromium:617914 LOG=N CQ_INCLUDE_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq Review-Url: https://codereview.chromium.org/2138133003 Cr-Commit-Position: refs/heads/master@{#405099} [modify] https://crrev.com/c5acd3851a2f34de15dd89119f0be48c22a9aa72/tools/perf/page_sets/data/dual_browser_story.json [add] https://crrev.com/c5acd3851a2f34de15dd89119f0be48c22a9aa72/tools/perf/page_sets/data/dual_browser_story_002.wpr.sha1 [modify] https://crrev.com/c5acd3851a2f34de15dd89119f0be48c22a9aa72/tools/perf/page_sets/data/memory_top_10_mobile.json [add] https://crrev.com/c5acd3851a2f34de15dd89119f0be48c22a9aa72/tools/perf/page_sets/data/memory_top_10_mobile_001.wpr.sha1 [modify] https://crrev.com/c5acd3851a2f34de15dd89119f0be48c22a9aa72/tools/perf/page_sets/data/top_10_mobile.json [add] https://crrev.com/c5acd3851a2f34de15dd89119f0be48c22a9aa72/tools/perf/page_sets/data/top_10_mobile_004.wpr.sha1 [modify] https://crrev.com/c5acd3851a2f34de15dd89119f0be48c22a9aa72/tools/perf/page_sets/dual_browser_story.py [modify] https://crrev.com/c5acd3851a2f34de15dd89119f0be48c22a9aa72/tools/perf/page_sets/text_selection_sites.py [modify] https://crrev.com/c5acd3851a2f34de15dd89119f0be48c22a9aa72/tools/perf/page_sets/top_10_mobile.py
,
Jul 13 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c5acd3851a2f34de15dd89119f0be48c22a9aa72 commit c5acd3851a2f34de15dd89119f0be48c22a9aa72 Author: mythria <mythria@chromium.org> Date: Wed Jul 13 09:02:45 2016 [perf] Change google.com to google.co.uk in top_10_mobile_memory and dual_browser page sets. In the top10_mobile page set google.com gets redirected to google.ca. This sometimes causes problems in telemetry with scrolling. As a short term fix, replace google.com with google.co.uk in the page set. We cannot change it to google.ca because memory_top_10_mobile uses the same page set but with a different wpr that has a recording of google.co.uk. Updated dual_browser page set to be consistent with top_10_mobile page set. Also recorded new wprs for memory_top_10_mobile, dual_browser and top_10_mobile_memory. BUG= chromium:617914 LOG=N CQ_INCLUDE_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq Review-Url: https://codereview.chromium.org/2138133003 Cr-Commit-Position: refs/heads/master@{#405099} [modify] https://crrev.com/c5acd3851a2f34de15dd89119f0be48c22a9aa72/tools/perf/page_sets/data/dual_browser_story.json [add] https://crrev.com/c5acd3851a2f34de15dd89119f0be48c22a9aa72/tools/perf/page_sets/data/dual_browser_story_002.wpr.sha1 [modify] https://crrev.com/c5acd3851a2f34de15dd89119f0be48c22a9aa72/tools/perf/page_sets/data/memory_top_10_mobile.json [add] https://crrev.com/c5acd3851a2f34de15dd89119f0be48c22a9aa72/tools/perf/page_sets/data/memory_top_10_mobile_001.wpr.sha1 [modify] https://crrev.com/c5acd3851a2f34de15dd89119f0be48c22a9aa72/tools/perf/page_sets/data/top_10_mobile.json [add] https://crrev.com/c5acd3851a2f34de15dd89119f0be48c22a9aa72/tools/perf/page_sets/data/top_10_mobile_004.wpr.sha1 [modify] https://crrev.com/c5acd3851a2f34de15dd89119f0be48c22a9aa72/tools/perf/page_sets/dual_browser_story.py [modify] https://crrev.com/c5acd3851a2f34de15dd89119f0be48c22a9aa72/tools/perf/page_sets/text_selection_sites.py [modify] https://crrev.com/c5acd3851a2f34de15dd89119f0be48c22a9aa72/tools/perf/page_sets/top_10_mobile.py
,
Jul 18 2016
,
Jul 19 2016
|
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by mythria@chromium.org
, Jun 7 2016