Very strange blink_perf.layout large-table-with-collapsed-borders results |
||||
Issue descriptionThe dashboard accidentally filed two bugs and kicked off two bisects on the exact same regression: https://chromeperf.appspot.com/group_report?bug_id=615418 The bisects are http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6541 and http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6540 They both got really similar results, with r395578 taking longer than the other runs. But r395578 just marks some tests flaky: ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@395578 301.978 1.90187 5 good chromium@395579 294.852 2.51474 5 bad <-- chromium@395580 292.373 2.50669 5 bad chromium@395582 289.705 2.00795 5 bad chromium@395586 295.306 1.23845 5 bad chromium@395594 292.013 2.57381 5 bad chromium@395578 302.725 2.71425 12 good chromium@395579 295.684 3.35878 8 bad <-- chromium@395580 293.139 2.39214 5 bad chromium@395582 290.119 2.07695 5 bad chromium@395586 295.575 3.11144 5 bad chromium@395594 298.497 5.14943 12 bad Looking farther down the graphs, another thing is weird: the ref build on both graphs jumps around r395965 even though there are no changes to the test or telemetry, that's just the same build of chrome stable we used for all the runs before it. I think something is strange about this test. Emil, any ideas?
,
May 28 2016
Nope, hasn't been modified in over a year: https://chromium.googlesource.com/chromium/src/+log/master/third_party/WebKit/PerformanceTests/Layout/large-table-with-collapsed-borders-and-colspans.html https://chromium.googlesource.com/chromium/src/+log/master/third_party/WebKit/PerformanceTests/Layout/large-table-with-collapsed-borders-and-no-colspans.html
,
May 28 2016
,
May 30 2016
Annie: it's possible that the jump to both TOT & ref is due to some hardware change then.
,
May 31 2016
That's truly bizarre. Really looks like either the test itself or the way we run it changed between those two runs. Can't find anything relevant anywhere near that regression range though.
,
May 31 2016
Hmhh, I think that there is a bug in telemetry or the recipe that make the ref build system no longer work. I am looking at issue 614858 & found that the tracing_over_head of ref build just regressed, which seems impossible due to just hardware/test change.
,
May 31 2016
At least for this case, I can confirm this is either a bug in recipe or a bug in dashboard. I clicked on the graph of large-table-with-collapsed-borders-and-colspans.html_ref, then click on "Buildbot stdio" link & here is the log: https://build.chromium.org/p/chromium.perf/builders/Win%207%20Low-End%20Perf%20%282%29/builds/3818/steps/blink_perf.layout/logs/stdio The benchmark command in the log is: C:\b\build\slave\Win_7_Low-End_Perf__2_\build\src\tools\perf\run_benchmark' -v '--output-format=chartjson' --upload-results blink_perf.layout '--browser=release'
,
Jun 1 2016
,
Sep 13 2017
|
||||
►
Sign in to add a comment |
||||
Comment 1 by nedngu...@google.com
, May 28 2016