Issue metadata
Sign in to add a comment
|
22.9% regression in rasterize_and_record_micro.top_25 at 496020:496140 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Aug 25 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8970294886827306240
,
Aug 25 2017
=== Auto-CCing suspected CL author kerrnel@chromium.org === Hi kerrnel@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : Greg Kerr Commit : 66d4a01a3d80c6cb507112d230824d1e9cdafee7 Date : Mon Aug 21 22:45:05 2017 Subject: Mac V2 Sandbox: Disable re-execution of helper executable. Bisect Details Configuration: mac_10_12_mini_8gb_perf_bisect Benchmark : rasterize_and_record_micro.top_25 Metric : rasterize_time/Pinterest Change : 22.45% | 0.802666666667 -> 0.982833333333 Revision Result N chromium@496019 0.802667 +- 0.0225241 6 good chromium@496080 0.798833 +- 0.019205 6 good chromium@496095 0.799667 +- 0.00867948 6 good chromium@496099 0.789333 +- 0.545368 9 good chromium@496100 1.01533 +- 0.123529 6 bad <-- chromium@496101 0.981167 +- 0.0126029 6 bad chromium@496103 0.966333 +- 0.162915 6 bad chromium@496110 0.997 +- 0.0235372 6 bad chromium@496140 0.982833 +- 0.0126029 6 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=Pinterest rasterize_and_record_micro.top_25 More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8970294886827306240 For feedback, file a bug with component Speed>Bisection
,
Aug 25 2017
I reverted the finch trial enabling the feature, so none of that code is executing anymore. The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4fa660ddcdc1744e5f4380c336df5c8e2226642c commit 4fa660ddcdc1744e5f4380c336df5c8e2226642c Author: Greg Kerr <kerrnel@chromium.org> Date: Thu Aug 24 20:48:27 2017 Revert "Enable Finch Trial Test for MacV2Sandbox." This reverts commit 9705b7aaabf4c81d4e9cb13174338df9bde61cc4. Reason for revert: This CL seems to be causing performance regressions. crbug.com/757993 Original change's description: > Enable Finch Trial Test for MacV2Sandbox. > > Cq-depend: CL:611386 > Bug: 749839 > Change-Id: I03057065380f5e49b899f1c9f9869e175d3223dc > Reviewed-on: https://chromium-review.googlesource.com/612323 > Reviewed-by: Robert Sesek <rsesek@chromium.org> > Reviewed-by: Steven Holte <holte@chromium.org> > Commit-Queue: Greg Kerr <kerrnel@chromium.org> > Cr-Commit-Position: refs/heads/master@{#494502} TBR=rsesek@chromium.org,holte@chromium.org Bug: 756145 , 756623 Change-Id: Iccada6b649f20f07ea4af4fdc995dc1a8426ded4 Reviewed-on: https://chromium-review.googlesource.com/633623 Reviewed-by: Greg Kerr <kerrnel@chromium.org> Reviewed-by: Robert Sesek <rsesek@chromium.org> Commit-Queue: Greg Kerr <kerrnel@chromium.org> Cr-Commit-Position: refs/heads/master@{#497180} [modify] https://crrev.com/4fa660ddcdc1744e5f4380c336df5c8e2226642c/testing/variations/fieldtrial_testing_config.json
,
Aug 25 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8970271514345415056
,
Aug 25 2017
kernel: This is strange, looks like the revert landed in r497180, but we've had a few data points come in since then and the numbers didn't go back down. The bisect in #3 points pretty clearly to your CL and has the same magnitude, so it's confusing that the numbers didn't come down. Kicking off another bisect to see if it reproduces. Adding rasterize_and_record_micro owners to see if they have any thoughts on what might be going on.
,
Aug 25 2017
That is strange. Here's what I'm doing: 1) Running a build locally to make sure the V2 sandbox is disabled (through abort() statements) 2) I am happy to revert the CL being blamed by this bisect, I just doubt that it will help. But it was only landed to try and get the perf regressions to go down, so if we do think it's still somehow causing issues, I am happy to revert and get all my recent changes out of the picture.
,
Aug 25 2017
vmpstr, wkorman: Should kernel revert that CL? So far just this single regression on a Mac with spinning disk on pinterest is known.
,
Aug 25 2017
My only initial thought is that last week I landed a move to WPRgo which re-recorded the Pinterest page producing an actual html page but lacking styles, where prior to that it was basically a static 404 page html. See: https://bugs.chromium.org/p/chromium/issues/detail?id=756946#c8 So we basically planned to treat the new results for Pinterest as the new baseline but the graph for this issue doesn't seem to reflect the change?
,
Aug 25 2017
If you look at the graph in https://chromeperf.appspot.com/group_report?bug_id=758907, there is a big jump that links to bug 756946 and then a smaller jump associated with this one. So it looks like we have a new baseline from that change, then on top of it r496100 seems to have caused an additional 22% regression. But IIUC, it's in an experiment that was later flagged off in r497180, and the graph doesn't go back down at the flag off. Given that this only happened on one page on one bot, should we investigate further? Or WontFix unless more alerts/bisects come back telling us that r496100 is problematic?
,
Aug 25 2017
I am ok doing WontFix and waiting to see if the issue is broader than the one Pinterest page/bot.
,
Aug 25 2017
Thanks everyone. If this reaches a point where it's easier to diagnose by reverting the CL, just send me a bug, I'm happy to do it.
,
Aug 25 2017
stgm. WontFix-ing for now.
,
Aug 25 2017
=== BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : Greg Kerr Commit : 66d4a01a3d80c6cb507112d230824d1e9cdafee7 Date : Mon Aug 21 22:45:05 2017 Subject: Mac V2 Sandbox: Disable re-execution of helper executable. Bisect Details Configuration: mac_10_12_mini_8gb_perf_bisect Benchmark : rasterize_and_record_micro.top_25 Metric : rasterize_time/Pinterest Change : 20.46% | 0.822666666667 -> 0.991 Revision Result N chromium@496019 0.822667 +- 0.18351 9 good chromium@496080 0.803167 +- 0.0157745 6 good chromium@496095 0.805167 +- 0.0289971 6 good chromium@496099 0.7995 +- 0.019532 6 good chromium@496100 0.987833 +- 0.0230398 6 bad <-- chromium@496101 0.990167 +- 0.0261693 6 bad chromium@496103 0.953444 +- 0.231781 9 bad chromium@496110 0.994667 +- 0.0248059 6 bad chromium@496140 0.991 +- 0.0179444 6 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=Pinterest rasterize_and_record_micro.top_25 More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8970271514345415056 For feedback, file a bug with component Speed>Bisection
,
Aug 25 2017
[this bisect is the exact same page/device as before, still WontFix unless more data comes in pointing at this cl] |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by 42576172...@developer.gserviceaccount.com
, Aug 25 2017