Issue metadata
Sign in to add a comment
|
10.9% regression in dromaeo.domcorequery at 424013:424071 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Oct 12 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8999003206035240624
,
Oct 12 2016
=== Auto-CCing suspected CL author mathp@chromium.org === Hi mathp@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : [Autofill] Log count histogram for the number of server cards Author : mathp Commit description: BUG= 653636 TEST=AutofillMetrics Review-Url: https://codereview.chromium.org/2395243002 Cr-Commit-Position: refs/heads/master@{#424053} Commit : 2af1bc9bc29520e5b3893b8c31d008d1b384b296 Date : Sat Oct 08 03:04:41 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@424012 2880.43 113.447 5 good chromium@424042 2857.93 50.1121 5 good chromium@424050 2890.96 57.6191 5 good chromium@424052 2853.33 86.2051 5 good chromium@424053 2668.79 54.033 5 bad <-- chromium@424054 2650.38 95.0233 5 bad chromium@424057 2424.93 38.259 5 bad chromium@424071 2477.14 68.0115 5 bad Bisect job ran on: android_nexus7_perf_bisect Bug ID: 655140 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests dromaeo.domcorequery Test Metric: dom/dom Relative Change: 14.00% Score: 99.0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3389 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8999003206035240624 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5842791261798400 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Oct 12 2016
Hi, I don't think logging two histograms will incur a 10% regression. Is it possible another change is to blame? Thanks!
,
Oct 13 2016
,
Oct 13 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8998905668595918048
,
Oct 13 2016
=== Auto-CCing suspected CL author ssid@chromium.org === Hi ssid@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Add LevelDB database names to tracing path for easier debugging Author : ssid Commit description: The data from field trials show large databases, but not sure what are their names. So, adding names to the path. BUG= 645126 Review-Url: https://codereview.chromium.org/2399903004 Cr-Commit-Position: refs/heads/master@{#424055} Commit : 279f8d702e8b2fffda1b7f50b311c5202331e121 Date : Sat Oct 08 03:21:14 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@424012 2864.61 85.3338 5 good chromium@424042 2842.83 60.3292 5 good chromium@424050 2888.03 54.8605 5 good chromium@424054 2722.15 43.6809 5 good chromium@424055 2466.06 57.904 5 bad <-- chromium@424056 2452.38 31.5053 5 bad chromium@424057 2445.74 41.0923 5 bad chromium@424071 2465.7 35.4628 5 bad Bisect job ran on: android_nexus7_perf_bisect Bug ID: 655140 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests dromaeo.domcorequery Test Metric: dom/dom Relative Change: 13.93% Score: 99.9 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3393 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8998905668595918048 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5846651900526592 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Oct 13 2016
This is a V8 specific benchmark. My change was in browser and in tracing. So, it should not affect. I don't have much context about V8. I think dglazkov@ change could have cause this regression. V8 related CLs: commit 28e4cd6cdb271055f445717508ebea7595fa4676 author dglazkov <dglazkov@chromium.org> Sat Oct 08 04:51:00 2016 committer Commit bot <commit-bot@chromium.org> Sat Oct 08 04:52:49 2016 Move the setting of bindings code generator up the stack. This opens the path for cleanly switching different types of code generators. Also eliminates the need for IdlCompilerV8 and IdlCompilerDictionaryImpl. R=bashi,yukishiino BUG= 654129 Review-Url: https://codereview.chromium.org/2401043003 Cr-Commit-Position: refs/heads/master@{#424070} commit a420fe39c98570d80347675134d3738af20c63b7 author iclelland <iclelland@chromium.org> Sat Oct 08 01:10:00 2016 committer Commit bot <commit-bot@chromium.org> Sat Oct 08 01:12:31 2016 [Origin Trials] Clarify the distinction between trials and features There is some confusion in the origin trial method names and comments between the name of a feature, which is declared in RuntimeEnabledFeatures, and a trial, which can enable multiple features. This CL attempts to make a distinction between the two, most visibly by renaming OriginTrialContext::isFeatureEnabled to OriginTrialContext::isTrialEnabled. R=chasej@chromium.org Review-Url: https://codereview.chromium.org/2398793003 Cr-Commit-Position: refs/heads/master@{#424033}
,
Oct 13 2016
ssid@ that's unlikely. My change didn't make any changes to the generated code. Ian, would you check and see if your change could be the culprit?
,
Oct 13 2016
I don't think that's possible; my change (https://crrev.com/a420fe39c98570d80347675134d3738af20c63b7) only renamed one method, one instance var, and updated comments. Shouldn't have changed the binary at all.
,
Oct 13 2016
ssid@, any other candidates? What's the range? The bisect links are confusing.
,
Oct 13 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8998895386487162352
,
Oct 13 2016
I do not think the bisects are going to work, the metric is very noisy. Having said that I am not even sure if this a real regression since the metric came back up to a good extent on the next runs. The range of CLs that could be responsible: https://chromium.googlesource.com/chromium/src/+log/29d66d91e305746076cadfc02065a908beeff129%5E..98e99ecd2fb3fbffbe6a622b0f38a3e930a37964?pretty=fuller +dtu, Ned who wrote this benchmark, Is this a significant regression?
,
Oct 13 2016
===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Add LevelDB database names to tracing path for easier debugging Author : ssid Commit description: The data from field trials show large databases, but not sure what are their names. So, adding names to the path. BUG= 645126 Review-Url: https://codereview.chromium.org/2399903004 Cr-Commit-Position: refs/heads/master@{#424055} Commit : 279f8d702e8b2fffda1b7f50b311c5202331e121 Date : Sat Oct 08 03:21:14 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@424012 2881.79 39.0689 5 good chromium@424042 2897.59 19.5091 5 good chromium@424050 2881.12 64.5898 5 good chromium@424054 2712.14 78.972 5 good chromium@424055 2479.13 35.4708 5 bad <-- chromium@424056 2419.38 14.1561 5 bad chromium@424057 2459.23 37.5299 5 bad chromium@424071 2434.11 32.6227 5 bad Bisect job ran on: android_nexus7_perf_bisect Bug ID: 655140 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests dromaeo.domcorequery Test Metric: dom/dom Relative Change: 15.53% Score: 99.8 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3395 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8998895386487162352 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=6417643643338752 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Oct 13 2016
I agree that looking at the graph, it's hard to tell whether this is just an outlier due to noise. Let wait for a few week, look at the graph again & decide.
,
Oct 13 2016
The bisect shows values that look very consistent in comment #14. But, the code that is called only on tracing. I am confused.
,
Oct 13 2016
ssid@: can you make a speculative revert to see if the benchmark recovered?
,
Oct 26 2016
Super noisy graph, but it's long term trend seems worse that it was before the regression, although there appears to have been some recovery. I doubt we can bisect this :/
,
Oct 26 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8997725546431567584
,
Oct 26 2016
===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : [Autofill] Log count histogram for the number of server cards Author : mathp Commit description: BUG= 653636 TEST=AutofillMetrics Review-Url: https://codereview.chromium.org/2395243002 Cr-Commit-Position: refs/heads/master@{#424053} Commit : 2af1bc9bc29520e5b3893b8c31d008d1b384b296 Date : Sat Oct 08 03:04:41 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@423850 2707.03 70.8167 12 good chromium@424000 2809.3 79.0863 5 good chromium@424038 2801.87 83.8838 5 good chromium@424048 2858.89 66.389 8 good chromium@424051 2860.53 82.0628 8 good chromium@424052 2893.07 64.9662 8 good chromium@424053 2654.0 109.568 5 bad <-- chromium@424057 2472.92 31.7114 5 bad chromium@424075 2560.72 62.9558 5 bad chromium@424150 2553.53 117.656 8 bad Bisect job ran on: android_nexus7_perf_bisect Bug ID: 655140 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests dromaeo.domcorequery Test Metric: dom/dom Relative Change: 7.43% Score: 98.0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3426 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8997725546431567584 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5861809947410432 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Dec 20 2016
,
Aug 16 2017
This bug is over 8 months old and bisect pointed to different CLs. I think it's likely noise. |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by tdres...@chromium.org
, Oct 12 2016