Issue metadata
Sign in to add a comment
|
7.2% regression in top_10_mobile_memory at 403925:403936 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Jul 7 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@403924 6110910 220010 18 good chromium@403936 6188727 153579 18 bad Bisect job ran on: win_8_perf_bisect Bug ID: 626327 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests top_10_mobile_memory Test Metric: memory:chrome:all_processes:reported_by_chrome:v8:effective_size_avg/http___www.baidu.com_s?word_google Relative Change: 0.52% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_8_perf_bisect/builds/2011 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9007792840680607328 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5298690521038848 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Jul 7 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@403900 6188436 171796 18 good chromium@403950 6203463 115075 18 bad Bisect job ran on: win_8_perf_bisect Bug ID: 626327 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests top_10_mobile_memory Test Metric: memory:chrome:all_processes:reported_by_chrome:v8:effective_size_avg/http___www.baidu.com_s?word_google Relative Change: 1.71% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_8_perf_bisect/builds/2012 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9007778563479701952 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5786495726649344 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Jul 8 2016
Lets try again with more repeats.
,
Jul 8 2016
=== Auto-CCing suspected CL author jochen@chromium.org === Hi jochen@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Remove the prototype from all V8 functions that aren't constructors Author : jochen Commit description: BUG= 625823 R=haraken@chromium.org TBR=eroman@chromium.org,yzshen@chromium.org Review-Url: https://codereview.chromium.org/2126763002 Cr-Commit-Position: refs/heads/master@{#403888} Commit : 596fd5ed32fdcce384a3a5f6453582919bef6528 Date : Wed Jul 06 12:31:13 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@403524 6353446 256458 8 good chromium@403830 6525242 57501.4 5 good chromium@403869 6497775 60369.2 5 good chromium@403879 6499379 24334.8 5 good chromium@403884 6525762 23886.8 5 good chromium@403886 6519273 57400.1 5 good chromium@403887 6514997 23098.5 5 good chromium@403888 6243962 44482.0 5 bad <-- chromium@403907 6142704 139670 5 bad chromium@403983 6251737 58447.8 5 bad chromium@404136 6159671 132497 8 bad Bisect job ran on: win_8_perf_bisect Bug ID: 626327 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests top_10_mobile_memory Test Metric: memory:chrome:all_processes:reported_by_chrome:v8:effective_size_avg/http___www.baidu.com_s?word_google Relative Change: 2.13% Score: 99.9 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_8_perf_bisect/builds/2015 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9007718014894080864 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5875632941039616 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Jul 12 2016
so in that bisect, my rev actually has a lower (i.e. better) value. Also, in the graph, my CL is before the regression...
,
Jul 12 2016
It look like the bot doesn't realize lower is better for this metric, +sulivan@ as an FYI. yosin@ The bisect log suggests the memory regression is at chromium@403887. Could your patch (https://codereview.chromium.org/2126743002) have caused it?
,
Jul 13 2016
Current alert says 403925:403936, My patch 403887 and before alter range. It seems there is a high spike at 403902-403911 then low spike at 403912-403924. Also, there is low spike at 404116-404120. It seems this test is flaky...
,
Jul 13 2016
Ulan points out that the graph for the renderer looks the same, so I guess it's a renderer regression. Also, the _max graph doesn't change, so I suspect it's the initial memory size that regressed (as the _avg is the average between the start and end size afaik)
,
Aug 18 2016
Perf sheriff ping: reminder to follow up on possible performance issues
,
Oct 6 2016
There's a subsequent regression on that chart that was marked WontFix: https://bugs.chromium.org/p/chromium/issues/detail?id=628120 That chart ends when the benchmark moved to TBMv2 in August, IIUC. Want to mark this bug WontFix?
,
Oct 6 2016
Yeah we can drop this. |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by alexclarke@chromium.org
, Jul 7 2016