New issue
Advanced search Search tips

Issue 671076 link

Starred by 0 users

Issue metadata

Status: WontFix
Owner: ----
Closed: Jul 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocked on:
issue 672130



Sign in to add a comment

27.6% regression in startup.cold.blank_page at 435538:435575

Project Member Reported by qyears...@chromium.org, Dec 5 2016

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=671076

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgx8mC4gsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgp-rw8wkM


Bot(s) for this bug's original alert(s):

chromium-rel-win8-dual
Cc: chrome-cron@google.com
Owner: chrome-cron@google.com

=== PERF REGRESSION ===


=== Auto-CCing suspected CL author chrome-cron@google.com ===

Hi chrome-cron@google.com, the bisect results pointed to your CL, please take a look at the
results.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Updating trunk VERSION from 2938.0 to 2939.0
Author  : chrome-cron
Commit description:
  
Cr-Commit-Position: refs/heads/master@{#435548}
Commit  : e4d6b5b750ed9ff02c811a8a601105e3659f382c
Date    : Thu Dec 01 04:05:21 2016


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@435537  3744.63  1206.94  8  good
chromium@435547  3827.0   214.63   8  good
chromium@435548  3156.2   89.9378  5  bad    <--
chromium@435549  3265.6   106.626  5  bad
chromium@435550  3157.2   126.644  5  bad
chromium@435552  3198.4   294.973  5  bad
chromium@435556  3140.4   151.793  5  bad
chromium@435575  3209.4   185.933  5  bad

Bisect job ran on: win_8_perf_bisect
Bug ID: 671076

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests startup.cold.blank_page
Test Metric: foreground_tab_request_start/foreground_tab_request_start
Relative Change: 14.29%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_8_perf_bisect/builds/2291
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994150709543201744


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5895796849901568

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Owner: ----
That seems unlikely. Trying again.

===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@435537  3882.7   1310.61  27  good
chromium@435575  3809.78  684.125  27  bad

Bisect job ran on: win_8_perf_bisect
Bug ID: 671076

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests startup.cold.blank_page
Test Metric: foreground_tab_load_complete/foreground_tab_load_complete
Relative Change: 1.88%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_8_perf_bisect/builds/2292
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994106435198205824


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5266546828509184

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Cc: -chrome-cron@google.com
Note, the noise for both ref build and TOT build increased at this point.

===== BISECT JOB RESULTS =====
Status: started


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@435537  3490.15  2110.5   27  good
chromium@435556  3505.2   125.55   5   good
chromium@435566  3361.0   259.969  5   good
chromium@435568  3864.88  169.183  8   bad
chromium@435569  3861.25  210.968  8   bad
chromium@435571  3764.25  263.32   8   bad
chromium@435575  3836.78  655.115  18  bad

Bisect job ran on: win_8_perf_bisect
Bug ID: 671076

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests startup.cold.blank_page
Test Metric: foreground_tab_request_start/foreground_tab_request_start
Relative Change: 9.93%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_8_perf_bisect/builds/2296
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993987036118652048


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5774179045998592

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
That bisect looks like it actually completed, but the bug update doesn't reflect that. Logged https://github.com/catapult-project/catapult/issues/3083
Blockedon: 672130
Project Member

Comment 14 by 42576172...@developer.gserviceaccount.com, Apr 12 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: win_8_perf_bisect
  Benchmark    : startup.cold.blank_page
  Metric       : foreground_tab_load_complete/foreground_tab_load_complete

Revision             Result                  N
chromium@435537      3922.86 +- 1997.36      21      good
chromium@435575      3781.33 +- 1201.36      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests startup.cold.blank_page

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982632270192570608

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5266546828509184


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Status: WontFix (was: Untriaged)
This alert was found before M-60 branched. Closing as WontFix as this is believed to either be invalid or non-reproducible. 

Sign in to add a comment