New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 673402 link

Starred by 1 user
Status: WontFix
Owner:
█ hubbe@chromium.org
Last visit > 30 days ago
Closed: Jun 2017
Cc:
█ hubbe@google.com
crouleau@chromium.org
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

10.9%-16.4% regression in media.tough_video_cases at 437363:437418

Project Member Reported by hubbe@chromium.org, Dec 12 2016 
See the link to graphs below.
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Dec 12 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: 
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N    Good?
chromium@437362  85.6607  606.278  156  good
chromium@437395  98.3411  704.849  156  unknown
chromium@437418  98.9641  703.88   156  bad

Bisect job ran on: mac_10_11_perf_bisect
Bug ID: 673402

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.tough_video_cases
Test Metric: idle_wakeups/idle_wakeups
Relative Change: 15.53%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_11_perf_bisect/builds/1112
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993457926530794080


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5867109320491008

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 4 by hubbe@chromium.org, Dec 13 2016

Owner: hubbe@chromium.org
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Dec 13 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N    Good?
chromium@437362  319.179  3011.89  156  good
chromium@437418  366.096  3588.37  156  bad

Bisect job ran on: mac_10_11_perf_bisect
Bug ID: 673402

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.tough_video_cases
Test Metric: idle_wakeups_renderer/idle_wakeups_renderer
Relative Change: 14.70%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_11_perf_bisect/builds/1118
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993435440446639744


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5805803494178816

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Dec 13 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N    Good?
chromium@437362  1185.96  10055.9  156  good
chromium@437418  1350.87  11475.5  156  bad

Bisect job ran on: mac_10_11_perf_bisect
Bug ID: 673402

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.tough_video_cases
Test Metric: idle_wakeups_total/idle_wakeups_total
Relative Change: 13.91%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_11_perf_bisect/builds/1119
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993435431665483632


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5852988407545856

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 10 by 42576172...@developer.gserviceaccount.com, Dec 13 2016 


===== BISECT JOB RESULTS =====
Status: failed


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N    Good?
chromium@437362  83.754   601.913  156  good
chromium@437418  98.5284  699.758  156  bad

Bisect job ran on: mac_10_11_perf_bisect
Bug ID: 673402

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.tough_video_cases
Test Metric: idle_wakeups/idle_wakeups
Relative Change: 17.64%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_11_perf_bisect/builds/1122
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993365776651013328


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5872336698343424

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Jan 10 2017 


=== BISECT JOB RESULTS ===
Perf regression found but unable to continue

Bisect was stopped because a commit couldn't be classified as either
good or bad.


Bisect Details
  Configuration: mac_10_11_perf_bisect
  Benchmark    : media.tough_video_cases
  Metric       : idle_wakeups/idle_wakeups

Revision             Result                  N
chromium@437362      85.7595 +- 606.257      156      good
chromium@437395      98.5543 +- 705.796      156      unknown
chromium@437418      99.4212 +- 705.833      156      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.tough_video_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8990822537435221968

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5814775029694464


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 14 by crouleau@chromium.org, Jan 13 2017

Cc: crouleau@chromium.org
Project Member

Comment 15 by 42576172...@developer.gserviceaccount.com, Jan 14 2017 


=== BISECT JOB RESULTS ===
Perf regression found but unable to narrow commit range

Build failures prevented the bisect from narrowing the range further.


Bisect Details
  Configuration: mac_10_11_perf_bisect
  Benchmark    : media.tough_video_cases
  Metric       : idle_wakeups_renderer/idle_wakeups_renderer
  Change       : 14.64% | 319.217948718 -> 365.955128205

Suspected Commit Range
  15 commits in range
  https://chromium.googlesource.com/chromium/src/+log/f32ade75b611e31c6f43914a93c3f7067bcc8cce..5c49d183f4713e682965e831387a1f4bdac32034


Revision             Result                  N
chromium@437362      319.218 +- 24.7172      6        good
chromium@437379      320.814 +- 21.0483      6        good
chromium@437380      318.372 +- 7.93589      6        good
chromium@437381      ---                     ---      build failure
---                  ---                     ---      too many build failures to list
chromium@437394      ---                     ---      build failure
chromium@437395      364.532 +- 35.3545      6        bad
chromium@437418      365.955 +- 24.1844      6        bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.tough_video_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8990561367425749360

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5792866737061888


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 17 by hubbe@chromium.org, Jun 1 2017

Status: WontFix (was: Untriaged)
►

Sign in to add a comment