New issue
Advanced search Search tips

Issue 670442 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Jan 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug-Regression

Blocked on:
issue 669608



Sign in to add a comment

3.9%-12% regression in media.tough_video_cases at 434230:434295

Project Member Reported by jrumm...@chromium.org, Dec 1 2016

Issue description

See the link to graphs below.
 

===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N    Good?
chromium@434229  50.2692  1349.35  130  good
chromium@434295  56.0615  1506.39  130  bad

Bisect job ran on: mac_10_10_perf_bisect
Bug ID: 670442

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.tough_video_cases
Test Metric: idle_wakeups_gpu/idle_wakeups_gpu
Relative Change: None

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_10_perf_bisect/builds/2494
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994444128961516320


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5836305848598528

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Blockedon: 669608
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Jan 12 2017

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_10_perf_bisect/builds/2532
Failure reason: the build has failed.

Cc: crouleau@chromium.org
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Jan 14 2017

Cc: nedngu...@google.com
Owner: nedngu...@google.com

=== Auto-CCing suspected CL author nednguyen@google.com ===

Hi nednguyen@google.com, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : nednguyen
  Commit : 4e71cc39324b205febf5a4cfa3a089a1e2aeab74
  Date   : Wed Nov 23 17:55:09 2016
  Subject: Update ts_proxy to latest commit

Bisect Details
  Configuration: mac_10_12_perf_bisect
  Benchmark    : media.tough_video_cases
  Metric       : idle_wakeups_renderer/idle_wakeups_renderer
  Change       : 11.29% | 372.435897436 -> 414.487179487

Revision                                 Result                  N
chromium@434229                          372.436 +- 10.9667      6      good
chromium@434246                          366.994 +- 8.8871       6      good
chromium@434248                          356.705 +- 42.8573      6      good
chromium@434248,catapult@982d024dad      364.538 +- 8.57701      6      good
chromium@434248,catapult@4e71cc3932      406.571 +- 17.8371      6      bad       <--
chromium@434249                          408.788 +- 17.1894      6      bad
chromium@434250                          417.09 +- 21.519        6      bad
chromium@434254                          408.756 +- 20.7492      6      bad
chromium@434262                          411.205 +- 40.5924      6      bad
chromium@434295                          414.487 +- 19.4181      6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.tough_video_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8990561326073427680

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6434671216295936


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Labels: -Pri-2 Pri-3
Status: Assigned (was: Untriaged)
Seems like this is caused by changes in tsproxy.py which looks to be a script for simulating network conditions. So this is not a bug in Chrome. I'm lowering this to P3 and Ned can close if there is nothing to be done here.
Status: WontFix (was: Assigned)
Won't fix because this is a regression due to benchmarking changes, not due to Chrome changes.

Sign in to add a comment