New issue
Advanced search Search tips

Issue 726800 link

Starred by 1 user
Status: WontFix
Owner: ----
Closed: Oct 2017
Cc:
tguilbert@chromium.org
scottmg@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

208.4%-1850% regression in media.android.tough_video_cases at 471771:472096

Project Member Reported by tguilbert@chromium.org, May 26 2017 
See the link to graphs below.
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, May 27 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : media.android.tough_video_cases
  Metric       : vm_shared_dirty_delta/crowd.ogg_gpu

Revision             Result                  N
chromium@471770      34.0952 +- 13.4093      21      good
chromium@472096      32.9524 +- 18.6803      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.android.tough_video_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8978507543988944192

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5267523437592576


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, May 27 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : media.android.tough_video_cases
  Metric       : vm_shared_dirty_delta/crowd.ogg_gpu

Revision             Result                 N
chromium@471770      34.6667 +- 17.282      21      good
chromium@472096      32.7619 +- 20.095      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.android.tough_video_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8978468995119229904

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5267523437592576


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Jun 3 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : media.android.tough_video_cases
  Metric       : vm_shared_dirty_delta/crowd.ogg_gpu

Revision             Result                  N
chromium@471770      36.5714 +- 19.8278      21      good
chromium@472096      34.2857 +- 18.3925      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.android.tough_video_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8977858265643417296

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5064000934510592


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Jun 3 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : media.android.tough_video_cases
  Metric       : vm_shared_dirty_delta/crowd.ogg_total

Revision             Result                   N
chromium@471770      -115.429 +- 70.2648      21      good
chromium@472096      -122.857 +- 80.2282      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.android.tough_video_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8977858249523756176

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6057615848636416


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Jun 3 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : media.android.tough_video_cases
  Metric       : vm_shared_dirty_delta/crowd.ogg_gpu

Revision             Result                  N
chromium@471770      34.0952 +- 14.5537      21      good
chromium@472096      31.4286 +- 17.2378      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.android.tough_video_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8977834815028929296

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5064000934510592


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 13 by 42576172...@developer.gserviceaccount.com, Jun 3 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : media.android.tough_video_cases
  Metric       : vm_shared_dirty_delta/crowd.ogg_total

Revision             Result                   N
chromium@471770      -111.429 +- 102.338      21      good
chromium@472096      -126.857 +- 87.6617      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.android.tough_video_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8977834807327958224

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6057615848636416


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 15 by 42576172...@developer.gserviceaccount.com, Aug 18 2017

Cc: scottmg@chromium.org
Owner: scottmg@chromium.org
Status: Assigned (was: Untriaged)

=== Auto-CCing suspected CL author scottmg@chromium.org ===

Hi scottmg@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : scottmg
  Commit : d2021c969e449a7a5d8c3e6513081ccda63e3196
  Date   : Mon May 15 16:58:05 2017
  Subject: network service: Stub of hooking up service worker URLLoaderFactory

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : media.android.tough_video_cases
  Metric       : vm_shared_dirty_delta/crowd.ogg_gpu
  Change       : 12.70% | 36.0 -> 31.4285714286

Revision             Result                  N
chromium@471770      36.0 +- 14.9666         14      good
chromium@471791      34.6667 +- 11.3137      9       good
chromium@471801      36.0 +- 11.3137         9       good
chromium@471803      34.2222 +- 5.96285      9       good
chromium@471804      29.3333 +- 4.6188       6       bad       <--
chromium@471806      28.6667 +- 3.65148      6       bad
chromium@471811      29.7778 +- 8.21922      9       bad
chromium@471852      29.3333 +- 4.6188       6       bad
chromium@471933      29.3333 +- 4.6188       6       bad
chromium@472096      31.4286 +- 17.7603      14      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.android.tough_video_cases

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8970989507067527856


For feedback, file a bug with component Speed>Bisection

Comment 16 by scottmg@chromium.org, Aug 18 2017

Owner: ----
Status: Untriaged (was: Assigned)
The code landed in that CL wasn't enabled. (Noise seems pretty high on that test too)

Comment 17 by dtapu...@chromium.org, Oct 30 2017

Components: Internals>Media

Comment 18 by tguilbert@chromium.org, Oct 31 2017

Status: WontFix (was: Untriaged)
It seems like the tests are flaky and the regressions have returned to normal (or continued flaking). I don't think there is anything actionable.
►

Sign in to add a comment