New issue
Advanced search Search tips

Issue 772965 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner:
Closed: Jan 2018
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

6.7%-18.3% regression in media.tough_video_cases_tbmv2 at 507199:507282

Project Member Reported by jrumm...@chromium.org, Oct 9 2017

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=772965

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=63db5c44754c327221033b35148914d6f54fa25833101df397b249f1d5770a9e


Bot(s) for this bug's original alert(s):

win-high-dpi
Cc: m...@chromium.org
Owner: m...@chromium.org
Status: Assigned (was: Untriaged)

=== Auto-CCing suspected CL author miu@chromium.org ===

Hi miu@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Yuri Wiitala
  Commit : 081f9dcc223a535202a52c351392514ad5635650
  Date   : Fri Oct 06 23:50:16 2017
  Subject: Add scaling capability to CopyOutputRequests. (Part 2 of 2)

Bisect Details
  Configuration: winx64_high_dpi_perf_bisect
  Benchmark    : media.tough_video_cases_tbmv2
  Metric       : memory:chrome:all_processes:reported_by_os:system_memory:proportional_resident_size_avg/video.html?src_tulip2.ogg_type_audio
  Change       : 18.29% | 187707297.5 -> 222034335.167

Revision             Result                    N
chromium@507198      187707298 +- 1622057      6      good
chromium@507219      187229869 +- 4232813      6      good
chromium@507230      187687969 +- 1702872      6      good
chromium@507231      187960821 +- 2078815      6      good
chromium@507232      220428621 +- 3686156      6      bad       <--
chromium@507233      221162699 +- 3827999      6      bad
chromium@507235      222334997 +- 2263746      6      bad
chromium@507240      220979525 +- 2954700      6      bad
chromium@507282      222034335 +- 3335706      6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=video.html.src.tulip2.ogg.type.audio media.tough_video_cases_tbmv2

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8966187959242384016


For feedback, file a bug with component Speed>Bisection

Comment 4 by m...@chromium.org, Oct 10 2017

Owner: crouleau@chromium.org
crouleau: Looks like my change regressed the memory usage of your perf test. The thing is, it should not be possible unless your perf test is doing screen video capture (i.e., making CopyOutputRequests with the "source" property set). Can you shed some light on how that might happen?

If your perf test does, in fact, intend to capture the screen; then please resolve this as WAI. The memory increase is attributed to the extra caching of textures introduced by my change (for performance reasons).
Cc: -m...@chromium.org crouleau@chromium.org
Owner: m...@chromium.org
Yuri, 

John filed this bug since he was the sheriff during this time. As part of his sheriff responsibilities, he will help you look into it and answer questions. There are instructions above on how to run the test and the code for it is here https://cs.chromium.org/chromium/src/tools/perf/page_sets/tough_video_cases/video.html and here https://cs.chromium.org/chromium/src/tools/perf/page_sets/tough_video_cases.py

If there are questions that John cannot answer, then I would be happy to help. I don't think the tests should be doing any video capture, but it's possible that something in Telemetry that I don't know about is doing it. This particular test intends to simply play an audio file.
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Oct 11 2017

Cc: nzolghadr@chromium.org
 Issue 773734  has been merged into this issue.

Comment 8 by m...@chromium.org, Oct 12 2017

Status: Started (was: Assigned)
Hmm...An unrelated Android test too. Weird. I wonder if this has to do with tab snapshots of backgrounded tabs...
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Oct 12 2017


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Yuri Wiitala
  Commit : 081f9dcc223a535202a52c351392514ad5635650
  Date   : Fri Oct 06 23:50:16 2017
  Subject: Add scaling capability to CopyOutputRequests. (Part 2 of 2)

Bisect Details
  Configuration: android_nexus5_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/long_running_tools/long_running_tools_gmail-background
  Change       : 11.99% | 60965278.9542 -> 68272442.5621

Revision             Result                   N
chromium@507175      60965279 +- 178605       6      good
chromium@507208      60945093 +- 177492       6      good
chromium@507225      60890473 +- 59804.0      6      good
chromium@507229      60901336 +- 139.107      6      good
chromium@507231      61076145 +- 598305       6      good
chromium@507232      68359770 +- 529662       6      bad       <--
chromium@507233      68316060 +- 358911       6      bad
chromium@507241      68272443 +- 75581.8      6      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=long.running.tools.gmail.background system_health.memory_mobile

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8965962803868298512


For feedback, file a bug with component Speed>Bisection
Project Member

Comment 11 by 42576172...@developer.gserviceaccount.com, Oct 17 2017


=== BISECT JOB RESULTS ===
NO Perf regression found, tests failed to produce values

Bisect Details
  Configuration: winx64_high_dpi_perf_bisect
  Benchmark    : media.desktop
  Metric       : memory:chrome:all_processes:reported_by_os:system_memory:proportional_resident_size_avg/video.html?src_crowd.ogg_type_audio


To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=video.html.src.crowd.ogg.type.audio media.desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8965465797213315024


For feedback, file a bug with component Speed>Bisection
Project Member

Comment 14 by 42576172...@developer.gserviceaccount.com, Oct 24 2017


=== BISECT JOB RESULTS ===
NO Perf regression found, tests failed to produce values

Bisect Details
  Configuration: winx64_high_dpi_perf_bisect
  Benchmark    : media.desktop
  Metric       : memory:chrome:all_processes:reported_by_os:system_memory:native_heap:proportional_resident_size_avg/video.html?src_crowd1080_vp9.webm


To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=video.html.src.crowd1080.vp9.webm media.desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8964816103938388528


For feedback, file a bug with component Speed>Bisection
Bisects aren't going to work anymore for this because we changed the benchmark's name :( Sorry.

Bisect is very confident that your CL was the culprit. You can rerun the benchmark manually. It is now called media.desktop instead of media.tough_video_cases.
Project Member

Comment 16 by 42576172...@developer.gserviceaccount.com, Oct 24 2017


=== BISECT JOB RESULTS ===
NO Perf regression found, tests failed to produce values

Bisect Details
  Configuration: winx64_high_dpi_perf_bisect
  Benchmark    : media.desktop
  Metric       : memory:chrome:all_processes:reported_by_os:system_memory:native_heap:proportional_resident_size_avg/video.html?src_tulip2.ogg_type_audio


To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=video.html.src.tulip2.ogg.type.audio media.desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8964816085618881584


For feedback, file a bug with component Speed>Bisection

=== BISECT JOB RESULTS ===
NO Perf regression found, tests failed to produce values

Bisect Details
  Configuration: winx64_high_dpi_perf_bisect
  Benchmark    : media.desktop
  Metric       : memory:chrome:all_processes:reported_by_os:system_memory:proportional_resident_size_avg/video.html?src_tulip2.vp9.webm


To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=video.html.src.tulip2.vp9.webm media.desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8963909174533902448


For feedback, file a bug with component Speed>Bisection

Comment 19 by m...@chromium.org, Jan 25 2018

Status: WontFix (was: Started)
Closing, as metrics are much better nowadays than pre-regression. I'm pretty sure the media metrics regressed for non-media reasons.

Sign in to add a comment