New issue
Advanced search Search tips

Issue 710149 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Jul 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocked on:
issue 710055



Sign in to add a comment

41.6% regression in media.android.tough_video_cases at 460340:460368

Project Member Reported by wolenetz@chromium.org, Apr 10 2017

Issue description

This alert seems valid.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=710149

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg7Mnr1gkM


Bot(s) for this bug's original alert(s):

android-nexus7v2
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Apr 10 2017


=== BISECT JOB RESULTS ===
Bisect failed for unknown reasons

Please contact the team (see below) and report the error.


Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : media.android.tough_video_cases
  Metric       : vm_private_dirty_delta/crowd720_vp9.webm_gpu


To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.android.tough_video_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982669553871298128

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6103498426417152


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Cc: simonhatch@chromium.org
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Apr 10 2017


=== BISECT JOB RESULTS ===
Bisect failed for unknown reasons

Please contact the team (see below) and report the error.


Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : media.android.tough_video_cases
  Metric       : vm_private_dirty_delta/crowd720_vp9.webm_gpu


To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.android.tough_video_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982664866339525424

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4605999425519616


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Apr 10 2017


=== BISECT JOB RESULTS ===
Bisect failed for unknown reasons

Please contact the team (see below) and report the error.


Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : media.android.tough_video_cases
  Metric       : vm_private_dirty_delta/crowd720_vp9.webm_gpu


To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.android.tough_video_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982663563959260800

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4605999425519616


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Blockedon: 710055
Blocking on  crbug.com/710055 , nexus 7v2 bots are all failing atm.
n7v2 bots are working again. Sorry about the disturbance.
Ref trace is missing, and graph has more recent change. Regardless, starting a bisect (see #11) to see if there's still something identifiable and needing fixed here.
Project Member

Comment 13 by 42576172...@developer.gserviceaccount.com, Jun 30 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : media.android.tough_video_cases
  Metric       : vm_private_dirty_delta/crowd720_vp9.webm_gpu

Revision             Result                  N
chromium@460339      1713.9 +- 2053.42       21      good
chromium@460519      1846.29 +- 2369.93      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.android.tough_video_cases

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8975428164185661872


For feedback, file a bug with component Speed>Bisection
Project Member

Comment 15 by 42576172...@developer.gserviceaccount.com, Jun 30 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : media.android.tough_video_cases
  Metric       : vm_private_dirty_delta/crowd720_vp9.webm_gpu

Revision             Result                  N
chromium@460339      1692.38 +- 2576.63      21      good
chromium@460519      1560.38 +- 2676.96      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.android.tough_video_cases

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8975392283888064224


For feedback, file a bug with component Speed>Bisection
#16 is a wider bisect range. If it finds no regression, seems like this is no longer reproducible and will be WontFix.

=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : media.android.tough_video_cases
  Metric       : vm_private_dirty_delta/crowd720_vp9.webm_gpu

Revision             Result                  N
chromium@460304      1604.19 +- 2070.58      21      good
chromium@460519      1734.86 +- 2465.79      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.android.tough_video_cases

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8974894973105453168


For feedback, file a bug with component Speed>Bisection
Status: WontFix (was: Assigned)
Per #17-18 and #12, lack of repro even with a wider bisect, and lack of ref trace on this old alert make it hard to see if it's still actionable.

Marking this WontFix.

simonhatch@ / crouleau@ -- any ideas why the ref trace is greyed-out in the "original alert graph": https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg7Mnr1gkM ? Is this a known issue? Do you agree with WontFix-ing this issue?
Not sure on the specifics of why that's greyed out there, but finding that same chart and graphing it and ref:

https://chromeperf.appspot.com/report?sid=7f8233e1dba093faf5c5b81fb6396a92fb50dc41d9a7e8832013358e2e8d2e7a&start_rev=431546&end_rev=484554

Shows that ref does exist but only in a narrow range from mid april->mid may. The build page is long gone, although logdog still works. Seems to be the same device both before/after.

Ref build seems to be failing, must have been failing for a while since the last data point is sometime in May: https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus7v2%20Perf/builds/235

Sign in to add a comment