New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 713113 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Apr 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Bug-Regression

Blocked on:
issue 713149



Sign in to add a comment

50.5% regression in media.android.tough_video_cases_tbmv2 at 464134:464233

Project Member Reported by alexclarke@chromium.org, Apr 19 2017

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=713113

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgguXmtwkM


Bot(s) for this bug's original alert(s):

android-nexus5X
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Apr 19 2017


=== BISECT JOB RESULTS ===
Bisect failed for unknown reasons

Please contact the team (see below) and report the error.


Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : media.android.tough_video_cases_tbmv2
  Metric       : cpu_time_percentage_avg/video.html?src_tulip2.ogv


To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=video.html.src.tulip2.ogv media.android.tough_video_cases_tbmv2

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8981888069724745408

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5812046991458304


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Blockedon: 713149
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Apr 19 2017

Cc: johnchen@chromium.org
Owner: johnchen@chromium.org

=== Auto-CCing suspected CL author johnchen@chromium.org ===

Hi johnchen@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : johnchen
  Commit : 97a6fea550e9d2fcd81649cffbb0de8b4750af9a
  Date   : Wed Apr 12 23:49:54 2017
  Subject: Disable atrace in TBMv2 media benchmarks

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : media.android.tough_video_cases_tbmv2
  Metric       : cpu_time_percentage_avg/video.html?src_tulip2.ogv
  Change       : 38.67% | 0.809635391184 -> 1.12268116024

Revision             Result                     N
chromium@464133      0.809635 +- 0.0395952      6      good
chromium@464183      0.816349 +- 0.036941       6      good
chromium@464208      0.818206 +- 0.0568559      6      good
chromium@464209      0.812239 +- 0.0603066      6      good
chromium@464210      1.14372 +- 0.0793586       6      bad       <--
chromium@464212      1.12888 +- 0.0901339       6      bad
chromium@464215      1.12617 +- 0.0623402       6      bad
chromium@464221      1.1257 +- 0.0687016        6      bad
chromium@464233      1.12268 +- 0.0731864       6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=video.html.src.tulip2.ogv media.android.tough_video_cases_tbmv2

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8981876813986874016

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5812046991458304


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
The affected media, tulip2.ogv, isn't supported on Android, and the results reported for this media isn't actually meaningful. This isn't a true perf regression. I will stop testing this media on Android.
Project Member

Comment 8 by bugdroid1@chromium.org, Apr 20 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5b42152cb32481bca8a330b023a1f2c57ad38cd6

commit 5b42152cb32481bca8a330b023a1f2c57ad38cd6
Author: johnchen <johnchen@chromium.org>
Date: Thu Apr 20 05:00:06 2017

Remove theora from Android media benchmarks

Some media benchmark pages use theora format, which isn't supported
on Android. One of the media, tulip2.ogv was measured on Android and
consumed lab resources while reporting meaningless data. This CL
modifies media benchmarks to stop playing theora format media on
Android.

BUG= 713113 

Review-Url: https://codereview.chromium.org/2826293003
Cr-Commit-Position: refs/heads/master@{#465901}

[modify] https://crrev.com/5b42152cb32481bca8a330b023a1f2c57ad38cd6/tools/perf/benchmarks/media.py

Labels: OS-Android
Status: Fixed (was: Untriaged)

Sign in to add a comment