New issue
Advanced search Search tips

Issue 734622 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Jan 2018
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

30.2%-578% regression in media.tough_video_cases_tbmv2 at 479667:479838

Project Member Reported by dalecur...@chromium.org, Jun 19 2017

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=734622

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtu3zhgsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg9qPGtwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtsXi7QkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtpPaxAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtsvRjAgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtrrPzAgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg9vS39QgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtq_epAkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtrjS4QsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtoKD0wsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgto-lowsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtti_7ggM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtrfaiwgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1oeoigkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg9tS-9AsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg9oC2xwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtqzzqAsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgttPW2woM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg9vm3pQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtvOtiwgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg9rzT6QsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg9v30owkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtvyNyAgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtov3iAgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgto_-_AgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg9uCglwgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtoqo0goM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg9uSa_QkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtsmv7AsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg9rzT6QoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtpfhjggM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtrzAnAgM


Bot(s) for this bug's original alert(s):

chromium-rel-mac11-air
chromium-rel-mac11-pro
chromium-rel-mac12
chromium-rel-win10
chromium-rel-win7-dual
chromium-rel-win7-gpu-ati
chromium-rel-win7-gpu-intel
chromium-rel-win7-x64-dual
chromium-rel-win8-dual
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Jun 19 2017

Cc: shivanisha@chromium.org
Owner: shivanisha@chromium.org

=== Auto-CCing suspected CL author shivanisha@chromium.org ===

Hi shivanisha@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : shivanisha
  Commit : 54d4504bd6a17d6bb7a8628446b9e0ef5c3a74cf
  Date   : Thu Jun 15 16:52:44 2017
  Subject: [cache lock fix] parallel validation: reset the values correctly on restarting

Bisect Details
  Configuration: mac_air_perf_bisect
  Benchmark    : media.tough_video_cases_tbmv2
  Metric       : memory:chrome:all_processes:reported_by_chrome:net:effective_size_avg/video.html?src_garden2_10s.mp4
  Change       : 285.95% | 929.142857143 -> 3586.0

Revision             Result                  N
chromium@479666      929.143 +- 15.418       14      good
chromium@479708      930.286 +- 25.6571      21      good
chromium@479729      928.0 +- 0.0            14      good
chromium@479734      928.0 +- 0.0            6       good
chromium@479737      928.0 +- 0.0            6       good
chromium@479738      928.0 +- 0.0            6       good
chromium@479739      5360.67 +- 4838.22      6       bad       <--
chromium@479749      3586.0 +- 9945.33       14      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=video.html.src.garden2.10s.mp4 media.tough_video_cases_tbmv2

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8976339626860954320

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5855657460236288


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Owner: ----
It seems unlikely for this CL to be related to a memory regression, since all it is doing is fixing a crash by resetting member fields correctly when restarting a transaction.
Kicking another bisect.(https://chromeperf.appspot.com/buildbucket_job_status/8975705740774744272)
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Jun 27 2017


=== BISECT JOB RESULTS ===
Bisect was unable to run to completion

Error: INFRA_FAILURE

The bisect was able to narrow the range, you can try running with:
  good_revision: f660386651c6b5767034e00eb31d3ef725995add
  bad_revision : ffff364f06b3fa261dcf249ded47675c2d0d48a3

If failures persist contact the team (see below) and report the error.


Bisect Details
  Configuration: mac_air_perf_bisect
  Benchmark    : media.tough_video_cases_tbmv2
  Metric       : memory:chrome:browser_process:reported_by_chrome:net:effective_size_avg/memory:chrome:browser_process:reported_by_chrome:net:effective_size_avg

Revision             Result                  N
chromium@479666      929.043 +- 3.11105      8      good
chromium@479708      928.696 +- 1.9676       6      good
chromium@479729      928.696 +- 0.0          6      good
chromium@479749      1134.69 +- 393.99       9      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.tough_video_cases_tbmv2

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8975705740774744272

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5823630274985984


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Jun 27 2017


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : shivanisha
  Commit : 54d4504bd6a17d6bb7a8628446b9e0ef5c3a74cf
  Date   : Thu Jun 15 16:52:44 2017
  Subject: [cache lock fix] parallel validation: reset the values correctly on restarting

Bisect Details
  Configuration: mac_air_perf_bisect
  Benchmark    : media.tough_video_cases_tbmv2
  Metric       : memory:chrome:browser_process:reported_by_chrome:net:effective_size_avg/memory:chrome:browser_process:reported_by_chrome:net:effective_size_avg
  Change       : 21.71% | 929.468599034 -> 1131.2826087

Revision             Result                  N
chromium@479666      929.469 +- 3.86628      9       good
chromium@479708      929.59 +- 2.49439       14      good
chromium@479729      928.85 +- 2.1504        9       good
chromium@479734      928.928 +- 2.54016      6       good
chromium@479737      929.275 +- 1.17096      6       good
chromium@479738      929.507 +- 3.60354      6       good
chromium@479739      1206.33 +- 204.086      5       bad       <--
chromium@479749      1131.28 +- 391.291      8       bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.tough_video_cases_tbmv2

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8975656713869706880

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5823630274985984


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Owner: shivanisha@chromium.org
I am OOO for 2 days and will take a look at this once I return on July 5.  
Cc: erikc...@chromium.org
Looking at the graph at https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtrfaiwgM

there seem to be spikes in the value but it is also returning to normal values consistently after the change mentioned in the bisect. That makes me think it could be noise. Any thoughts?
Status: WontFix (was: Untriaged)
Agreed.

Sign in to add a comment