New issue
Advanced search Search tips

Issue 668169 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Jan 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocked on:
issue 669608



Sign in to add a comment

22.7% regression in v8.key_mobile_sites_smooth at 433811:433835

Project Member Reported by nzolghadr@chromium.org, Nov 23 2016

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=668169

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgm9HFrgoM


Bot(s) for this bug's original alert(s):

android-nexus5
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Nov 23 2016


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N    Good?
chromium@433810  101.261  1946.67  195  good
chromium@433835  118.478  2567.28  195  bad

Bisect job ran on: android_nexus5_perf_bisect
Bug ID: 668169

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.key_mobile_sites_smooth
Test Metric: v8_gc_total_outside_idle/v8_gc_total_outside_idle
Relative Change: None

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4378
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8995189882269714496


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5852980295761920

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, Nov 24 2016


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N    Good?
chromium@433750  99.2588  1923.85  195  good
chromium@433850  119.519  2577.57  195  bad

Bisect job ran on: android_nexus5_perf_bisect
Bug ID: 668169

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.key_mobile_sites_smooth
Test Metric: v8_gc_total_outside_idle/v8_gc_total_outside_idle
Relative Change: None

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4380
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8995084466016808464


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5244209416634368

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Blockedon: 669608
Pretty sure this is a side effect of enabling wrapper tracing. You can see the original regression, later improvement, and followup regression all matching landing, reverting and relanding of that commit.


Here is the commit in question:

Reland "[wrapper-tracing] Enable flag per default"

BUG= chromium:468240 

Review-Url: https://codereview.chromium.org/2503043002
Cr-Commit-Position: refs/heads/master@{#433823}

Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Jan 13 2017

Cc: mlippautz@chromium.org
Owner: mlippautz@chromium.org

=== Auto-CCing suspected CL author mlippautz@chromium.org ===

Hi mlippautz@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : mlippautz
  Commit : e23de6962be8b4967127ae95040f32bbb533b4d1
  Date   : Tue Nov 22 09:28:34 2016
  Subject: Reland "[wrapper-tracing] Enable flag per default"

Bisect Details
  Configuration: android_nexus6_perf_bisect
  Benchmark    : v8.key_mobile_sites_smooth
  Metric       : v8_gc_total_outside_idle/v8_gc_total_outside_idle
  Change       : 15.55% | 62.5220595293 -> 72.2437988936

Revision             Result                  N
chromium@433810      62.5221 +- 4.89921      6      good
chromium@433817      60.6147 +- 6.50606      9      good
chromium@433821      61.3471 +- 4.22906      8      good
chromium@433822      60.6992 +- 4.85585      6      good
chromium@433823      72.1708 +- 4.0549       6      bad       <--
chromium@433824      72.3208 +- 4.36176      9      bad
chromium@433837      72.2438 +- 4.84572      6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.key_mobile_sites_smooth

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8990640848768895520

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4977439345737728


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Status: WontFix (was: Untriaged)
Yes, it then got reverted. It relanded later on with a performance fix following a few days afterwards. This is all visible in the graph.
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Apr 11 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus5_perf_bisect
  Benchmark    : v8.key_mobile_sites_smooth
  Metric       : v8_gc_total_outside_idle/v8_gc_total_outside_idle

Revision             Result                  N
chromium@433750      99.2588 +- 1923.85      195      good
chromium@433850      119.519 +- 2577.57      195      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.key_mobile_sites_smooth

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982640109161837200

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5244209416634368


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Sign in to add a comment