New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 636360 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Aug 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

24.8%-96.9% regression in memory.long_running_idle_gmail_background_tbmv2 at 410113:410495

Project Member Reported by pmeenan@chromium.org, Aug 10 2016

Issue description

See the link to graphs below.
 
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Aug 10 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev   N   Good?
chromium@410420  96948932  9816088   18  good
chromium@410452  97200211  12973946  18  bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 636360

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests memory.long_running_idle_gmail_background_tbmv2
Test Metric: memory:chrome:renderer_processes:reported_by_chrome:v8:allocated_objects_size_max/memory:chrome:renderer_processes:reported_by_chrome:v8:allocated_objects_size_max
Relative Change: 12.66%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/486
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004709909947149024


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5229948235677696

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Aug 10 2016


===== BISECT JOB RESULTS =====
Status: failed


===== TESTED REVISIONS =====
Revision         Mean     Std Dev   N  Good?
chromium@410112  71.022   14.5961   5  good
chromium@410304  60.0293  23.1029   5  good
chromium@410316  57.0054  21.1521   5  good
chromium@410322  64.8941  19.0614   5  good
chromium@410323  69.8933  14.3694   5  good
chromium@410324  3.97352  5.36528   5  bad
chromium@410325  3.89975  5.18351   5  bad
chromium@410328  1.38198  0.174393  5  bad
chromium@410352  1.33006  0.248017  5  bad
chromium@410400  1.43395  0.286068  5  bad
chromium@410495  1.69013  0.299284  5  bad

Bisect job ran on: mac_10_11_perf_bisect
Bug ID: 636360

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests memory.long_running_idle_gmail_background_tbmv2
Test Metric: v8-gc-incremental-step_percentage_idle/v8-gc-incremental-step_percentage_idle
Relative Change: 97.62%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_11_perf_bisect/builds/824
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004696379316318912


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5894879503187968

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Aug 10 2016


===== BISECT JOB RESULTS =====
Status: failed


===== TESTED REVISIONS =====
Revision         Mean     Std Dev   N  Good?
chromium@410112  71.022   14.5961   5  good
chromium@410304  60.0293  23.1029   5  good
chromium@410316  57.0054  21.1521   5  good
chromium@410322  64.8941  19.0614   5  good
chromium@410323  69.8933  14.3694   5  good
chromium@410324  3.97352  5.36528   5  bad
chromium@410325  3.89975  5.18351   5  bad
chromium@410328  1.38198  0.174393  5  bad
chromium@410352  1.33006  0.248017  5  bad
chromium@410400  1.43395  0.286068  5  bad
chromium@410495  1.69013  0.299284  5  bad

Bisect job ran on: mac_10_11_perf_bisect
Bug ID: 636360

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests memory.long_running_idle_gmail_background_tbmv2
Test Metric: v8-gc-incremental-step_percentage_idle/v8-gc-incremental-step_percentage_idle
Relative Change: 97.62%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_11_perf_bisect/builds/824
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004696379316318912


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5894879503187968

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Owner: l...@chromium.org
lpy@ - looks like the tbm version of these gc metrics depended on your v8 runtime call stats in tracing (https://codereview.chromium.org/2187693002/ ) that was reverted in the v8 roll here: https://chromium.googlesource.com/v8/v8/+log/743b9df1..ef574268
Cc: pmeenan@chromium.org
 Issue 636359  has been merged into this issue.
Issue 636357 has been merged into this issue.

Comment 12 by l...@chromium.org, Aug 11 2016

Cc: fmea...@chromium.org
I don't understand here, does the bisect mean that this regression is because the CL gets reverted?
I don't know that it caused all of them but a few of the graphs dropped down to zero.  Is it possible that the trace-based-measurements (tbm) were relying on trace events that were part of your CL and they went away when it reverted?

The bisect only got as far as the v8 roll and failed to bisect into it so I want through the v8 CL's included in the roll and the revert of the trace stats looked like the only one that could/should have been able to impact them (nothing else in there looked gc or trace related).

Comment 14 by l...@chromium.org, Aug 11 2016

I am surprised because I expected the revert would make things better since the CL itself would have negative impact on memory allocation. So if the reason is because the CL got reverted, then I guess we can close this since the CL already relanded?

Comment 15 by l...@chromium.org, Aug 11 2016

The range on the chromeperf dashboard has one v8-autoroll, which updated the chromium to version 5.4.374, but the reverted CL is in 5.4.367, before this v8-autoroll, and the relanded CL is in 5.4.392, after this v8-autoroll.

So CLs that relate to my changes are not in the chromium commit position range given by the dashboard, which is 410113 - 410495 https://chromium.googlesource.com/v8/v8/+log/d67f12f7..199f49c5
Cc: hpayer@chromium.org
The range actually spans multiple pages and the second page has several v8 auto roll's, including the one that the bisect stopped on.

+hpayer in case it is related to the gc work he was doing that landed in a different roll but in the same commit range (though bisect didn't pick them up).

Do you know when the CL re-landed?  The graphs look like they are current up to earlier today so if it relanded before today it's clearly no the cause.

Comment 17 by l...@chromium.org, Aug 11 2016

The reland happened in v8 version 5.4.392, the v8-autoroll for it was commit c61b8edd4dbe18a68b79c2eabc4c59ca40b51990, which happened on Wed Aug 10 10:31:22 2016 -0700
Owner: pmeenan@chromium.org
Reassigning to myself.  Back to the drawing board.

Comment 19 by l...@chromium.org, Aug 11 2016

Cc: l...@chromium.org
Project Member

Comment 21 by 42576172...@developer.gserviceaccount.com, Aug 11 2016

Cc: u...@chromium.org
Owner: u...@chromium.org

=== Auto-CCing suspected CL author ulan@chromium.org ===

Hi ulan@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Unify memory optimization modes and flags.
Author  : ulan
Commit description:
  
This patch folds --optimize-for-size flag and check for low-memory device
into Heap::ShouldOptimizeForMemoryUsage() predicate.

It has the following side effects:
- the heap growing factor for low-memory devices is capped at 1.3 (old value was 2.0).
- the memory reducer will be more aggressive for low-memory devices.

BUG= chromium:634900 

Review-Url: https://codereview.chromium.org/2218703004
Cr-Commit-Position: refs/heads/master@{#38406}
Commit  : 6cebf7d963c18e2ab09f2320c949baf0a4bed721
Date    : Sun Aug 07 15:36:10 2016


===== TESTED REVISIONS =====
Revision                       Mean     Std Dev   N  Good?
chromium@410112                67.5586  14.6154   5  good
chromium@410304                49.1759  18.3169   5  good
chromium@410316                53.5749  19.4393   5  good
chromium@410322                59.882   16.556    5  good
chromium@410323                66.8095  18.4689   5  good
chromium@410323,v8@422d14350d  62.7286  17.3665   5  good
chromium@410323,v8@6cebf7d963  1.41782  0.390165  5  bad    <--
chromium@410323,v8@5e685567ea  1.29979  0.20193   5  bad
chromium@410323,v8@265399ed9a  1.45005  0.192341  5  bad
chromium@410324                1.26844  0.169591  5  bad
chromium@410325                1.342    0.255064  5  bad
chromium@410328                3.27472  4.07225   5  bad
chromium@410352                1.65242  0.194051  5  bad
chromium@410400                1.5588   0.196053  5  bad
chromium@410495                1.37264  0.202114  5  bad

Bisect job ran on: mac_10_11_perf_bisect
Bug ID: 636360

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests memory.long_running_idle_gmail_background_tbmv2
Test Metric: v8-gc-incremental-step_percentage_idle/v8-gc-incremental-step_percentage_idle
Relative Change: 97.97%
Score: 99.8

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_11_perf_bisect/builds/827
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004696379316318912


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5894879503187968

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Owner: hpayer@chromium.org
Status: Started (was: Assigned)
Should be fixed with:

commit b992a8e1899091e3c180a050c1a63fa1e29886d5
Author: hpayer <hpayer@chromium.org>
Date: Fri Aug 12 08:13:35 2016

Fix wrong state of IsolateInBackgroundNotification and IsolateInForegroundNotification.

BUG= chromium:636368 , chromium:635965 , chromium:634900 

Review-Url: https://codereview.chromium.org/2245483004
Cr-Commit-Position: refs/heads/master@{#38598}

[modify] https://crrev.com/b992a8e1899091e3c180a050c1a63fa1e29886d5/src/isolate.cc
Status: Fixed (was: Started)

Sign in to add a comment