New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 669603 link

Starred by 1 user
Status: WontFix
Owner:
verwa...@chromium.org
Closed: Aug 2017
Cc:
machenb...@chromium.org
simonhatch@chromium.org
█ nednguyen@chromium.org
hablich@chromium.org
tdres...@chromium.org
sullivan@chromium.org
█ benhenry@chromium.org
verwa...@chromium.org
rmcilroy@chromium.org
ishell@chromium.org
u...@chromium.org
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocked on:
issue 664760


Sign in to add a comment

15.8% regression in v8.infinite_scroll-ignition_tbmv2 at 433502:434299

Project Member Reported by benhenry@chromium.org, Nov 29 2016 
See graph below.

Comment 1 by benhenry@chromium.org, Nov 29 2016 

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=669603

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgh96TrQoM


Bot(s) for this bug's original alert(s):

linux-release
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Nov 29 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev   N   Good?
chromium@433501  31312861  28259148  25  good
chromium@434299  31539746  25842708  25  bad

Bisect job ran on: linux_perf_bisect
Bug ID: 669603

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.infinite_scroll-ignition_tbmv2
Test Metric: memory:chrome:all_processes:reported_by_chrome:v8:allocated_objects_size_avg/memory:chrome:all_processes:reported_by_chrome:v8:allocated_objects_size_avg
Relative Change: None

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6854
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994633437347104576


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5235839464898560

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 5 by benhenry@google.com, Nov 30 2016 

Weird. Widened the CL range.
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Nov 30 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev   N   Good?
chromium@433477  31834037  26445621  25  good
chromium@434299  30932905  29692556  25  bad

Bisect job ran on: linux_perf_bisect
Bug ID: 669603

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.infinite_scroll-ignition_tbmv2
Test Metric: memory:chrome:all_processes:reported_by_chrome:v8:allocated_objects_size_avg/memory:chrome:all_processes:reported_by_chrome:v8:allocated_objects_size_avg
Relative Change: None

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6855
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994610975967371008


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5054108929622016

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 7 by benhenry@google.com, Nov 30 2016

Cc: tdres...@chromium.org nednguyen@chromium.org
Tim and Ned - there's no owner listed in go/chrome-benchmarks for this benchmark. This seems like a real thing, but there have been a bunch of infra changes lately that have "caused" regressions. I need some help with this one.

Comment 8 by benhenry@google.com, Nov 30 2016 

There are a bunch of scrolling CLs in the blamelist: https://chromium.googlesource.com/chromium/src/+log/d42d7efd5e208eb564d0f4c060fc24121d7e6919%5E..0274b6dfab86377a44f06dfe41895c9763f9f449?pretty=fuller

Not sure how to evaluate which one is bad or good.

Comment 9 by nedngu...@google.com, Nov 30 2016

Owner: u...@chromium.org
ulan@: do you know who in v8 team should own this?

Comment 11 by tdres...@chromium.org, Nov 30 2016 

Kicked off a tumblr specific bisect - it appears to be a tumblr specific regression.

Comment 12 by sullivan@chromium.org, Nov 30 2016

Cc: hablich@chromium.org
Also +hablich may know who the owner is for v8.infinite_scroll-ignition_tbmv2

Comment 13 by hablich@chromium.org, Nov 30 2016

Cc: rmcilroy@chromium.org
Ross is the owner.
Project Member

Comment 14 by 42576172...@developer.gserviceaccount.com, Dec 1 2016

Cc: ishell@chromium.org
Owner: ishell@chromium.org

=== PERF REGRESSION ===


=== Auto-CCing suspected CL author ishell@chromium.org ===

Hi ishell@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : [ic] Remove names table from type feedback metadata.
Author  : ishell
Commit description:
  
BUG=chromium:576312, v8:5561

Review-Url: https://codereview.chromium.org/2515233002
Cr-Commit-Position: refs/heads/master@{#41130}
Commit  : 7d61ddfa912cd2c9f219054b076d895638cb6147
Date    : Mon Nov 21 10:46:16 2016


===== TESTED REVISIONS =====
Revision                       Mean      Std Dev  N   Good?
chromium@433501                33145872  37805.9  8   good
chromium@433601                33149036  43553.3  12  good
chromium@433605                33115637  258243   12  good
chromium@433606                33118234  201914   8   good
chromium@433606,v8@1834ab7246  33153010  27096.4  5   good
chromium@433606,v8@7d61ddfa91  33054779  29988.5  12  bad    <--
chromium@433606,v8@b94b53a28c  33047089  186108   12  bad
chromium@433606,v8@4513532f63  33037145  190730   12  bad
chromium@433606,v8@4097c8503e  33025489  255842   12  bad
chromium@433607                33036862  200246   12  bad
chromium@433608                33038449  196559   12  bad
chromium@433614                33065038  43300.3  5   bad
chromium@433626                33050095  10448.4  5   bad
chromium@433651                33063344  41974.3  5   bad
chromium@433701                33007147  15804.6  5   bad
chromium@433900                32990830  184925   5   bad
chromium@434299                33888460  36149.0  5   bad

Bisect job ran on: linux_perf_bisect
Bug ID: 669603

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=tumblr v8.infinite_scroll-ignition_tbmv2
Test Metric: memory:chrome:all_processes:reported_by_chrome:v8:allocated_objects_size_avg/tumblr
Relative Change: 2.24%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6862
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994548373809281248


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4935192584650752

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 16 by benhenry@google.com, Dec 1 2016

Status: Assigned (was: Untriaged)
Project Member

Comment 17 by 42576172...@developer.gserviceaccount.com, Dec 1 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev   N   Good?
chromium@433501  31305818  26471668  25  good
chromium@434299  31586420  25906636  25  bad

Bisect job ran on: linux_perf_bisect
Bug ID: 669603

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.infinite_scroll-ignition_tbmv2
Test Metric: memory:chrome:all_processes:reported_by_chrome:v8:allocated_objects_size_avg/memory:chrome:all_processes:reported_by_chrome:v8:allocated_objects_size_avg
Relative Change: None

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6866
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994459878263492048


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5852038582239232

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 18 by ishell@chromium.org, Dec 2 2016

Cc: machenb...@chromium.org
Owner: benhenry@chromium.org
The bisection went wrong again. My CL reduces the memory usage and we can even see that in the results (we use 100Kb less memory):
chromium@433606,v8@1834ab7246  33153010  27096.4  5   good
chromium@433606,v8@7d61ddfa91  33054779  29988.5  12  bad    <--

However from the same data we can see this 1Mb bump:
chromium@433900                32990830  184925   5   bad
chromium@434299                33888460  36149.0  5   bad

Assigning back for further triaging.

Comment 19 by benhenry@chromium.org, Dec 6 2016

Owner: rmcilroy@chromium.org
assigning to benchmark owner to help triage/confirm priority.

Comment 20 by simonhatch@chromium.org, Dec 6 2016

Blockedon: 664760

Comment 21 by simonhatch@chromium.org, Dec 6 2016 

Bisect in #c14 seems really off, especially at this step:

chromium@433900                32990830  184925   5   bad

This got tagged as a "bad" revision, even though it's really close to the good reference value. Should be addressed by crbug.com/664760

Comment 22 by rmcilroy@chromium.org, Dec 6 2016

Owner: u...@chromium.org
I'm not the owner of v8.infinite_scroll (this bug is occurring on both  v8.infinite_scroll and v8.infinite_scroll-ignition which runs the same benchmark with Ignition enabled, so it is not an Ignition specific issue).

I'm not the right person to own this bug - assigning to Ulan who is the creator of the  v8.infinite_scroll and I think should also be the benchmark owner.
Project Member

Comment 26 by 42576172...@developer.gserviceaccount.com, Dec 6 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev   N   Good?
chromium@433501  31111784  27636437  25  good
chromium@434299  31813317  24493876  25  bad

Bisect job ran on: linux_perf_bisect
Bug ID: 669603

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.infinite_scroll-ignition_tbmv2
Test Metric: memory:chrome:all_processes:reported_by_chrome:v8:allocated_objects_size_avg/memory:chrome:all_processes:reported_by_chrome:v8:allocated_objects_size_avg
Relative Change: 2.25%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6894
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994002422508980816


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5284090259963904

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 27 by 42576172...@developer.gserviceaccount.com, Dec 6 2016 


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : [ic] Remove names table from type feedback metadata.
Author  : ishell
Commit description:
  
BUG=chromium:576312, v8:5561

Review-Url: https://codereview.chromium.org/2515233002
Cr-Commit-Position: refs/heads/master@{#41130}
Commit  : 7d61ddfa912cd2c9f219054b076d895638cb6147
Date    : Mon Nov 21 10:46:16 2016


===== TESTED REVISIONS =====
Revision                       Mean      Std Dev  N   Good?
chromium@433555                32100986  190692   12  good
chromium@433588                32104011  164438   8   good
chromium@433604                32082122  139121   8   good
chromium@433606                32092275  136039   8   good
chromium@433606,v8@1834ab7246  32071302  35253.0  8   good
chromium@433606,v8@7d61ddfa91  31956451  37699.0  5   bad    <--
chromium@433606,v8@b94b53a28c  31985205  46234.5  5   bad
chromium@433606,v8@4513532f63  32015365  214599   12  bad
chromium@433606,v8@4097c8503e  32015663  210948   12  bad
chromium@433607                31990338  157538   12  bad
chromium@433608                31991990  156771   8   bad
chromium@433612                32007631  187683   12  bad
chromium@433620                31975832  98508.0  8   bad
chromium@433684                31988792  280594   18  bad
chromium@433812                31998005  266165   12  bad
chromium@434068                32892349  198385   5   bad
chromium@434580                33003365  131830   5   bad

Bisect job ran on: linux_perf_bisect
Bug ID: 669603

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=tumblr v8.infinite_scroll_tbmv2
Test Metric: memory:chrome:all_processes:reported_by_chrome:v8:heap:allocated_objects_size_avg/tumblr
Relative Change: 2.84%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6892
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994005364374827760


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5236463367618560

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 28 by benhenry@google.com, Dec 6 2016

Cc: u...@chromium.org
Owner: ishell@chromium.org
It looks like the owner has been updated to ulan@.

Assigning to regression CL author.

Comment 29 by ishell@chromium.org, Dec 7 2016

Cc: simonhatch@chromium.org sullivan@chromium.org
Owner: robert...@chromium.org
The result is wrong again. See #18.

Inviting infra team to the discussion.

Comment 31 by simonhatch@chromium.org, Dec 7 2016 

Kinda think this goes south on chromium@433812

Output of the comparison is:

vs Good:
{
  "result": {
    "U": 0, 
    "p": 0.004310358499535094, 
    "significance": "REJECT"
  }, 
  "sampleA": [
    31990500, 
    31945204, 
    31966888, 
    31969580, 
    31932892
  ], 
  "sampleB": [
    32077728, 
    32073700, 
    32082328, 
    32066024, 
    32191136, 
    32083940, 
    32104392, 
    32055976
  ]
}


vs Bad:
{
  "result": {
    "U": 0, 
    "p": 0.012185780355344789, 
    "significance": "NEED_MORE_DATA"
  }, 
  "sampleA": [
    31990500, 
    31945204, 
    31966888, 
    31969580, 
    31932892
  ], 
  "sampleB": [
    32835048, 
    32816300, 
    32809440, 
    33004488, 
    32996468
  ]
}


Roberto has a cl in staging that increases the default samples to prevent situations like this (crbug.com/664760). I'll try re-running this bisect on the linux staging bot.
Project Member

Comment 33 by 42576172...@developer.gserviceaccount.com, Dec 7 2016

Cc: verwa...@chromium.org
Owner: verwa...@chromium.org

=== PERF REGRESSION ===


=== Auto-CCing suspected CL author verwaest@chromium.org ===

Hi verwaest@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : [runtime] Always normalize prototype maps that aren't marked as 'should be fast' yet
Author  : verwaest
Commit description:
  
This makes the test in the bug ~10x faster. It could inadvertently make other things slower, so revert eagerly if included in a range where performance tanks.

BUG= chromium:666852 

Review-Url: https://codereview.chromium.org/2525573002
Cr-Commit-Position: refs/heads/master@{#41178}
Commit  : 30d2fb6b509f44194ee5a6a7c1caa6b36fa61074
Date    : Tue Nov 22 13:25:17 2016


===== TESTED REVISIONS =====
Revision                       Mean      Std Dev  N  Good?
chromium@433555                32114445  180486   8  good
chromium@433812                31966854  155101   8  good
chromium@433940                31976038  143574   8  good
chromium@433956                31957070  38505.7  8  good
chromium@433964                31975047  128863   8  good
chromium@433966                32007846  212734   8  good
chromium@433966,v8@2ba24a71b6  31958670  36937.2  8  good
chromium@433966,v8@a0e9160066  31992151  190736   8  good
chromium@433966,v8@30d2fb6b50  32835280  47805.9  5  bad    <--
chromium@433967                32823152  48727.5  5  bad
chromium@433968                32855074  119091   5  bad
chromium@433972                32878264  267065   5  bad
chromium@434004                32810582  60297.8  5  bad
chromium@434068                32843590  153246   5  bad
chromium@434580                33006341  206474   5  bad

Bisect job ran on: linux_perf_bisect
Bug ID: 669603

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=tumblr v8.infinite_scroll_tbmv2
Test Metric: memory:chrome:all_processes:reported_by_chrome:v8:allocated_objects_size_avg/tumblr
Relative Change: 2.78%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6904
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993925613033373520


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5868395596087296

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 34 by 42576172...@developer.gserviceaccount.com, Dec 8 2016 


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : [runtime] Always normalize prototype maps that aren't marked as 'should be fast' yet
Author  : verwaest
Commit description:
  
This makes the test in the bug ~10x faster. It could inadvertently make other things slower, so revert eagerly if included in a range where performance tanks.

BUG= chromium:666852 

Review-Url: https://codereview.chromium.org/2525573002
Cr-Commit-Position: refs/heads/master@{#41178}
Commit  : 30d2fb6b509f44194ee5a6a7c1caa6b36fa61074
Date    : Tue Nov 22 13:25:17 2016


===== TESTED REVISIONS =====
Revision                       Mean      Std Dev  N  Good?
chromium@433501                33150837  31977.7  6  good
chromium@433900                33026780  24148.1  6  good
chromium@433950                33038628  40150.4  6  good
chromium@433963                33035282  36475.7  6  good
chromium@433966                33004025  197102   6  good
chromium@433966,v8@2ba24a71b6  33012610  152982   6  good
chromium@433966,v8@a0e9160066  33031196  29857.0  6  good
chromium@433966,v8@30d2fb6b50  33904367  12608.4  6  bad    <--
chromium@433967                33892644  24661.0  6  bad
chromium@433968                33896457  43560.0  6  bad
chromium@433969                33893821  16878.9  6  bad
chromium@433975                33860543  192471   6  bad
chromium@434000                33869558  169437   6  bad
chromium@434100                33897414  39590.1  6  bad
chromium@434299                33898309  62939.3  6  bad

Bisect job ran on: staging_linux_perf_bisect
Bug ID: 669603

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.infinite_scroll-ignition_tbmv2
Test Metric: memory:chrome:all_processes:reported_by_chrome:v8:allocated_objects_size_avg/tumblr
Relative Change: 2.25%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/staging_linux_perf_bisect/builds/49
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993913116891756832


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5901597706551296

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 38 by 42576172...@developer.gserviceaccount.com, Apr 11 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: linux_perf_bisect
  Benchmark    : v8.infinite_scroll-ignition_tbmv2
  Metric       : memory:chrome:all_processes:reported_by_chrome:v8:allocated_objects_size_avg/memory:chrome:all_processes:reported_by_chrome:v8:allocated_objects_size_avg

Revision             Result                   N
chromium@433477      31780299 +- 4435433      21      good
chromium@434299      31915720 +- 3969420      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.infinite_scroll-ignition_tbmv2

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982652063962752480

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5054108929622016


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Comment 39 by sullivan@chromium.org, Aug 16 2017

Status: WontFix (was: Assigned)
Looks like too much time passed, unlikely we'll be able to follow up. WontFix-ing.
►

Sign in to add a comment