New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 647355 link

Starred by 1 user
Status: WontFix
Owner:
█ bmcquade@chromium.org
Last visit > 30 days ago
Closed: Sep 2016
Cc:
█ bmcquade@chromium.org
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

37% regression in blink_perf.layout at 417298:417381

Project Member Reported by primiano@chromium.org, Sep 15 2016 
This is going to be tough, pretty bimodal

Comment 1 by primiano@chromium.org, Sep 15 2016 

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=647355

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg2YPEvQoM


Bot(s) for this bug's original alert(s):

chromium-rel-mac-hdd
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Sep 16 2016 


===== BISECT JOB RESULTS =====
Status: failed


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@417297  1420.25  14.4055  5  good
chromium@417318  1430.25  13.2771  5  good
chromium@417329  1424.92  26.0247  5  good
chromium@417334  1825.88  51.2665  5  bad
chromium@417339  1861.73  29.7214  5  bad
chromium@417381  1860.67  29.9948  5  bad

Bisect job ran on: mac_hdd_perf_bisect
Bug ID: 647355

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests blink_perf.layout
Test Metric: floats_10_1000/floats_10_1000
Relative Change: 31.01%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/797
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9001428887339690000


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5841407435079680

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, Sep 22 2016

Cc: bmcquade@chromium.org
Owner: bmcquade@chromium.org

=== Auto-CCing suspected CL author bmcquade@chromium.org ===

Hi bmcquade@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Instrument parser blocking script execution time.
Author  : bmcquade
Commit description:
  
We already track parser blocking script load time. This patch
adds support for tracking parser blocking script execution time.

We have a few use cases for this:
* allows us to understand % of parse time blocked on script exec
  time, to better understand whether script exec time is a
  significant contributor to page load time
* having this would have helped us to catch a regression that took
  hours of time to identify the root cause of (crbug.com/608424)
* there is a desire to understand how much time scripts inserted
  via doc.write spend executing, in addition to loading

BUG= 640260 

Review-Url: https://codereview.chromium.org/2238543002
Cr-Commit-Position: refs/heads/master@{#417330}
Commit  : cb637fa03a98d0f764cfcfac6fae5cebc4c7c473
Date    : Thu Sep 08 17:49:42 2016


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@417297  1436.57  60.099   5  good
chromium@417318  1429.78  19.2189  5  good
chromium@417329  1425.08  17.8305  5  good
chromium@417330  1816.57  43.5284  5  bad    <--
chromium@417331  1861.35  34.503   5  bad
chromium@417332  1813.78  30.4244  4  bad
chromium@417334  1841.35  38.7955  5  bad
chromium@417339  1885.1   169.43   5  bad
chromium@417381  1833.73  24.0397  5  bad

Bisect job ran on: mac_hdd_perf_bisect
Bug ID: 647355

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests blink_perf.layout
Test Metric: floats_10_1000/floats_10_1000
Relative Change: 27.65%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/807
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000818821934008096


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5785879036035072

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Sep 23 2016 

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/811
Failure reason: the build has failed due to infrastructure failure.
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Sep 23 2016 

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/812
Failure reason: the build has failed due to infrastructure failure.
Project Member

Comment 11 by 42576172...@developer.gserviceaccount.com, Sep 23 2016 

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/814
Failure reason: the build has failed due to infrastructure failure.
Project Member

Comment 13 by 42576172...@developer.gserviceaccount.com, Sep 24 2016 


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Instrument parser blocking script execution time.
Author  : bmcquade
Commit description:
  
We already track parser blocking script load time. This patch
adds support for tracking parser blocking script execution time.

We have a few use cases for this:
* allows us to understand % of parse time blocked on script exec
  time, to better understand whether script exec time is a
  significant contributor to page load time
* having this would have helped us to catch a regression that took
  hours of time to identify the root cause of (crbug.com/608424)
* there is a desire to understand how much time scripts inserted
  via doc.write spend executing, in addition to loading

BUG= 640260 

Review-Url: https://codereview.chromium.org/2238543002
Cr-Commit-Position: refs/heads/master@{#417330}
Commit  : cb637fa03a98d0f764cfcfac6fae5cebc4c7c473
Date    : Thu Sep 08 17:49:42 2016


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@417297  1430.83  12.6118  5  good
chromium@417318  1442.93  7.72859  5  good
chromium@417329  1431.77  7.13518  5  good
chromium@417330  1836.22  6.34412  4  bad    <--
chromium@417331  1839.91  25.7118  5  bad
chromium@417332  1830.78  44.1072  4  bad
chromium@417334  1853.77  40.9208  5  bad
chromium@417339  1870.64  23.0056  5  bad
chromium@417381  1856.82  44.8899  6  bad

Bisect job ran on: mac_hdd_perf_bisect
Bug ID: 647355

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests blink_perf.layout
Test Metric: floats_10_1000/floats_10_1000
Relative Change: 29.97%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/815
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000707690651112480


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6342315671552000

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 14 by bmcquade@chromium.org, Sep 24 2016

Status: WontFix (was: Assigned)
Though the bisect repro'd, when I run the trybot against trunk vs a revert of this change, I do not see any significant change in performance: http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-09-24_16-07-58. Here, the patch (revert) shows slightly better performance than trunk.

Additionally, looking at the graph for this test https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg2YPEvQoM shows that it has largely returned to normal levels. Historically, I can see that this test has been a bit all over the place. It appears that we're just looking at a noisy test result here, rather than an actual regression.

Given the above, I'm going to close this as WontFix.
►

Sign in to add a comment