New issue
Advanced search Search tips

Issue 643252 link

Starred by 1 user
Status: WontFix
Owner:
ksakamoto@chromium.org
Closed: Sep 2016
Cc:
ksakamoto@chromium.org
mustaq@chromium.org
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocked on:
issue 644865


Sign in to add a comment

77.2%-112.5% regression in page_cycler_v2.intl_ar_fa_he at 415614:415642

Project Member Reported by mustaq@chromium.org, Sep 1 2016 
See the link to graphs below.
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Sep 1 2016 

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2081
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Comment 4 by mustaq@chromium.org, Sep 2 2016

Cc: mustaq@chromium.org
Owner: kouhei@chromium.org
kouhei@: There are quite a few big regressions in page_cycler_v2 recently, and bisects are always failing to find the cause(s). I am assigning the bugs to you since you seem to have made many (most?) PCv2 changed recently. If I am wrong, could you please help me find the right owner of these bugs?

Comment 5 by robert...@chromium.org, Sep 9 2016

Blockedon: 644865
There is a known issue with bisecting pcv2 metrics.. I am blocking this bug on it.
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Sep 30 2016

Cc: ksakamoto@chromium.org
Owner: ksakamoto@chromium.org

=== Auto-CCing suspected CL author ksakamoto@chromium.org ===

Hi ksakamoto@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Update FirstMeaningfulPaint to use Blink's implementation
Author  : ksakamoto
Commit description:
  
Note to perf sheriffs: this is going to affect timeToFirstMeaningfulPaint
values in page_cycler_v2 benchmarks.

This updates FirstMeaningfulPaint metric to use trace events logged by
Blink's FirstMeaningfulPaintDetector, instead of computing layout
significance from FrameView::performLayout trace event.

Since Blink implementation counts more layout objects / uses the end
time of the paint phase, this patch will change TTFMP value, but it
increases accuracy (see  https://crbug.com/638124#c8  for details).

This requires the new trace event added in https://crrev.com/2276573003/.

BUG= chromium:638124 

Review-Url: https://codereview.chromium.org/2259723002
Commit  : 7bf2ae5c26055537f6c07d88d15d29485bca65b1
Date    : Wed Aug 31 01:50:43 2016


===== TESTED REVISIONS =====
Revision                             Mean     Std Dev  N  Good?
chromium@415617                      2629.64  279.706  5  good
chromium@415622                      2696.77  470.419  5  good
chromium@415624                      2735.92  220.961  5  good
chromium@415624,catapult@e403ae0791  2784.31  212.179  5  good
chromium@415624,catapult@17bda7618b  2820.63  132.494  5  good
chromium@415624,catapult@dccd40a985  2725.67  186.207  5  good
chromium@415624,catapult@7bf2ae5c26  5257.46  142.717  5  bad    <--
chromium@415625                      5185.75  80.0227  5  bad
chromium@415626                      5152.78  56.9063  5  bad

Bisect job ran on: android_nexus7_perf_bisect
Bug ID: 643252

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests page_cycler_v2.intl_ar_fa_he
Test Metric: timeToFirstMeaningfulPaint_avg/pcv1-warm/http___ynet.co.il_
Relative Change: 95.95%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3339
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000179796384942160


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5855975351779328

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 10 by ksakamoto@chromium.org, Sep 30 2016

Status: WontFix (was: Assigned)
This is expected, my CL changed the way to compute FirstMeaningfulPaint.
Project Member

Comment 11 by 42576172...@developer.gserviceaccount.com, Sep 30 2016 


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Update FirstMeaningfulPaint to use Blink's implementation
Author  : ksakamoto
Commit description:
  
Note to perf sheriffs: this is going to affect timeToFirstMeaningfulPaint
values in page_cycler_v2 benchmarks.

This updates FirstMeaningfulPaint metric to use trace events logged by
Blink's FirstMeaningfulPaintDetector, instead of computing layout
significance from FrameView::performLayout trace event.

Since Blink implementation counts more layout objects / uses the end
time of the paint phase, this patch will change TTFMP value, but it
increases accuracy (see  https://crbug.com/638124#c8  for details).

This requires the new trace event added in https://crrev.com/2276573003/.

BUG= chromium:638124 

Review-Url: https://codereview.chromium.org/2259723002
Commit  : 7bf2ae5c26055537f6c07d88d15d29485bca65b1
Date    : Wed Aug 31 01:50:43 2016


===== TESTED REVISIONS =====
Revision                             Mean     Std Dev  N  Good?
chromium@415624                      3053.18  272.292  5  good
chromium@415624,catapult@e403ae0791  3032.88  297.333  5  good
chromium@415624,catapult@17bda7618b  3070.54  274.768  5  good
chromium@415624,catapult@dccd40a985  2906.54  310.414  5  good
chromium@415624,catapult@7bf2ae5c26  5877.87  89.9709  5  bad    <--
chromium@415625                      5605.69  463.326  5  bad
chromium@415626                      5830.42  59.9929  5  bad
chromium@415627                      5847.35  96.5815  5  bad
chromium@415629                      5818.4   46.3294  5  bad
chromium@415633                      5686.28  504.128  5  bad
chromium@415642                      5881.37  54.1688  5  bad

Bisect job ran on: android_nexus5_perf_bisect
Bug ID: 643252

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests page_cycler_v2.intl_ar_fa_he
Test Metric: timeToFirstMeaningfulPaint_avg/pcv1-warm/http___ynet.co.il_
Relative Change: 92.63%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4161
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000179810095806416


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6426658830024704

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Sep 30 2016 


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Update FirstMeaningfulPaint to use Blink's implementation
Author  : ksakamoto
Commit description:
  
Note to perf sheriffs: this is going to affect timeToFirstMeaningfulPaint
values in page_cycler_v2 benchmarks.

This updates FirstMeaningfulPaint metric to use trace events logged by
Blink's FirstMeaningfulPaintDetector, instead of computing layout
significance from FrameView::performLayout trace event.

Since Blink implementation counts more layout objects / uses the end
time of the paint phase, this patch will change TTFMP value, but it
increases accuracy (see  https://crbug.com/638124#c8  for details).

This requires the new trace event added in https://crrev.com/2276573003/.

BUG= chromium:638124 

Review-Url: https://codereview.chromium.org/2259723002
Commit  : 7bf2ae5c26055537f6c07d88d15d29485bca65b1
Date    : Wed Aug 31 01:50:43 2016


===== TESTED REVISIONS =====
Revision                             Mean     Std Dev  N  Good?
chromium@415615                      256.192  19.2877  5  good
chromium@415623                      273.857  36.6907  5  good
chromium@415624                      290.685  57.1108  5  good
chromium@415624,catapult@e403ae0791  346.619  74.6234  5  good
chromium@415624,catapult@17bda7618b  332.716  47.9522  5  good
chromium@415624,catapult@dccd40a985  321.149  80.565   5  good
chromium@415624,catapult@7bf2ae5c26  560.384  44.0107  5  bad    <--
chromium@415625                      525.071  31.2462  5  bad
chromium@415627                      562.744  52.2375  5  bad
chromium@415631                      529.97   70.3797  5  bad

Bisect job ran on: linux_perf_bisect
Bug ID: 643252

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests page_cycler_v2.intl_ar_fa_he
Test Metric: timeToFirstMeaningfulPaint_avg/pcv1-cold/http___ynet.co.il_
Relative Change: 106.86%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6732
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000179802099006672


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5286549560754176

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
►

Sign in to add a comment