New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 667351 link

Starred by 1 user
Status: WontFix
Owner:
kouhei@chromium.org
Closed: Mar 2017
Cc:
kouhei@chromium.org
█ nednguyen@chromium.org
mustaq@chromium.org
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

61.3%-342.2% regression in page_cycler_v2.intl_ja_zh at 433192:433275

Project Member Reported by mustaq@chromium.org, Nov 21 2016 
See the link to graphs below.
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Nov 21 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@433191  134.716  789.268  27  good
chromium@433275  103.344  33.2802  27  bad

Bisect job ran on: linux_perf_bisect
Bug ID: 667351

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...zol.com.cn. page_cycler_v2.intl_ja_zh
Test Metric: timeToFirstContentfulPaint_avg/pcv1-cold/http___zol.com.cn_
Relative Change: None

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6848
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8995364691745989936


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5262440042856448

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 4 by sullivan@chromium.org, Nov 21 2016

Cc: kouhei@chromium.org
kouhei, this is really strange, a 300% regression seen on the linux bots for time to first contentful paint that the bisect can't reproduce on the same hardware. Can you take a look at the traces from the dashboard and see if anything seems off?

before regression: https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_21-2016-11-18_15-06-11-79136.html
after regression: https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_21-2016-11-18_20-00-19-99865.html

I'll try kicking off another bisect just in case.
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Nov 21 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@433191  134.063  857.733  27  good
chromium@433275  104.432  46.0034  27  bad

Bisect job ran on: linux_perf_bisect
Bug ID: 667351

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...zol.com.cn. page_cycler_v2.intl_ja_zh
Test Metric: timeToFirstContentfulPaint_avg/pcv1-cold/http___zol.com.cn_
Relative Change: None

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6849
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8995358687671565856


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5799241698508800

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Nov 22 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@433191  127.217  699.905  27  good
chromium@433275  98.9349  25.9941  27  bad

Bisect job ran on: linux_perf_bisect
Bug ID: 667351

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...zol.com.cn. page_cycler_v2.intl_ja_zh
Test Metric: timeToFirstContentfulPaint_avg/pcv1-cold/http___zol.com.cn_
Relative Change: None

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6850
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8995272458995773408


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4930976310886400

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 9 by simonhatch@chromium.org, Nov 29 2016

Cc: nednguyen@chromium.org
Owner: kouhei@chromium.org
+nednguyen listed as the other page_cycler_v2 owner

Those means are a bit misleading, since it's the very first run in each of those bisects that takes an astonishing amount of time vs every subsequent run. Often the average time is around 100-120 ms, but the very first is 800-900. Kinda feels like there might be something wrong with the test if the first run is so dramatically different from the others.

Comment 10 by benhenry@chromium.org, Jan 17 2017

Status: Assigned (was: Untriaged)
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Mar 23 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: linux_perf_bisect
  Benchmark    : page_cycler_v2.intl_ja_zh
  Metric       : timeToFirstContentfulPaint_avg/pcv1-cold/http___zol.com.cn_

Revision             Result                  N
chromium@433191      154.32 +- 1122.2        21      good
chromium@433275      99.0483 +- 23.6536      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...zol.com.cn. page_cycler_v2.intl_ja_zh

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8984378596088224864

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4930976310886400


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Comment 13 by mustaq@chromium.org, Mar 23 2017

Status: WontFix (was: Assigned)
All recovered.
Project Member

Comment 15 by 42576172...@developer.gserviceaccount.com, Mar 24 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: linux_perf_bisect
  Benchmark    : page_cycler_v2.intl_ja_zh
  Metric       : timeToFirstContentfulPaint_avg/pcv1-cold/http___dtiblog.com_

Revision             Result                  N
chromium@433191      228.281 +- 1077.89      21      good
chromium@433275      176.575 +- 33.4034      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...dtiblog.com. page_cycler_v2.intl_ja_zh

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8984280324828471200

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4930976310886400


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 17 by 42576172...@developer.gserviceaccount.com, Mar 31 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: linux_perf_bisect
  Benchmark    : page_cycler_v2.intl_ja_zh
  Metric       : timeToFirstMeaningfulPaint_avg/pcv1-cold/http___dtiblog.com_

Revision             Result                  N
chromium@433191      224.238 +- 1053.31      21      good
chromium@433275      176.589 +- 49.4523      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...dtiblog.com. page_cycler_v2.intl_ja_zh

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8983641933890777328

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4930976310886400


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 19 by 42576172...@developer.gserviceaccount.com, Apr 11 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: linux_perf_bisect
  Benchmark    : page_cycler_v2.intl_ja_zh
  Metric       : timeToFirstContentfulPaint_avg/pcv1-cold/http___zol.com.cn_

Revision             Result                  N
chromium@433191      134.716 +- 789.268      27      good
chromium@433275      103.344 +- 33.2802      27      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...zol.com.cn. page_cycler_v2.intl_ja_zh

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982632332188356224

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5262440042856448


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
►

Sign in to add a comment