New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 656682 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner: ----
Closed: Jul 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

17.5%-96% regression in page_cycler_v2.top_10_mobile at 424542:424625

Project Member Reported by jasontiller@google.com, Oct 17 2016

Issue description

See the link to graphs below.
 
Project Member

Comment 4 by 42576172...@developer.gserviceaccount.com, Oct 17 2016

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2188
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Cc: -jasontiller@google.com jasontiller@chromium.org
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Oct 17 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@424569  450.343  99.3839  18  good
chromium@424579  387.465  93.3151  18  bad

Bisect job ran on: android_nexus9_perf_bisect
Bug ID: 656682

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests page_cycler_v2.top_10_mobile
Test Metric: timeToFirstMeaningfulPaint_avg/pcv1-cold/https___www.google.co.uk__hl_en_q_science
Relative Change: 5.42%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2187
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8998537813669420672


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6396619577098240

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Oct 17 2016

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2190
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Project Member

Comment 10 by 42576172...@developer.gserviceaccount.com, Oct 18 2016

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_11_perf_bisect/builds/967
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Project Member

Comment 11 by sheriffbot@chromium.org, Oct 18 2016

Labels: Hotlist-Google
Project Member

Comment 14 by 42576172...@developer.gserviceaccount.com, Oct 18 2016

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_11_perf_bisect/builds/968
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Project Member

Comment 15 by 42576172...@developer.gserviceaccount.com, Oct 18 2016

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2196
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Project Member

Comment 17 by 42576172...@developer.gserviceaccount.com, Oct 19 2016

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4267
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Project Member

Comment 19 by 42576172...@developer.gserviceaccount.com, Oct 26 2016

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4297
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Project Member

Comment 21 by 42576172...@developer.gserviceaccount.com, Dec 17 2016


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@424315  1142.48  40.3496  5  good
chromium@425088  1938.63  15.1024  3  bad

Bisect job ran on: android_nexus5_perf_bisect
Bug ID: 656682

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...m.youtube.com.results.q.science page_cycler_v2.top_10_mobile
Test Metric: timeToFirstMeaningfulPaint_avg/pcv1-warm/http___m.youtube.com_results?q_science
Relative Change: 69.69%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4468
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993073429926256480


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5296777845014528

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 24 by 42576172...@developer.gserviceaccount.com, Apr 11 2017

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4865
Failure reason: the build has failed due to infrastructure failure.

Project Member

Comment 25 by 42576172...@developer.gserviceaccount.com, Apr 11 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus5_perf_bisect
  Benchmark    : page_cycler_v2.top_10_mobile
  Metric       : timeToFirstMeaningfulPaint_avg/pcv1-warm/http___m.youtube.com_results?q_science

Revision             Result                  N
chromium@424315      1142.48 +- 40.3496      5      good
chromium@425088      1938.63 +- 15.1024      3      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...m.youtube.com.results.q.science page_cycler_v2.top_10_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982624186039320208

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5296777845014528


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Status: WontFix (was: Untriaged)
This alert was found before M-60 branched. Closing as WontFix as this is believed to either be invalid or non-reproducible.

Sign in to add a comment