Issue metadata
Sign in to add a comment
|
114.2% regression in page_cycler_v2.intl_es_fr_pt-BR at 419111:419159 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Sep 21 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9000921749874876432
,
Sep 21 2016
,
Sep 21 2016
Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_bisect/builds/1014 Failure reason: the build has failed. Additional errors: The metric was not found in the test output. Either of the initial "good" or "bad" revisions failed to be tested or built.
,
Sep 21 2016
,
Sep 21 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9000870900387577616
,
Sep 22 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9000807950039656096
,
Sep 22 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@419110 898.151 280.186 18 good chromium@419159 1096.38 217.963 18 bad Bisect job ran on: android_s5_perf_bisect Bug ID: 648893 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests page_cycler_v2.intl_es_fr_pt-BR Test Metric: timeToFirstContentfulPaint_avg/pcv1-warm/http___www.ebay.es_sch_i.html?_sacat_382__trkparms_clkid%3D6548971389060485883__qi_RTM1381637 Relative Change: 42.82% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_bisect/builds/1018 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000870900387577616 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5840096287260672 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Sep 22 2016
Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_bisect/builds/1021 Failure reason: the build has failed. Additional errors: The metric was not found in the test output. Either of the initial "good" or "bad" revisions failed to be tested or built.
,
Sep 26 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9000477583037353616
,
Sep 26 2016
Regression is likely happened in the graph, but Std Dev wasn't narrow enough in the bisect. I kicked another bisect with a large N, 40.
,
Sep 27 2016
Still running ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@419110 948.768 185.271 12 good chromium@419135 866.666 192.739 5 good chromium@419159 1254.32 112.274 8 bad
,
Sep 27 2016
===== BISECT JOB RESULTS ===== Status: failed ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@419110 948.768 185.271 12 good chromium@419135 866.666 192.739 5 good chromium@419159 1254.32 112.274 8 bad Bisect job ran on: android_s5_perf_bisect Bug ID: 648893 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests page_cycler_v2.intl_es_fr_pt-BR Test Metric: timeToFirstContentfulPaint_avg/pcv1-warm/http___www.ebay.es_sch_i.html?_sacat_382__trkparms_clkid%3D6548971389060485883__qi_RTM1381637 Relative Change: 29.83% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_bisect/builds/1035 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000477583037353616 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5873003856920576 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Sep 28 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9000313304104328592
,
Sep 28 2016
Kicked another bisect within 419135:419159. Also, this might be recovered around r419884 that is the last build at this point.
,
Sep 29 2016
===== BISECT JOB RESULTS ===== Status: failed ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@419135 881.591 215.518 12 good chromium@419159 1191.3 189.176 18 bad Bisect job ran on: android_s5_perf_bisect Bug ID: 648893 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests page_cycler_v2.intl_es_fr_pt-BR Test Metric: timeToFirstContentfulPaint_avg/pcv1-warm/http___www.ebay.es_sch_i.html?_sacat_382__trkparms_clkid%3D6548971389060485883__qi_RTM1381637 Relative Change: 10.28% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_bisect/builds/1041 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000313304104328592 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5213129913401344 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Sep 29 2016
cc: sullivan, perfbot sheriff Could you help me to succeed to run a bisect? > "failure_reason": "INFRA_FAILURE"
,
Sep 29 2016
Roberto, I can't find the error message in the output (I no error in stdio for "Working on revision chromium@419147.Performance Test 4 of 6 (2)". Can you clarify the error? Dave, how far along is the story-filter work? Could we re-run these bisects with story filters?
,
Sep 29 2016
This job timed out after 24 hrs.
,
Oct 11 2016
Perf sheriff ping
,
Oct 20 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8998328522073651648
,
Oct 20 2016
Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_bisect/builds/1116 Failure reason: the build has failed. Additional errors: The metric was not found in the test output. Either of the initial "good" or "bad" revisions failed to be tested or built.
,
Nov 17 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8995785690798052640
,
Nov 17 2016
===== BISECT JOB RESULTS ===== Status: failed === Bisection aborted === The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence. Please contact the the team (see below) if you believe this is in error. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@419110 621.127 1312.18 27 good chromium@419159 717.424 1479.56 27 bad Bisect job ran on: android_nexus5X_perf_bisect Bug ID: 648893 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...www.ebay.es.sch.i.html..sacat.382..trkparms.clkid.3D6548971389060485883..qi.RTM1381637 page_cycler_v2.intl_es_fr_pt-BR Test Metric: timeToFirstContentfulPaint_avg/pcv1-warm/http___www.ebay.es_sch_i.html?_sacat_382__trkparms_clkid%3D6548971389060485883__qi_RTM1381637 Relative Change: None Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/880 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8995785690798052640 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5231048730869760 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Dec 15 2016
Closing as the s5 bot has been decommissioned.
,
Apr 11 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8982651822800608480
,
Apr 11 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8982651590149366048
,
Apr 11 2017
=== BISECT JOB RESULTS === Perf regression found but unable to narrow commit range Build failures prevented the bisect from narrowing the range further. Bisect Details Configuration: android_s5_perf_bisect Benchmark : page_cycler_v2.intl_es_fr_pt-BR Metric : timeToFirstContentfulPaint_avg/pcv1-warm/http___www.ebay.es_sch_i.html?_sacat_382__trkparms_clkid%3D6548971389060485883__qi_RTM1381637 Suspected Commit Range 1 commits in range https://chromium.googlesource.com/chromium/src/+log/325c64a962cb5c007368ef431dfa863c26c7847b..d527839060fe162c8cf2e9841098b19f5963d6e8 Revision Result N chromium@419135 881.591 +- 215.518 12 good chromium@419159 1191.3 +- 189.176 18 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests page_cycler_v2.intl_es_fr_pt-BR Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8982651822800608480 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5213129913401344 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Apr 11 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8982584657942723424
,
Apr 11 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8982584470285843648 |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by toyoshim@chromium.org
, Sep 21 2016