New issue
Advanced search Search tips

Issue 670743 link

Starred by 0 users
Status: WontFix
Owner: ----
Closed: Dec 2016
Cc:
qyears...@chromium.org
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

38.1% regression in page_cycler_v2.intl_ja_zh at 435537:435574

Project Member Reported by qyears...@chromium.org, Dec 2 2016 
See the link to graphs below.
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Dec 2 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@435536  245.518  143.343  27  good
chromium@435574  249.258  107.209  27  bad

Bisect job ran on: winx64_10_perf_bisect
Bug ID: 670743

Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...www.amazon.co.jp page_cycler_v2.intl_ja_zh
Test Metric: timeToFirstMeaningfulPaint_avg/pcv1-cold/http___www.amazon.co.jp
Relative Change: 1.52%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_10_perf_bisect/builds/795
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994368426187150576


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6124856247058432

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, Dec 6 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@435536  285.446  912.533  27  good
chromium@435574  250.462  24.8256  27  bad

Bisect job ran on: winx64_10_perf_bisect
Bug ID: 670743

Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...www.amazon.co.jp page_cycler_v2.intl_ja_zh
Test Metric: timeToFirstMeaningfulPaint_avg/pcv1-cold/http___www.amazon.co.jp
Relative Change: 12.26%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_10_perf_bisect/builds/802
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994062063883517104


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4793332633436160

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 7 by qyears...@chromium.org, Dec 6 2016 

One platform, no ref build; if the next bisect comes up with nothing, then this should probably be closed.
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Dec 7 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@435536  249.241  17.6704  27  good
chromium@435574  246.27   114.462  27  bad

Bisect job ran on: winx64_10_perf_bisect
Bug ID: 670743

Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...www.amazon.co.jp page_cycler_v2.intl_ja_zh
Test Metric: timeToFirstMeaningfulPaint_avg/pcv1-cold/http___www.amazon.co.jp
Relative Change: 1.19%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_10_perf_bisect/builds/805
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993985215304038464


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5262633844867072

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 9 by pmeenan@chromium.org, Dec 13 2016

Status: WontFix (was: Untriaged)
Closing - see #7.  Also, the confidence intervals went crazy at the point of the regression so it looks like a very noisy test now.
Project Member

Comment 11 by 42576172...@developer.gserviceaccount.com, Mar 23 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: winx64_10_perf_bisect
  Benchmark    : page_cycler_v2.intl_ja_zh
  Metric       : timeToFirstMeaningfulPaint_avg/pcv1-cold/http___www.amazon.co.jp

Revision             Result                  N
chromium@435536      795.205 +- 1260.53      21      good
chromium@435574      750.725 +- 542.246      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...www.amazon.co.jp page_cycler_v2.intl_ja_zh

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8984378675005551248

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4793332633436160


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 13 by 42576172...@developer.gserviceaccount.com, Mar 24 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: winx64_10_perf_bisect
  Benchmark    : page_cycler_v2.intl_ja_zh
  Metric       : timeToFirstMeaningfulPaint_avg/pcv1-cold/http___www.amazon.co.jp

Revision             Result                  N
chromium@435536      752.777 +- 751.222      21      good
chromium@435574      704.819 +- 378.193      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...www.amazon.co.jp page_cycler_v2.intl_ja_zh

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8984280337800118672

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4793332633436160


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 15 by 42576172...@developer.gserviceaccount.com, Mar 31 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: winx64_10_perf_bisect
  Benchmark    : page_cycler_v2.intl_ja_zh
  Metric       : timeToFirstContentfulPaint_avg/pcv1-cold/http___www.amazon.co.jp

Revision             Result                  N
chromium@435536      748.79 +- 698.509       21      good
chromium@435574      752.788 +- 864.194      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...www.amazon.co.jp page_cycler_v2.intl_ja_zh

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8983638172892444208

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4793332633436160


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 17 by 42576172...@developer.gserviceaccount.com, Apr 11 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: winx64_10_perf_bisect
  Benchmark    : page_cycler_v2.intl_ja_zh
  Metric       : timeToFirstMeaningfulPaint_avg/pcv1-cold/http___www.amazon.co.jp

Revision             Result                  N
chromium@435536      812.92 +- 1325.58       21      good
chromium@435574      805.346 +- 927.326      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...www.amazon.co.jp page_cycler_v2.intl_ja_zh

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982632325142567424

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5262633844867072


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
►

Sign in to add a comment