New issue
Advanced search Search tips

Issue 671237 link

Starred by 0 users

Issue metadata

Status: WontFix
Owner: ----
Closed: Jul 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocked on:
issue 669608



Sign in to add a comment

3.2% regression in thread_times.key_silk_cases at 435403:435496

Project Member Reported by qyears...@chromium.org, Dec 5 2016

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=671237

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgp_W5twoM


Bot(s) for this bug's original alert(s):

android-nexus5

===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N    Good?
chromium@435402  6.11183  63.0835  145  good
chromium@435496  6.39186  65.8412  145  bad

Bisect job ran on: android_nexus5_perf_bisect
Bug ID: 671237

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests thread_times.key_silk_cases
Test Metric: thread_GPU_cpu_time_per_frame/thread_GPU_cpu_time_per_frame
Relative Change: 4.58%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4407
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994097224483037024


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5821257960914944

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Blockedon: 669608

===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N    Good?
chromium@435402  6.12605  64.8374  145  good
chromium@435496  6.37931  64.9627  145  bad

Bisect job ran on: android_nexus5_perf_bisect
Bug ID: 671237

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests thread_times.key_silk_cases
Test Metric: thread_GPU_cpu_time_per_frame/thread_GPU_cpu_time_per_frame
Relative Change: 4.13%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4408
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994063451666733472


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5812825597935616

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
One alert, one builder, no ref build, noisy results. Probably not a regression in Chrome.

===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N    Good?
chromium@435402  6.14081  64.5766  145  good
chromium@435496  6.39368  66.8818  145  bad

Bisect job ran on: android_nexus5_perf_bisect
Bug ID: 671237

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests thread_times.key_silk_cases
Test Metric: thread_GPU_cpu_time_per_frame/thread_GPU_cpu_time_per_frame
Relative Change: 4.12%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4413
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993985063368099872


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5017932747243520

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 11 by 42576172...@developer.gserviceaccount.com, Jan 13 2017


=== BISECT JOB RESULTS ===
Perf regression found but unable to narrow commit range

Build failures prevented the bisect from narrowing the range further.


Bisect Details
  Configuration: android_nexus5_perf_bisect
  Benchmark    : thread_times.key_silk_cases
  Metric       : thread_GPU_cpu_time_per_frame/thread_GPU_cpu_time_per_frame

Suspected Commit Range
  12 commits in range
  https://chromium.googlesource.com/chromium/src/+log/5942181be68e28305ce40d64c5ba26854d9e3e3c..93df626d0de8a7af99127f34159eb4020989d93b


Revision             Result                    N
chromium@435402      6.15015 +- 0.0744545      6        good
chromium@435414      6.15511 +- 0.0844379      6        good
chromium@435415      ---                       ---      build failure
---                  ---                       ---      too many build failures to list
chromium@435425      ---                       ---      build failure
chromium@435426      6.37313 +- 0.0965538      6        bad
chromium@435449      6.36327 +- 0.0776215      6        bad
chromium@435496      6.44699 +- 0.0763899      6        bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests thread_times.key_silk_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8990641140650811360

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5865262168932352


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 13 by 42576172...@developer.gserviceaccount.com, Apr 10 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus5_perf_bisect
  Benchmark    : thread_times.key_silk_cases
  Metric       : thread_GPU_cpu_time_per_frame/thread_GPU_cpu_time_per_frame

Revision             Result                  N
chromium@435402      6.14081 +- 64.5766      145      good
chromium@435496      6.39368 +- 66.8818      145      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests thread_times.key_silk_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982659747526197824

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5017932747243520


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Status: WontFix (was: Untriaged)
This alert was found before M-60 branched. Closing as WontFix as this is believed to either be invalid or non-reproducible.

Sign in to add a comment