New issue
Advanced search Search tips

Issue 645480 link

Starred by 1 user

Issue metadata

Status: Duplicate
Merged: issue 645315
Owner:
Closed: Oct 2016
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

3.1%-72.8% regression in thread_times.key_silk_cases at 416418:416456

Project Member Reported by rsch...@chromium.org, Sep 9 2016

Issue description

See the link to graphs below.
 
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Sep 10 2016


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Roll src/third_party/skia/ 54682e856..af92a7f85 (1 commit).
Author  : skia-deps-roller
Commit description:
  
https://chromium.googlesource.com/skia.git/+log/54682e856cb6..af92a7f85a5f

$ git log 54682e856..af92a7f85 --date=short --no-merges --format='%ad %ae %s'
2016-09-03 mtklein kNumWindows is only used in GPU builds

CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_precise_blink_rel
TBR=bsalomon@google.com

Review-Url: https://codereview.chromium.org/2305203002
Cr-Commit-Position: refs/heads/master@{#416454}
Commit  : 06aa20af65c784d4130d73791e701cc39ae14a88
Date    : Sat Sep 03 14:38:17 2016


===== TESTED REVISIONS =====
Revision                         Mean      Std Dev    N   Good?
chromium@416453                  0.302273  0.0329108  12  good
chromium@416453,skia@af92a7f85a  0.304892  0.0160818  12  good
chromium@416454                  0.278558  0.0367142  12  bad    <--

Bisect job ran on: android_s5_perf_bisect
Bug ID: 645480

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests thread_times.key_silk_cases
Test Metric: thread_other_cpu_time_per_frame/thread_other_cpu_time_per_frame
Relative Change: 3.77%
Score: 80.0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_bisect/builds/988
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9001985865360025616


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5598380368068608

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Owner: mtklein@chromium.org
mtklein, any chance this is your change?

Comment 5 by mtkl...@google.com, Sep 12 2016

Ha, definitely not!  That's a compile-time-only change.
Owner: sullivan@chromium.org
I'm seeing some movement on the reference build around this time too, but it's across multiple devices so I'm really not sure what to think. Annie, any ideas here?
Ryan in #6: it's possible that this is due to telemetry/ perf benchmark change.
Project Member

Comment 10 by 42576172...@developer.gserviceaccount.com, Oct 13 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@416453  8.92583  3.21391  18  good
chromium@416454  8.74239  3.12779  18  bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 645480

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests smoothness.sync_scroll.key_mobile_sites_smooth
Test Metric: mean_input_event_latency/http___news.yahoo.com
Relative Change: 19.75%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2642
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8998986239444625744


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5821737332113408

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Oct 27 2016

Mergedinto: 645315
Status: Duplicate (was: Assigned)

===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : [Telemetry] Switch ShouldTearDownStateAfterEachStoryRun to True as default
Author  : perezju
Commit description:
  
To reduce complexity of benchmarks and imcrease stability of measurements
benchmarks will now tear down the state (i.e. restart the browser) between
each page/story of a test.

Perf sherfiffs warning: This CL is expected to affect measurements of
many benchmarks.

BUG= chromium:640990 

Review-Url: https://codereview.chromium.org/2304893002
Commit  : 4f812c9035e6cf9bad6774bb6ff71c1822167e0a
Date    : Sat Sep 03 10:45:26 2016


===== TESTED REVISIONS =====
Revision                             Mean      Std Dev     N   Good?
chromium@416300                      0.202018  0.0134501   8   good
chromium@416380                      0.203099  0.0162714   8   good
chromium@416420                      0.198785  0.0089443   8   good
chromium@416440                      0.202646  0.0321707   8   good
chromium@416450                      0.199923  0.0176291   12  good
chromium@416452                      0.19299   0.00505466  5   good
chromium@416452,catapult@4f812c9035  0.30098   0.0237625   5   bad    <--
chromium@416453                      0.277169  0.0481359   12  bad
chromium@416455                      0.295271  0.00978893  8   bad
chromium@416460                      0.289826  0.0365597   8   bad

Bisect job ran on: android_s5_perf_bisect
Bug ID: 645480

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests thread_times.key_silk_cases
Test Metric: thread_other_cpu_time_per_frame/thread_other_cpu_time_per_frame
Relative Change: 36.70%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_bisect/builds/1124
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8997725985918650208


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5813234068619264

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Sign in to add a comment