New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 798370 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner: ----
Closed: Jan 2018
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

590.5% regression in smoothness.tough_animation_cases at 526194:526196

Project Member Reported by alexclarke@chromium.org, Jan 2 2018

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=798370

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=2bb912d975bd01de1488da9bcc9be2e4d54303f80a44cf60020907a7763da9a9


Bot(s) for this bug's original alert(s):

android-nexus5X

=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : smoothness.tough_animation_cases
  Metric       : frame_times/css_animations_many_keyframes.html?N_0316

Revision             Result                  N
chromium@526193      24.4285 +- 1.64198      21      good
chromium@526196      24.4844 +- 1.19715      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=css.animations.many.keyframes.html.N.0316 smoothness.tough_animation_cases

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8958513222526690032


For feedback, file a bug with component Speed>Bisection

=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : smoothness.tough_animation_cases
  Metric       : frame_times/css_animations_many_keyframes.html?N_0316

Revision             Result                   N
chromium@524000      24.6356 +- 0.646916      6       good
chromium@526196      24.6987 +- 1.22112       21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=css.animations.many.keyframes.html.N.0316 smoothness.tough_animation_cases

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8958417597822853216


For feedback, file a bug with component Speed>Bisection
Cc: rogerm@chromium.org
Owner: rogerm@chromium.org
Status: Assigned (was: Untriaged)

=== Auto-CCing suspected CL author rogerm@chromium.org ===

Hi rogerm@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Roger McFarlane
  Commit : f407f479ac2b02b2a28d46ed2ed4b2590ad790e2
  Date   : Wed Dec 20 21:31:39 2017
  Subject: [Sheriff] AwImeTest#testImeDpadMovesFocusOutOfWebView is flaky

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : smoothness.tough_animation_cases
  Metric       : frame_times/css_animations_many_keyframes.html?N_0316
  Change       : 1.54% | 24.8589113032 -> 24.5456249054

Revision             Result                   N
chromium@525000      24.8589 +- 0.986505      14      good
chromium@525299      24.8069 +- 1.03512       14      good
chromium@525448      24.7552 +- 1.16993       14      good
chromium@525457      24.6527 +- 0.569325      14      good
chromium@525462      24.7563 +- 0.706362      9       good
chromium@525464      24.633 +- 0.732251       14      good
chromium@525465      24.304 +- 1.09887        14      bad       <--
chromium@525490      24.4457 +- 1.04947       14      bad
chromium@525524      24.377 +- 1.05977        14      bad
chromium@525598      24.4709 +- 1.39143       21      bad
chromium@526196      24.5456 +- 0.660397      14      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=css.animations.many.keyframes.html.N.0316 smoothness.tough_animation_cases

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8958417578751601152


For feedback, file a bug with component Speed>Bisection

Comment 8 by rogerm@chromium.org, Jan 12 2018

Owner: rlanday@chromium.org
The "culprit" CL is a sheriff-disable of a flaky test, so is likely a red herring.

Bouncing to the test owner (rlanday@chromium.org) to see if this could indeed be related.
The "culprit" indeed seems unrelated.
Status: Available (was: Assigned)
Owner: ----
Cc: alancutter@chromium.org
Status: WontFix (was: Available)
+alancutter, owner of smoothness.tough_animation_cases: Looking at the graph, this metric is super noisy, to the point where it's randomly assigning perf bugs to developers. Can you look at fixing or disabling?

WontFix-ing the perf regression as it's noise.

Sign in to add a comment