New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 597648 link

Starred by 0 users

Issue metadata

Status: Fixed
Owner:
Closed: Oct 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

15.6% regression in smoothness.key_silk_cases at 382798:382804

Project Member Reported by mustaq@chromium.org, Mar 24 2016

Issue description

The regression is ~5% if we ignore the noise.


 

Comment 1 by mustaq@chromium.org, Mar 24 2016

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=597648

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg0JyRrwoM


Bot(s) for this bug's original alert(s):

android-nexus6
Project Member

Comment 2 by 42576172...@developer.gserviceaccount.com, Mar 30 2016

Cc: abhishek...@samsung.com
Owner: abhishek...@samsung.com

=== Auto-CCing suspected CL author abhishek.ka@samsung.com ===

Hi abhishek.ka@samsung.com, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Handling transition animation between non-interpolable values.
Author  : abhishek.ka
Commit description:
  
In current transition animations, for any lengthsize css property
irrespective of interpolable values animation does start.
This patch will check whether the values are interpolable and decides
as follows:
i) If values are interpoable, start animation,
ii) else, hard set the respective length property to the element .
BUG= 513127 

Review URL: https://codereview.chromium.org/1732323003

Cr-Commit-Position: refs/heads/master@{#382801}
Commit  : 4d344058522e01d85162fcd1a9e5f4b747db9383
Date    : Wed Mar 23 06:48:44 2016


===== TESTED REVISIONS =====
Revision                Mean Value  Std. Dev.   Num Values  Good?
chromium@382796         22.466513   0.351476    8           good
chromium@382799         22.519844   0.278691    8           good
chromium@382800         22.549257   0.135953    5           good
chromium@382801         23.917014   0.580893    5           bad         <-
chromium@382802         24.022036   0.811915    8           bad

Bisect job ran on: android_nexus5_perf_bisect
Bug ID: 597648

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --also-run-disabled-tests smoothness.key_silk_cases
Test Metric: mean_frame_time/mean_frame_time
Relative Change: 6.41%
Score: 99.5

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/3532
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9016740551450334112


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=597648

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Mar 30 2016


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Handling transition animation between non-interpolable values.
Author  : abhishek.ka
Commit description:
  
In current transition animations, for any lengthsize css property
irrespective of interpolable values animation does start.
This patch will check whether the values are interpolable and decides
as follows:
i) If values are interpoable, start animation,
ii) else, hard set the respective length property to the element .
BUG= 513127 

Review URL: https://codereview.chromium.org/1732323003

Cr-Commit-Position: refs/heads/master@{#382801}
Commit  : 4d344058522e01d85162fcd1a9e5f4b747db9383
Date    : Wed Mar 23 06:48:44 2016


===== TESTED REVISIONS =====
Revision                Mean Value  Std. Dev.   Num Values  Good?
chromium@382796         21.472771   0.28817     5           good
chromium@382800         21.627679   0.38421     5           good
chromium@382801         25.418229   0.146991    5           bad         <-
chromium@382802         25.527629   0.282825    5           bad
chromium@382804         25.544293   0.382891    5           bad

Bisect job ran on: android_nexus7_perf_bisect
Bug ID: 597648

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --also-run-disabled-tests smoothness.key_silk_cases
Test Metric: mean_frame_time/mean_frame_time
Relative Change: 18.96%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/2868
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9016732565789198528


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=597648

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 4 by 42576172...@developer.gserviceaccount.com, Mar 31 2016


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Handling transition animation between non-interpolable values.
Author  : abhishek.ka
Commit description:
  
In current transition animations, for any lengthsize css property
irrespective of interpolable values animation does start.
This patch will check whether the values are interpolable and decides
as follows:
i) If values are interpoable, start animation,
ii) else, hard set the respective length property to the element .
BUG= 513127 

Review URL: https://codereview.chromium.org/1732323003

Cr-Commit-Position: refs/heads/master@{#382801}
Commit  : 4d344058522e01d85162fcd1a9e5f4b747db9383
Date    : Wed Mar 23 06:48:44 2016


===== TESTED REVISIONS =====
Revision                Mean Value  Std. Dev.   Num Values  Good?
chromium@382794         20.166336   0.139342    5           good
chromium@382799         20.517843   0.481617    5           good
chromium@382800         20.563193   0.474084    5           good
chromium@382801         23.708064   0.27903     5           bad         <-
chromium@382802         23.68895    0.414525    5           bad
chromium@382804         23.775943   0.349857    5           bad
chromium@382813         23.600079   0.308781    5           bad
chromium@382831         23.961421   0.455002    5           bad

Bisect job ran on: android_webview_aosp_perf_bisect
Bug ID: 597648

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --also-run-disabled-tests smoothness.key_silk_cases
Test Metric: mean_frame_time/mean_frame_time
Relative Change: 18.82%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_webview_aosp_perf_bisect/builds/84
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9016740565137039728


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=597648

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 5 by mustaq@chromium.org, Mar 31 2016

This caused 200% regression in ChromiumPerf/android-nexus9/smoothness.key_silk_cases/mean_frame_time!
Started looking into the regression.
Seems like http://jsfiddle.net/xLuvC/1/ is failing from key_silk_cases.
Will update frequently in this bug.
I find it hard to believe the code added in https://codereview.chromium.org/1732323003 adds a whole 3ms of computation per frame.
I think we need to compare traces (about:tracing) of the performance test with and without that patch to try to find out where that extra time is being spent.
Cc: alancutter@chromium.org
When i tried running the key_silk_cases test suite. Only few were passing the test(around 11).
With and without patch i observe the following readings for mean_frame_time:
With patch: 40.8 ms
Without patch: 41.154 ms

Tested on:
Nexus 7 - Cyanogen mode.
data.xlsx
8.5 KB Download
The data sheet of mean frame time for key_silk_cases are only for 10 tests out of 17 tests present in the key_silk_cases testsuite.
Remaining 7 tests requires special permission to run locally.(raises CloudStortageError).
Could you please point out which particular testcase is degraded so that i can debug further.
Alan could you respond to #10?
Alancutter@ ping. Could you respond to #10
Labels: Needs-Feedback
Labels: TE-Triaged
Alancutter@ : Gentle ping..!
Cc: rsch...@chromium.org
Ryan, can you help Abhishek per comment 10?
Labels: -performance-sheriff Performance-Sheriff
Owner: rsch...@chromium.org
Following up on #15, assigning to Ryan to help regarding comment #10.

Please reassign back to abhishek.ka@samsung.com once you've provided the required information.
Apologies that I missed this. Abishek, I've granted you access to the test data.
Friendly perf-sheriff ping, any update on this?
Another perf sheriff ping: abhishek.ka, any updates?

Comment 20 Deleted

Owner: alancutter@chromium.org
My apologies, typo in the email address.
@alancutter, abhishek doesn't have edit bugs access. Can you help here?

abhishek.ka, it looks like the permission issue was resolved. Can you take another look at this regression, please?

Status: Fixed (was: Assigned)
Apologies for leaving this one for so long.

This drop in mean_frame_time is due to the transition animation no longer being triggered in the http://jsfiddle.net/xLuvC/1/ test case after https://codereview.chromium.org/1732323003. See  crbug.com/616072  for bug details.

This bug was fixed in https://codereview.chromium.org/2027933002 which corresponds with the recovery of the specific perf test case.
Screenshot from 2016-10-06 09:23:15.png
270 KB View Download

Sign in to add a comment