New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 594179 link

Starred by 0 users

Issue metadata

Status: WontFix
Owner:
Closed: Apr 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

11% regression in blink_perf.paint at 379962:380038

Project Member Reported by qyears...@chromium.org, Mar 11 2016

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=594179

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICggL3MqgoM


Bot(s) for this bug's original alert(s):

android-nexus7v2
Project Member

Comment 2 by 42576172...@developer.gserviceaccount.com, Mar 16 2016


=== Auto-CCing suspected CL author jyan@ca.ibm.com ===

Hi jyan@ca.ibm.com, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : S390: Initial impl of regexp
Author  : jyan
Commit description:
  
R=danno@chromium.org,jkummerow@chromium.org,jochen@chromium.org,joransiu@ca.ibm.com,michael_dawson@ca.ibm.com,mbrandy@us.ibm.com
BUG=

Review URL: https://codereview.chromium.org/1768383002

Cr-Commit-Position: refs/heads/master@{#34592}
Commit  : be33a9d89da370aee73a8b31011ab5b1b46dd0d7
Date    : Tue Mar 08 15:08:18 2016


===== TESTED REVISIONS =====
Revision                Mean Value  Std. Dev.   Num Values  Good?
chromium@379961         221.737273  3.36089     5           good
chromium@380000         277.963727  1.833511    5           good
chromium@380020         264.761091  2.598476    5           good
chromium@380029         261.212386  4.805617    8           good
chromium@380030         266.580795  5.16        8           good
chromium@380030,v8@be33a9d89d249.652     3.260212    5           bad
chromium@380030,v8@26abfc5099252.5475    5.324358    8           bad
chromium@380030,v8@f7934b6427249.707455  2.95258     5           bad
chromium@380031         244.050818  4.311169    5           bad
chromium@380032         244.565545  4.012933    5           bad
chromium@380034         252.015795  2.500025    8           bad
chromium@380038         247.422557  6.421556    8           bad

Bisect job ran on: android_nexus7_perf_bisect
Bug ID: 594179

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --also-run-disabled-tests blink_perf.paint
Test Metric: large-table-background-change-with-invisible-collapsed-borders/large-table-background-change-with-invisible-collapsed-borders
Relative Change: 11.61%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/2842
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9018465181869393200


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with label Cr-Tests-AutoBisect.  Thank you!
Cc: j...@ca.ibm.com yangguo@chromium.org jkummerow@chromium.org
jyan, jkummerow, yangguo

Does it seem to make sense that that CL might have affected blink_perf.paint performance test results? Or was that likely a false positive bisect result?
Project Member

Comment 4 by 42576172...@developer.gserviceaccount.com, Mar 17 2016


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Allow client process information to be passed via Connector::Connect().
Author  : ben
Commit description:
  
This allows applications to start processes themselves and have the shell use them instead of starting its own.
Allows us to eliminate Shell::CreateInstance().

BUG=
CQ_INCLUDE_TRYBOTS=tryserver.chromium.linux:linux_site_isolation

Review URL: https://codereview.chromium.org/1776513003

Cr-Commit-Position: refs/heads/master@{#380030}
Commit  : 40b6dc773e28490f14eaea639261c82154d948a0
Date    : Wed Mar 09 01:55:28 2016


===== TESTED REVISIONS =====
Revision                Mean Value  Std. Dev.   Num Values  Good?
chromium@379961         223.103818  1.565114    5           good
chromium@380000         272.244091  2.577449    5           good
chromium@380020         260.587364  4.230978    5           good
chromium@380029         258.905492  4.06957     12          good
chromium@380030         266.405545  2.7379      5           bad
chromium@380031         247.91125   15.973079   8           bad
chromium@380032         247.914716  13.596667   8           bad
chromium@380034         248.678295  2.720761    8           bad
chromium@380038         243.69608   3.319551    8           bad

Bisect job ran on: android_nexus7_perf_bisect
Bug ID: 594179

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --also-run-disabled-tests blink_perf.paint
Test Metric: large-table-background-change-with-invisible-collapsed-borders/large-table-background-change-with-invisible-collapsed-borders
Relative Change: 9.94%
Score: 90.0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/2852
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9017924024017683776


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with label Cr-Tests-AutoBisect.  Thank you!
Re #3: That's clearly a false positive. S390 code isn't even compiled for ARM devices.

To me this generally looks like a case of "go home, bisect bot, you're drunk". Looking at the graph, less is better, and we're looking for the change that made the score go from 215-230 (good) to 245-270 (bad). Why does the bisect bot say that revision 380000 at a score of 272 is good? Or 380020 at 260? It seems quite obvious that the regression happened in the range 379961-380000, or am I missing something?

Comment 6 by dtu@chromium.org, Mar 22 2016

Cc: robert...@chromium.org
https://codereview.chromium.org/1814763004 landed 4 days ago to mitigate this issue. Please try again :)

(+robertocn@: Though I think we should implement the iteration equalizer as a more robust solution -- use the first n values of the first and last samples, where n == min(len(first), len(last)).)
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Mar 26 2016

Cc: trchen@chromium.org
Owner: trchen@chromium.org

=== Auto-CCing suspected CL author trchen@chromium.org ===

Hi trchen@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : [SPv2] Hookup overflow clip/scroll property nodes for normal flow
Author  : trchen
Commit description:
  
This CL adds scoped clip/scroll property in BoxClipper and BlockPainter.
It associates normal flow display items with the correct property nodes.

BUG= 537409 

Review URL: https://codereview.chromium.org/1652313003

Cr-Commit-Position: refs/heads/master@{#379995}
Commit  : be6997777d5f17e40a8fb4b68756624483ebd91e
Date    : Wed Mar 09 00:33:55 2016


===== TESTED REVISIONS =====
Revision                Mean Value  Std. Dev.   Num Values  Good?
chromium@379961         220.150545  2.716946    5           good
chromium@379981         217.615273  2.205747    5           good
chromium@379991         223.919818  6.268101    5           good
chromium@379994         231.910818  3.622393    5           good
chromium@379995         283.508273  2.505073    5           bad         <-
chromium@379996         283.338091  3.366389    5           bad
chromium@380000         274.280909  2.665617    5           bad
chromium@380038         245.778545  2.400002    5           bad

Bisect job ran on: android_nexus7_perf_bisect
Bug ID: 594179

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --also-run-disabled-tests blink_perf.paint
Test Metric: large-table-background-change-with-invisible-collapsed-borders/large-table-background-change-with-invisible-collapsed-borders
Relative Change: 11.64%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/2863
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9017100829498340208


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=594179

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Cc: nyerramilli@chromium.org
Labels: TE-Triaged
based on bisect results, trchen@ could you please check the issue.
It is highly unlikely because my whole thing is behind a flag. Let me kick off the bisect bot again...
Owner: nyerramilli@chromium.org
Eh, when I hit confirm that bisect job 594179 is incorrect, the web interface responded with "TryJob doesn't exist.", is it broken?
Owner: wangxianzhu@chromium.org
from owners spreadsheet adding test 'blink_perf.paint' owner wangxianzhu@, could you please check the issue.

===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : [SPv2] Hookup overflow clip/scroll property nodes for normal flow
Author  : trchen
Commit description:
  
This CL adds scoped clip/scroll property in BoxClipper and BlockPainter.
It associates normal flow display items with the correct property nodes.

BUG= 537409 

Review URL: https://codereview.chromium.org/1652313003

Cr-Commit-Position: refs/heads/master@{#379995}
Commit  : be6997777d5f17e40a8fb4b68756624483ebd91e
Date    : Wed Mar 09 00:33:55 2016


===== TESTED REVISIONS =====
Revision                Mean Value  Std. Dev.   Num Values  Good?
chromium@379700         220.958818  3.366942    5           good
chromium@379925         220.806364  5.273038    5           good
chromium@379982         218.365545  3.808369    5           good
chromium@379989         221.197     4.397461    5           good
chromium@379993         218.788455  2.106116    5           good
chromium@379994         230.771182  3.37513     5           good
chromium@379995         284.887091  2.065701    5           bad         <-
chromium@379996         284.081909  3.958801    5           bad
chromium@380010         269.545818  3.011972    5           bad
chromium@380038         245.422455  5.544405    5           bad
chromium@380150         254.683455  3.507165    5           bad

Bisect job ran on: android_nexus7_perf_bisect
Bug ID: 594179

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --also-run-disabled-tests blink_perf.paint
Test Metric: large-table-background-change-with-invisible-collapsed-borders/large-table-background-change-with-invisible-collapsed-borders
Relative Change: 15.26%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/2898
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9015924072966352832


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=594179

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Status: WontFix (was: Assigned)
Still can't find the actual reason. Perhaps CL just changed memory distribution and affected cache performance. Nothing we can do for now.

Sign in to add a comment