New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 633122 link

Starred by 3 users

Issue metadata

Status: WontFix
Owner:
Last visit > 30 days ago
Closed: Aug 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

15.8% regression in blink_perf.bindings at 408622:408635

Project Member Reported by alexclarke@chromium.org, Aug 1 2016

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=633122

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgptiKvwoM


Bot(s) for this bug's original alert(s):

linux-release
Cc: leon....@intel.com
Owner: leon....@intel.com

=== Auto-CCing suspected CL author leon.han@intel.com ===

Hi leon.han@intel.com, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : [LevelDBWrapper] Fix an issue that check condition is always true.
Author  : leon.han
Commit description:
  
item->value.is_null() is always true, should check it.second.is_null() instead.

BUG=

Review-Url: https://codereview.chromium.org/2182993002
Cr-Commit-Position: refs/heads/master@{#408627}
Commit  : e687684082856f27631233caf4a103b4709ef300
Date    : Fri Jul 29 13:38:25 2016


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@408621  269.025  9.19392  12  good
chromium@408625  273.063  8.69905  8   good
chromium@408626  260.413  7.60496  5   good
chromium@408627  284.586  5.72593  8   bad    <--
chromium@408628  283.517  1.13024  8   bad
chromium@408635  277.153  8.23294  12  bad

Bisect job ran on: linux_perf_bisect
Bug ID: 633122

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests blink_perf.bindings
Test Metric: node-list-access/node-list-access
Relative Change: 3.37%
Score: 99.0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6612
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005542060849570320


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5852624239198208

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Owner: alexclarke@chromium.org
That doesn't look right, trying another bisect.
Cc: panicker@chromium.org
Owner: panicker@chromium.org

=== Auto-CCing suspected CL author panicker@chromium.org ===

Hi panicker@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Pre-req for adding usage counter to chrome.loadtimes:
Author  : panicker
Commit description:
  Switch to SetAccessor, this adds a Getter where the Usage counter will be added subsequently.

A side effect of SetAccessor is that it subtly changes the  behavior: previously loadtimes data fields were mutable and persisted modifications that updated the value programmatically (from javascript).
With this change, such modifications will not be persisted.
We think this is fine -- as we cannot think of scenarios where it makes sense to modify the values returned by chrome.loadtimes. Moreover this is a necessary step to get usage counts and move the deprecation further.

BUG=621512

Review-Url: https://codereview.chromium.org/2149933003
Cr-Commit-Position: refs/heads/master@{#408226}
Commit  : 99807eeee0d8426d239376d716cacba2a84cfddf
Date    : Wed Jul 27 20:26:12 2016


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@408000  249.503  6.39017  27  good
chromium@408200  239.767  6.67753  27  good
chromium@408213  248.375  3.09271  5   good
chromium@408220  236.193  6.87255  5   good
chromium@408223  246.573  5.40995  5   good
chromium@408225  247.799  4.97973  5   good
chromium@408226  257.53   1.13754  5   bad    <--
chromium@408252  247.022  7.47314  27  bad
chromium@408300  248.894  5.61837  18  bad
chromium@408400  253.281  5.91669  27  bad
chromium@408800  253.996  5.3026   27  bad

Bisect job ran on: linux_perf_bisect
Bug ID: 633122

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests blink_perf.bindings
Test Metric: node-list-access/node-list-access
Relative Change: 4.50%
Score: 98.0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6613
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005530298024937936


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5877206484516864

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Is this about survery that's good because in fact is I'm trying to do what
I can if it's about the service and one thing I'm trying to get my things
again it's on the truck and tractors on and stays on the problem is people
hacking into myself I don't want them taking up the alarm but the point is
I want to be able to know that someone's going to somebody need to ask
permission first before you get any information thank you goodbye have a
wonderful day I want to get paid for my surveys for soon as possible thank
you have a nice day

Sanchezlasandra445@gmail.com
the test here (node-list-access) is child node traversal. However the implicated CL in loadtimes extension code will not be invoked unless there's an explicit call to loadTimes() -- which is not present in the test.
Therefore this CL shouldn't have an impact on the test.

I'm verifying one more thing, meanwhile feel free to re-run bisect and check other CLs in the range.

so we need to keep the box and the bugs out of it so that I can get into
it. so we need to keep the box and the bugs out of it so that I can get into

Sanchezlasandra445@gmail.com
Owner: alexclarke@chromium.org
Back to you Alex :)

BTW the bisent results listed here either don't seem to make sense OR I am misunderstanding what's listed.
The regression here is 15.8% *decrease* in Runs/second. Right?
However the mean value in the bisect here is increasing at suspected CL (and by 4.5% ?)
Also the value at subsequent CLs is back to earlier numbers.
Can you shed some light on this for my understanding?
(this is my first bisect bug :))

 

Cc: rsch...@chromium.org
#6 is indeed and improvement. The number of repeats suggest that the difference if any is really small. This seem like a case of 'If you torture the data long enough, it will confess.' I'd say that bisect is not being able to quite reproduce the regression from the original graph.

The error bars in the graph seem to jump a lot more than the mean, I doubt however that a std_dev bisect would work given the data shown in the previous bisect.
That bisect is clearly wonky!  I'll try another one but I'm not holding a lot of hope for finding it.
Cc: chongz@chromium.org
Owner: chongz@chromium.org

=== Auto-CCing suspected CL author chongz@chromium.org ===

Hi chongz@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Remove |TouchEvent.initTouchEvent()| in M54
Author  : chongz
Commit description:
  
|TouchEvent.initTouchEvent()| is not standardized and not interoperate
with Firefox or Safari. Developers should be encouraged to use the new
TouchEvent constructor instead.

|initTouchEvent()| was deprecated in M49, and this CL removes it
in M54 (deferred from M53).

Intent to Deprecate and Remove:
https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/dqlJguVuIHs

Chrome Status Entry:
https://www.chromestatus.com/features/5730982598541312

TBR=timvolodine
BUG= 522100 

Review-Url: https://codereview.chromium.org/2182183003
Cr-Commit-Position: refs/heads/master@{#408579}
Commit  : 44cc3124a56262d937b36a3d6d0d3de626082716
Date    : Fri Jul 29 05:32:43 2016


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@408500  266.235  7.02666  8   good
chromium@408550  266.275  12.5571  8   good
chromium@408575  270.404  10.0923  12  good
chromium@408578  271.384  6.38137  12  good
chromium@408579  285.958  3.79506  5   bad    <--
chromium@408580  281.494  5.12397  12  bad
chromium@408581  281.05   6.52869  8   bad
chromium@408587  277.926  8.44671  12  bad
chromium@408599  277.185  8.50443  12  bad
chromium@408700  280.403  7.84281  8   bad

Bisect job ran on: linux_perf_bisect
Bug ID: 633122

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests blink_perf.bindings
Test Metric: node-list-access/node-list-access
Relative Change: 4.01%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6616
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005435330116396304


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5857664114884608

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
My CL is just removing an API that already deprecated and shouldn't be used by anyone. I cannot image how it would cause trouble to blink_perf.bindings...?
Status: WontFix (was: Assigned)
I'm going to give upon this.  The bisects are not finding anything useful.
Thank you

Sanchezlasandra445@gmail.com

Sign in to add a comment