New issue
Advanced search Search tips

Issue 683378 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Feb 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

11% regression in blink_perf.bindings at 444048:444070

Project Member Reported by mustaq@chromium.org, Jan 20 2017

Issue description

See the link to graphs below.
 

Comment 1 by mustaq@chromium.org, Jan 20 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=683378

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4OPdsAoM


Bot(s) for this bug's original alert(s):

android-nexus6
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Jan 21 2017

Cc: horo@chromium.org
Owner: horo@chromium.org

=== Auto-CCing suspected CL author horo@chromium.org ===

Hi horo@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : horo
  Commit : 5e608bb301fdf93e000771648059026832f1d3d1
  Date   : Tue Jan 17 16:16:57 2017
  Subject: Add leaking service-worker-navigation-preload-disabled tests to LeakExpectations.

Bisect Details
  Configuration: android_nexus6_perf_bisect
  Benchmark    : blink_perf.bindings
  Metric       : id-getter/id-getter
  Change       : 11.68% | 148.156732164 -> 130.845907328

Revision             Result                   N
chromium@444047      148.157 +- 1.24339       6      good
chromium@444059      148.127 +- 0.85684       6      good
chromium@444062      142.435 +- 0.863646      6      good
chromium@444064      140.084 +- 0.974492      5      good
chromium@444065      130.83 +- 9.3612         6      bad       <--
chromium@444070      130.846 +- 9.06364       6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.bindings

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8989911042739281360

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5341608179924992


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 5 by horo@chromium.org, Jan 23 2017

Owner: ----
Changing LeakExpectations file must not cause any regression.
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Jan 23 2017


=== BISECT JOB RESULTS ===
Perf regression found but unable to narrow commit range

Bisect Details
  Configuration: android_nexus6_perf_bisect
  Benchmark    : blink_perf.bindings
  Metric       : id-getter/id-getter

Suspected Commit Range
  25 commits in range
  https://chromium.googlesource.com/chromium/src/+log/2ef6d3ba0eecb7bbc12671f83ac8adfea0b9c00b..b20f0039e91883b8f001390a2f21616b4ec5752d


Revision             Result                   N
chromium@444000      154.948 +- 0.584087      6      good
chromium@444025      154.645 +- 0.407762      6      good
chromium@444050      142.556 +- 0.634895      6      bad
chromium@444100      132.429 +- 8.65074       6      bad
chromium@444200      131.357 +- 6.47557       6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.bindings

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8989708259186904704

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6144283343585280


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 7 by mustaq@chromium.org, Jan 23 2017

Cc: haraken@chromium.org
haraken@: Any chance the following v8 update might have caused this regression?
https://chromium.googlesource.com/chromium/src/+/f71584603a72dacb606c3176f72b713856f49e29

Asking haraken@ as one of the test owners listed in go/chrome-benchmarks. If you are not the right one, do you know who owns this benchmark?

Comment 8 by mustaq@chromium.org, Jan 26 2017

Owner: haraken@chromium.org
Ping for updates.
Owner: hablich@chromium.org
I suspect this is caused by the V8 roll:

https://chromium.googlesource.com/chromium/src/+/81b6d1dd59e60cac3c08a9a1e694eb1755283290

hablich@: Would you mind taking a look?

Cc: hablich@chromium.org
Owner: jochen@chromium.org
What are those tests testing exactly? There are some parser changes in the range but I suppose they should not interfere with the actual speed of the bindings. Assigning to Jochen for further triage on this.
Cc: simonhatch@chromium.org
oh, and interesting that the bisect seems to produce non-actionable results?
Owner: haraken@chromium.org
No, I don't think this is parser related

Does this repro locally?

Assigning back to haraken@
Project Member

Comment 14 by 42576172...@developer.gserviceaccount.com, Jan 30 2017


=== BISECT JOB RESULTS ===
Perf regression found but unable to narrow commit range

Bisect Details
  Configuration: android_nexus6_perf_bisect
  Benchmark    : blink_perf.bindings
  Metric       : id-getter/id-getter

Suspected Commit Range
  21 commits in range
  https://chromium.googlesource.com/chromium/src/+log/9956266b9646fac12c3641d7612b7872a3fb73d6..81b6d1dd59e60cac3c08a9a1e694eb1755283290


Revision             Result                   N
chromium@444000      155.025 +- 1.06204       6      good
chromium@444043      150.747 +- 0.795031      6      good
chromium@444064      135.12 +- 9.01819        6      bad
chromium@444085      132.945 +- 8.14004       5      bad
chromium@444170      130.267 +- 8.1787        6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.bindings

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8989031327988688272

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5223121558175744


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Status: WontFix (was: Untriaged)
This metric seems to have recovered.
Project Member

Comment 17 by 42576172...@developer.gserviceaccount.com, Apr 11 2017


=== BISECT JOB RESULTS ===
Perf regression found but unable to narrow commit range

Bisect Details
  Configuration: android_nexus6_perf_bisect
  Benchmark    : blink_perf.bindings
  Metric       : id-getter/id-getter

Suspected Commit Range
  21 commits in range
  https://chromium.googlesource.com/chromium/src/+log/9956266b9646fac12c3641d7612b7872a3fb73d6..81b6d1dd59e60cac3c08a9a1e694eb1755283290


Revision             Result                   N
chromium@444000      155.025 +- 1.06204       6      good
chromium@444043      150.747 +- 0.795031      6      good
chromium@444064      135.12 +- 9.01819        6      bad
chromium@444085      132.945 +- 8.14004       5      bad
chromium@444170      130.267 +- 8.1787        6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.bindings

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982651680626760512

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5223121558175744


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Sign in to add a comment