New issue
Advanced search Search tips

Issue 605100 link

Starred by 0 users

Issue metadata

Status: WontFix
Owner:
Closed: Apr 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

3.6% regression in kraken at 388141:388156

Project Member Reported by rmcilroy@chromium.org, Apr 20 2016

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=605100

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgxJOsrAoM


Bot(s) for this bug's original alert(s):

linux-release
Project Member

Comment 2 by 42576172...@developer.gserviceaccount.com, Apr 20 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision                Mean Value  Std. Dev.   Num Values  Good?
chromium@388140         809.416667  7.399702    18          good
chromium@388156         808.444444  6.125282    18          bad

Bisect job ran on: linux_perf_bisect
Bug ID: 605100

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests kraken
Test Metric: Total/Total
Relative Change: 0.38%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6444
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9014862507165731744


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=605100

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Apr 20 2016


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : bindings: Makes the window object be the inner global object.
Author  : yukishiino
Commit description:
  
Removes the hidden prototypes from the window and
worker/worklet objects.  Makes these global objects be
the inner global object.

BUG= 587393 

Review URL: https://codereview.chromium.org/1858613002

Cr-Commit-Position: refs/heads/master@{#388170}
Commit  : 7aff51476ff48588879709275a3c363c6114e085
Date    : Tue Apr 19 09:19:35 2016


===== TESTED REVISIONS =====
Revision                Mean Value  Std. Dev.   Num Values  Good?
chromium@388121         819.92      8.169272    5           good
chromium@388165         810.458333  6.713143    12          good
chromium@388168         817.241667  18.869189   12          good
chromium@388169         811.98      7.450638    5           good
chromium@388170         830.15      8.852603    8           bad         <-
chromium@388171         828.38      9.871148    10          bad
chromium@388176         822.9       7.256721    5           bad
chromium@388187         830.88      6.70798     5           bad
chromium@388208         846.38      3.928995    5           bad

Bisect job ran on: linux_perf_bisect
Bug ID: 605100

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests kraken
Test Metric: Total/Total
Relative Change: 3.23%
Score: 90.0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6445
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9014862463556948464


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=605100

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Cc: rmcilroy@chromium.org
Owner: yukishiino@chromium.org
yukishiino@, could you take a look at this regression in Kraken please.
Cc: jochen@chromium.org verwa...@chromium.org
I'm sorry that I don't know what 'kraken' is.  What's that?

Also, is 3.6% regression severe for kraken?

The suspicious CL is a quite important change for V8-binding.  The CL makes it possible for V8 team to simplify and improve the V8 engine.  Expecting the coming improvement from V8, I'd go with the CL kept landed.

As long as it's not that severe and no objection, I'd close this issue as WontFix.
Cc: bmeu...@chromium.org mvstan...@chromium.org
Kraken is Mozilla's JavaScript benchmark suite - http://krakenbenchmark.mozilla.org/. Adding the test owners (Benedikt and Micheal) to assess whether we are happy to swallow this regression - Benedikt / Micheal, WDYT?
Ping - Michael / Benedikt.
I'm fine with that. I think it's worth it.
Status: WontFix (was: Assigned)
Great, thanks. Marking as wontfix

Sign in to add a comment