New issue
Advanced search Search tips

Issue 747796 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Oct 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocking:
issue 690921



Sign in to add a comment

A zero-to-nonzero to 2.7% regression in v8.runtimestats.browsing_mobile at 486074:487676

Project Member Reported by bmeu...@chromium.org, Jul 24 2017

Issue description

See the link to graphs below.
 
Project Member

Comment 1 by 42576172...@developer.gserviceaccount.com, Jul 24 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=747796

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=e67fb2362604da87014801afe702305b7a7fb3e8faf5d481327c5c1e0de4d0e6


Bot(s) for this bug's original alert(s):

android-nexus5
android-nexus5X
android-nexus6
android-nexus7v2
android-one
android-webview-nexus5X
android-webview-nexus6
chromium-rel-mac-retina
chromium-rel-mac11
chromium-rel-mac11-air
chromium-rel-mac11-pro
chromium-rel-mac12
chromium-rel-mac12-mini-8gb
chromium-rel-win10
chromium-rel-win7-dual
chromium-rel-win7-gpu-ati
chromium-rel-win7-gpu-intel
chromium-rel-win7-gpu-nvidia
chromium-rel-win7-x64-dual
chromium-rel-win8-dual
linux-release
Project Member

Comment 4 by 42576172...@developer.gserviceaccount.com, Jul 24 2017

Cc: mythria@chromium.org
Owner: mythria@chromium.org

=== Auto-CCing suspected CL author mythria@chromium.org ===

Hi mythria@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Mythri Alle
  Commit : 51e582272983e7b49f395ccce1f3c56e1dc2d722
  Date   : Mon Jul 17 14:39:35 2017
  Subject: Enable memory metric, v8.compile category on v8.runtimestats.browse*

Bisect Details
  Configuration: android_webview_arm64_aosp_perf_bisect
  Benchmark    : v8.runtimestats.browsing_mobile
  Metric       : V8-Only:duration_avg/browse_media/browse_media_facebook_photos
  Change       : 6.55% | 4460.6335 -> 4752.58416667

Revision             Result                  N
chromium@487037      4460.63 +- 181.71       6      good
chromium@487057      4564.49 +- 258.079      9      good
chromium@487067      4534.7 +- 171.583       6      good
chromium@487072      4566.13 +- 99.0443      6      good
chromium@487074      4544.68 +- 145.91       6      good
chromium@487075      4771.57 +- 61.5535      6      bad       <--
chromium@487076      4752.58 +- 149.953      6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.media.facebook.photos v8.runtimestats.browsing_mobile

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8973207824123037616


For feedback, file a bug with component Speed>Bisection
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Jul 24 2017

Cc: mlippautz@chromium.org
 Issue 746252  has been merged into this issue.
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Jul 24 2017

Cc: jarin@google.com jarin@chromium.org
 Issue 747811  has been merged into this issue.
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Jul 24 2017

 Issue 747811  has been merged into this issue.
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Jul 24 2017

 Issue 746252  has been merged into this issue.
Cc: u...@chromium.org cbruni@chromium.org
I am not sure why enabling v8.compile should cause so many regressions. I am investigating it and update the bug.
Blocking: 690921
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Jul 24 2017


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Mythri Alle
  Commit : 51e582272983e7b49f395ccce1f3c56e1dc2d722
  Date   : Mon Jul 17 14:39:35 2017
  Subject: Enable memory metric, v8.compile category on v8.runtimestats.browse*

Bisect Details
  Configuration: mac_pro_perf_bisect
  Benchmark    : v8.runtimestats.browsing_desktop
  Metric       : Compile:duration_avg/browse_media/browse_media_pinterest
  Change       : 49.46% | 327.9705 -> 490.181666667

Revision             Result                  N
chromium@487067      327.97 +- 93.617        6      good
chromium@487071      359.639 +- 109.725      6      good
chromium@487073      370.535 +- 114.404      6      good
chromium@487074      313.116 +- 13.4125      6      good
chromium@487075      474.552 +- 13.6047      6      bad       <--
chromium@487083      465.116 +- 33.9788      6      bad
chromium@487098      530.198 +- 151.278      6      bad
chromium@487128      489.625 +- 120.93       6      bad
chromium@487189      484.629 +- 115.15       6      bad
chromium@487310      490.182 +- 128.368      6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.media.pinterest v8.runtimestats.browsing_desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8973191976813385248


For feedback, file a bug with component Speed>Bisection
Project Member

Comment 13 by 42576172...@developer.gserviceaccount.com, Jul 25 2017

 Issue 746252  has been merged into this issue.
Project Member

Comment 14 by 42576172...@developer.gserviceaccount.com, Jul 25 2017


=== BISECT JOB RESULTS ===
Bisect failed for unknown reasons

Please contact the team (see below) and report the error.


Bisect Details
  Configuration: win_x64_perf_bisect
  Benchmark    : v8.runtimestats.browsing_desktop
  Metric       : total:500ms_window:renderer_eqt:v8:compile_max/total:500ms_window:renderer_eqt:v8:compile_max


To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.runtimestats.browsing_desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8973207885713694864


For feedback, file a bug with component Speed>Bisection
Status: Assigned (was: Untriaged)
It turns out that in the compile bucket, the events tracked run for a very short time on many of the pages. For example, in pinterest the average time for compiling bytecode is ~15 microseconds. Since there will be one trace event for each of these tasks, we can notice the tracing overhead. It is around 15-20% on my desktop. It is also the same case on several other pages. 

Further, v8.compile events track the same events tracked by runtime call stats. So, I think it is better to implement eqt metric for compile, parse and optimize events using RCS instead of using v8.compile events. 

ulan@ pointed that for GC we cannot move to RCS because of the following reasons:
1. With the current implementation of RCS it is not possible to distinguish between latency-GC and memory-reducing-GC. ( https://cs.chromium.org/chromium/src/third_party/catapult/tracing/tracing/metrics/v8/utils.html?rcl=b5d2ffa3c55ad3cb4e92e8b79e42d24361755c91&l=149)

2. It would be tricky to filter out forced GCs (https://cs.chromium.org/chromium/src/third_party/catapult/tracing/tracing/metrics/v8/utils.html?rcl=b5d2ffa3c55ad3cb4e92e8b79e42d24361755c91&l=85).

3. We don't have all events in v8.gc tracked by RCS. 

So, as a midterm solution we will use RCS for calculating EQT due to Compile, Parse and other events and continue using v8.gc events for the GC related EQT. In the long term we may want to move v8.gc to RCS as well to maintain consistency. 
For now the original cl that enabled v8.compile on v8.runtimestats benchmark is reverted. (https://chromium.googlesource.com/chromium/src.git/+/bc93fdaf7aa3a0ba84e0c1f3802386484f86fc0c)
This is fixed by this cl: https://chromium-review.googlesource.com/725728. With this cl, we use RCS for measuring jank due to compile, parse, optimize times. So we don't need to enable v8.compile trace events.
Status: Fixed (was: Assigned)

Sign in to add a comment