New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 721413 link

Starred by 1 user

Issue metadata

Status: Started
Owner:
Last visit > 30 days ago
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug


Sign in to add a comment

Clean up v8 benchmarks

Project Member Reported by nedngu...@google.com, May 11 2017

Issue description

Plan:
_ Remove v8.google (already covered by v8.runtime_stats.top_25)
_ Remove v8.top_25_smooth & v8.key_mobile_sites_smooth (no longer watched by hpayer@ & we already have v8.browsing_* for real world coverage)
_ Migrate infinite scroll cases to system health browsing stories, then remove v8.infinite_scroll* & v8.mobile_infinite_scroll*.

 
Labels: -Pri-3 Pri-1
Owner: nedngu...@google.com
Status: Started (was: Untriaged)
Project Member

Comment 2 by bugdroid1@chromium.org, May 15 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/12b214a95fb98f5ce9eb412d28cda654f49b4d08

commit 12b214a95fb98f5ce9eb412d28cda654f49b4d08
Author: nednguyen <nednguyen@google.com>
Date: Mon May 15 15:34:26 2017

Remove v8.top_25_smooth & v8.key_mobile_sites_smooth benchmarks

These real world v8 benchmarks are now better covered by
v8.browsing_desktop & v8.browsing_mobile.

BUG=721413

Review-Url: https://codereview.chromium.org/2877803003
Cr-Commit-Position: refs/heads/master@{#471772}

[modify] https://crrev.com/12b214a95fb98f5ce9eb412d28cda654f49b4d08/testing/buildbot/chromium.perf.fyi.json
[modify] https://crrev.com/12b214a95fb98f5ce9eb412d28cda654f49b4d08/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/12b214a95fb98f5ce9eb412d28cda654f49b4d08/tools/perf/benchmark.csv
[modify] https://crrev.com/12b214a95fb98f5ce9eb412d28cda654f49b4d08/tools/perf/benchmarks/v8.py

Blockedon: 724262
Project Member

Comment 5 by bugdroid1@chromium.org, Jun 7 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b7b964bdc8be0f62c5a01483ed2edeeee44c0446

commit b7b964bdc8be0f62c5a01483ed2edeeee44c0446
Author: Ross McIlroy <rmcilroy@chromium.org>
Date: Wed Jun 07 16:36:54 2017

[Perf] Remove V8 Classic and Turbo variants.

The Ignition+Turbofan configuration of V8 has now shipped and the Classic
configuration is being removed. This CL removes all variants Classic and
Turbo variants for these benchmarks.

Removes the following benchmarks:
 - v8.infinite_scroll-turbo_tbmv2
 - v8.infinite_scroll-classic_tbmv2
 - v8.mobile_infinite_scroll-turbo_tbmv2
 - v8.mobile_infinite_scroll-classic_tbmv2
 - v8.browsing_desktop_turbo
 - v8.browsing_mobile_turbo
 - v8.browsing_desktop_classic
 - v8.browsing_mobile_classic
 - v8.runtimestats.browsing_desktop_turbo
 - v8.runtimestats.browsing_desktop_classic
 - v8.runtimestats.browsing_mobile_turbo
 - v8.runtimestats.browsing_mobile_classic
 - speedometer-turbo
 - speedometer-classic

BUG= chromium:639316 , chromium:721413

Change-Id: Ie2264c5ff133f43a23c0c0cc7959afd21739cdea
Reviewed-on: https://chromium-review.googlesource.com/527037
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#477676}
[modify] https://crrev.com/b7b964bdc8be0f62c5a01483ed2edeeee44c0446/testing/buildbot/chromium.perf.fyi.json
[modify] https://crrev.com/b7b964bdc8be0f62c5a01483ed2edeeee44c0446/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/b7b964bdc8be0f62c5a01483ed2edeeee44c0446/tools/perf/benchmark.csv
[modify] https://crrev.com/b7b964bdc8be0f62c5a01483ed2edeeee44c0446/tools/perf/benchmarks/speedometer.py
[modify] https://crrev.com/b7b964bdc8be0f62c5a01483ed2edeeee44c0446/tools/perf/benchmarks/v8.py
[modify] https://crrev.com/b7b964bdc8be0f62c5a01483ed2edeeee44c0446/tools/perf/benchmarks/v8_browsing.py
[modify] https://crrev.com/b7b964bdc8be0f62c5a01483ed2edeeee44c0446/tools/perf/benchmarks/v8_helper.py

Project Member

Comment 6 by bugdroid1@chromium.org, Jul 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/676fbd80584656f85a5f0f4b8b5c5e35784b74de

commit 676fbd80584656f85a5f0f4b8b5c5e35784b74de
Author: nednguyen <nednguyen@google.com>
Date: Tue Jul 11 13:31:12 2017

Remove v8 infinite scroll benchmarks

Now that infinite scroll stories are no longer disabled on v8.browsing*
benchmarks, we can remove v8 infinite scroll benchmarks.

Bug: 721413
Change-Id: I04433d1cac97fda219d309ac6fb46e37935c5019
Reviewed-on: https://chromium-review.googlesource.com/565804
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#485614}
[modify] https://crrev.com/676fbd80584656f85a5f0f4b8b5c5e35784b74de/testing/buildbot/chromium.perf.fyi.json
[modify] https://crrev.com/676fbd80584656f85a5f0f4b8b5c5e35784b74de/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/676fbd80584656f85a5f0f4b8b5c5e35784b74de/tools/perf/benchmark.csv
[modify] https://crrev.com/676fbd80584656f85a5f0f4b8b5c5e35784b74de/tools/perf/benchmarks/v8.py
[delete] https://crrev.com/3ebc815673812682541c6f4e7c9a53a7b010ee62/tools/perf/page_sets/data/infinite_scroll.json
[delete] https://crrev.com/3ebc815673812682541c6f4e7c9a53a7b010ee62/tools/perf/page_sets/data/infinite_scroll_000.wpr.sha1
[delete] https://crrev.com/3ebc815673812682541c6f4e7c9a53a7b010ee62/tools/perf/page_sets/data/infinite_scroll_001.wpr.sha1
[delete] https://crrev.com/3ebc815673812682541c6f4e7c9a53a7b010ee62/tools/perf/page_sets/data/infinite_scroll_002.wpr.sha1
[delete] https://crrev.com/3ebc815673812682541c6f4e7c9a53a7b010ee62/tools/perf/page_sets/data/infinite_scroll_003.wpr.sha1
[delete] https://crrev.com/3ebc815673812682541c6f4e7c9a53a7b010ee62/tools/perf/page_sets/data/infinite_scroll_004.wpr.sha1
[delete] https://crrev.com/3ebc815673812682541c6f4e7c9a53a7b010ee62/tools/perf/page_sets/data/mobile_infinite_scroll.json
[delete] https://crrev.com/3ebc815673812682541c6f4e7c9a53a7b010ee62/tools/perf/page_sets/data/mobile_infinite_scroll_000.wpr.sha1
[delete] https://crrev.com/3ebc815673812682541c6f4e7c9a53a7b010ee62/tools/perf/page_sets/data/mobile_infinite_scroll_001.wpr.sha1
[delete] https://crrev.com/3ebc815673812682541c6f4e7c9a53a7b010ee62/tools/perf/page_sets/infinite_scroll_cases.py

Blockedon: 690921
Cc: mythria@chromium.org
We are now down to 6 v8 benchmarks (used to be 21). They are:

v8.browsing_desktop
v8.browsing_mobile
(look like these two will be removed according to  issue 690921 )
v8.runtime_stats.top_25
v8.runtimestats.browsing_desktop
v8.runtimestats.browsing_mobile


Memory leak detection:
v8.detached_context_age_in_gc


Ulan: is it possible to only use memory.long_running_idle_gmail_tbmv2 for memory leak detection instead? 

This would get us to only 3 v8 benchmarks:
v8.runtime_stats.top_X <-- long tail of cases only v8 cares about
v8.runtimestats.browsing_desktop
 & v8.runtimestats.browsing_mobile
 <-- v8's tracking of system health stories

Project Member

Comment 8 by sheriffbot@chromium.org, Jul 17 2017

Labels: Hotlist-Google
Blockedon: 775635
Blockedon: 775639
The following two benchmarks are no longer active:

v8.runtimestats.browsing_desktop
v8.runtimestats.browsing_mobile

Which leaves us with:

v8.runtime_stats.top_25
v8.browsing_desktop
v8.browsing_desktop-future
v8.browsing_mobile
v8.browsing_mobile-future
v8.runtime_stats.top_25


We're currently in the making of the new v8.realworld_perf which will combine the runtime_stats and browsing versions into a single larger benchmark.

Thanks for the update, Camillo! It would be a great win in making v8 benchmarks simpler to understand & manage.
Owner: nednguyen@chromium.org

Sign in to add a comment