The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/014643829588899aed0b7608feea34d8826df8b3

commit 014643829588899aed0b7608feea34d8826df8b3
Author: fmeawad <fmeawad@chromium.org>
Date: Wed Nov 16 18:53:28 2016

Initial implementation of V8 Top 25 Runtime Stats benchmark

This cl creates a story set of top 25+ V8 website consisting of mostly
top websites + some use cases the V8 team is interested to track.

This is a migration from an internal benchmark by the V8 team to
telemetry. It uses the newly created runtimeStatsMetric.

The runtime stats metrics aims at building a breakdown of the time spent
inside of V8 into the different V8 components. The time and count of
key functions in V8 is tracked individually, and then grouped by the
metric into top level V8 groups.

BUG= 664318 

Removing the perf CQ try bots.

if this benchmark fails, please contact fmeawad@ first.

Review-Url: https://codereview.chromium.org/2493953002
Cr-Commit-Position: refs/heads/master@{#432575}

[modify] https://crrev.com/014643829588899aed0b7608feea34d8826df8b3/tools/perf/benchmarks/v8.py
[add] https://crrev.com/014643829588899aed0b7608feea34d8826df8b3/tools/perf/page_sets/data/v8_top_25.json
[add] https://crrev.com/014643829588899aed0b7608feea34d8826df8b3/tools/perf/page_sets/data/v8_top_25_000.wpr.sha1
[add] https://crrev.com/014643829588899aed0b7608feea34d8826df8b3/tools/perf/page_sets/v8_top_25.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/57fa41b035585407fa354d139dc24ab6ed356a9a

commit 57fa41b035585407fa354d139dc24ab6ed356a9a
Author: dbeam <dbeam@chromium.org>
Date: Thu Nov 17 01:14:11 2016

Revert of Initial implementation of V8 Top 25 Runtime Stats benchmark (patchset #4 id:50001 of https://codereview.chromium.org/2493953002/ )

Reason for revert:
Broke telemetry_perf_unittests

https://build.chromium.org/p/chromium.win/builders/Win%207%20Tests%20x64%20(1)
https://build.chromium.org/p/chromium.win/builders/Win7%20Tests%20(1)
https://build.chromium.org/p/chromium.mac/builders/Mac10.11%20Tests

Original issue's description:
> Initial implementation of V8 Top 25 Runtime Stats benchmark
>
> This cl creates a story set of top 25+ V8 website consisting of mostly
> top websites + some use cases the V8 team is interested to track.
>
> This is a migration from an internal benchmark by the V8 team to
> telemetry. It uses the newly created runtimeStatsMetric.
>
> The runtime stats metrics aims at building a breakdown of the time spent
> inside of V8 into the different V8 components. The time and count of
> key functions in V8 is tracked individually, and then grouped by the
> metric into top level V8 groups.
>
> BUG= 664318 
>
> Removing the perf CQ try bots.
>
> if this benchmark fails, please contact fmeawad@ first.
>
> Committed: https://crrev.com/014643829588899aed0b7608feea34d8826df8b3
> Cr-Commit-Position: refs/heads/master@{#432575}

TBR=nednguyen@google.com,fmeawad@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 664318 

Review-Url: https://codereview.chromium.org/2505283002
Cr-Commit-Position: refs/heads/master@{#432679}

[modify] https://crrev.com/57fa41b035585407fa354d139dc24ab6ed356a9a/tools/perf/benchmarks/v8.py
[delete] https://crrev.com/e57ad133dccdc6e18b41f0b2574b88fa53372211/tools/perf/page_sets/data/v8_top_25.json
[delete] https://crrev.com/e57ad133dccdc6e18b41f0b2574b88fa53372211/tools/perf/page_sets/data/v8_top_25_000.wpr.sha1
[delete] https://crrev.com/e57ad133dccdc6e18b41f0b2574b88fa53372211/tools/perf/page_sets/v8_top_25.py

Landing this on Linux only for now since this is the only tested platform, and then we can expand from there.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a92247e2e3b3904e9717ec590617e83a78a46f47

commit a92247e2e3b3904e9717ec590617e83a78a46f47
Author: fmeawad <fmeawad@chromium.org>
Date: Tue Nov 22 21:07:55 2016

Reland of Initial implementation of V8 Top 25 Runtime Stats benchmark (patchset #1 id:1 of https://codereview.chromium.org/2505283002/ )

Reason for revert:
The crash was not due to the benchmark itself, instead it was due to a bug that was another bug that was introduced and fixed while the benchmark was landing. Fix landed at
https://codereview.chromium.org/2505813002/

CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq

Original issue's description:
> Revert of Initial implementation of V8 Top 25 Runtime Stats benchmark (patchset #4 id:50001 of https://codereview.chromium.org/2493953002/ )
>
> Reason for revert:
> Broke telemetry_perf_unittests
>
> https://build.chromium.org/p/chromium.win/builders/Win%207%20Tests%20x64%20(1)
> https://build.chromium.org/p/chromium.win/builders/Win7%20Tests%20(1)
> https://build.chromium.org/p/chromium.mac/builders/Mac10.11%20Tests
>
> Original issue's description:
> > Initial implementation of V8 Top 25 Runtime Stats benchmark
> >
> > This cl creates a story set of top 25+ V8 website consisting of mostly
> > top websites + some use cases the V8 team is interested to track.
> >
> > This is a migration from an internal benchmark by the V8 team to
> > telemetry. It uses the newly created runtimeStatsMetric.
> >
> > The runtime stats metrics aims at building a breakdown of the time spent
> > inside of V8 into the different V8 components. The time and count of
> > key functions in V8 is tracked individually, and then grouped by the
> > metric into top level V8 groups.
> >
> > BUG= 664318 
> >
> > Removing the perf CQ try bots.
> >
> > if this benchmark fails, please contact fmeawad@ first.
> >
> > Committed: https://crrev.com/014643829588899aed0b7608feea34d8826df8b3
> > Cr-Commit-Position: refs/heads/master@{#432575}
>
> TBR=nednguyen@google.com,fmeawad@chromium.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG= 664318 
>
> Committed: https://crrev.com/57fa41b035585407fa354d139dc24ab6ed356a9a
> Cr-Commit-Position: refs/heads/master@{#432679}

TBR=nednguyen@google.com,dbeam@chromium.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG= 664318 

Review-Url: https://codereview.chromium.org/2514423002
Cr-Commit-Position: refs/heads/master@{#433977}

[modify] https://crrev.com/a92247e2e3b3904e9717ec590617e83a78a46f47/tools/perf/benchmarks/v8.py
[add] https://crrev.com/a92247e2e3b3904e9717ec590617e83a78a46f47/tools/perf/page_sets/data/v8_top_25.json
[add] https://crrev.com/a92247e2e3b3904e9717ec590617e83a78a46f47/tools/perf/page_sets/data/v8_top_25_000.wpr.sha1
[add] https://crrev.com/a92247e2e3b3904e9717ec590617e83a78a46f47/tools/perf/page_sets/v8_top_25.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e433914989869a14fe15f6369fd3149a1b3b7eb8

commit e433914989869a14fe15f6369fd3149a1b3b7eb8
Author: fmeawad <fmeawad@chromium.org>
Date: Thu Dec 08 22:48:52 2016

[v8.runtime_stats] Update page-repeat to 3 for v8.runtime_stats.top_25

The benchmark has been running for a couple of weeks now and a noticable
noise is observed. We are changing the page-repeat to 3 to reduce the
noise.

Note to perf bot owners: This test currently runs on linux and mac,
with a cycle time of ~25 minutes. It is expected that this cycle time
will become 3x that number, i.e. ~1.5 hours.

R=nednguyen@google.com,
CC=cbruni@chromium.org,eyaich@chromium.org

BUG= 664318 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq

Review-Url: https://codereview.chromium.org/2556393003
Cr-Commit-Position: refs/heads/master@{#437379}

[modify] https://crrev.com/e433914989869a14fe15f6369fd3149a1b3b7eb8/tools/perf/benchmarks/v8.py

Could we enable the benchmarks for Win & Android too?

Assigning to Ned for further discussion.

Hmhh, this benchmark is currently disabled on Win & Android. We shouldn't mark the bug as fixed until it's working there.

I agree, what are the next steps?

The next steps is to debug why this benchmark is failing on Win & Android & fix the bug.

I am moving this benchmark from the old annotation style disabling to StoryExpectations. To re-enable this benchmark after that, you need to remove this line (and if nothing else is there, replace it with a simple 'pass'):

self.PermanentlyDisableBenchmark(
    [story.expectations.ALL_ANDROID, story.expectations.ALL_WIN],
    ' crbug.com/665318 ')

I will link this bug to it so you will know when that has actually landed. Is there any update to re-enabling this on android and windows?

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9c0d15c37eccbfe357e6cb544501fdaa9a728180

commit 9c0d15c37eccbfe357e6cb544501fdaa9a728180
Author: rnephew <rnephew@chromium.org>
Date: Thu Aug 17 18:21:06 2017

[Telemetry] Move remaining .Disabled annotations to story expectations.

The remaining ones are all 'Shouldnt run' cases such as adroid only
test shouldnt run on desktop platforms. Thats why they were not part
of the fixit.

Bug: 713222,  664318 
Change-Id: I50a5551c02ea91185e30ff348513c8edea10122b
Reviewed-on: https://chromium-review.googlesource.com/617364
Commit-Queue: rnephew <rnephew@chromium.org>
Reviewed-by: Charlie Andrews <charliea@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#495233}
[modify] https://crrev.com/9c0d15c37eccbfe357e6cb544501fdaa9a728180/testing/buildbot/chromium.perf.fyi.json
[modify] https://crrev.com/9c0d15c37eccbfe357e6cb544501fdaa9a728180/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/9c0d15c37eccbfe357e6cb544501fdaa9a728180/tools/perf/benchmarks/benchmark_unittest.py
[modify] https://crrev.com/9c0d15c37eccbfe357e6cb544501fdaa9a728180/tools/perf/benchmarks/loading.py
[modify] https://crrev.com/9c0d15c37eccbfe357e6cb544501fdaa9a728180/tools/perf/benchmarks/media.py
[modify] https://crrev.com/9c0d15c37eccbfe357e6cb544501fdaa9a728180/tools/perf/benchmarks/memory.py
[modify] https://crrev.com/9c0d15c37eccbfe357e6cb544501fdaa9a728180/tools/perf/benchmarks/oopif.py
[modify] https://crrev.com/9c0d15c37eccbfe357e6cb544501fdaa9a728180/tools/perf/benchmarks/oortonline.py
[modify] https://crrev.com/9c0d15c37eccbfe357e6cb544501fdaa9a728180/tools/perf/benchmarks/smoothness.py
[modify] https://crrev.com/9c0d15c37eccbfe357e6cb544501fdaa9a728180/tools/perf/benchmarks/start_with_url.py
[modify] https://crrev.com/9c0d15c37eccbfe357e6cb544501fdaa9a728180/tools/perf/benchmarks/startup.py
[modify] https://crrev.com/9c0d15c37eccbfe357e6cb544501fdaa9a728180/tools/perf/benchmarks/system_health.py
[modify] https://crrev.com/9c0d15c37eccbfe357e6cb544501fdaa9a728180/tools/perf/benchmarks/tab_switching.py
[modify] https://crrev.com/9c0d15c37eccbfe357e6cb544501fdaa9a728180/tools/perf/benchmarks/thread_times.py
[modify] https://crrev.com/9c0d15c37eccbfe357e6cb544501fdaa9a728180/tools/perf/benchmarks/v8.py
[modify] https://crrev.com/9c0d15c37eccbfe357e6cb544501fdaa9a728180/tools/perf/benchmarks/v8_browsing.py
[modify] https://crrev.com/9c0d15c37eccbfe357e6cb544501fdaa9a728180/tools/perf/page_sets/system_health/expectations.py

This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

This bug can be closed & Fadi did most of the work