New issue
Advanced search Search tips

Issue 853749 link

Starred by 2 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

Add documentation for media benchmark harness

Project Member Reported by nednguyen@chromium.org, Jun 18 2018

Issue description

We should add documentation for media benchmark harness. The documentation should have these informations:
1) What is the media benchmark?
2) What does it measure? How to interpret the metrics.
3) How to run the media benchmark
4) How to add more tests


 
I can do this work. Where should I check it in? Where is the repo for it?
Owner: crouleau@chromium.org
Status: Assigned (was: Untriaged)
Ah nevermind. I see you have a link to it.
Labels: -Pri-1 Pri-2
Owner: dougman@chromium.org
Maybe +dougman can do this. Doug, all the information that Ned needs is already documented, but it is in a few places:

A. How to run the tests what the tests are for and lots of links are in go/videostack-sheriff
B. Metrics are documented in https://cs.chromium.org/chromium/src/third_party/catapult/tracing/tracing/metrics/media_metric.html?q=media_metric&g=0&l=1
C. Test cases are listed and documented in https://cs.chromium.org/chromium/src/tools/perf/page_sets/media_cases.py
D. Benchmark code is in https://cs.chromium.org/chromium/src/tools/perf/benchmarks/media.py?type=cs&q=media.desktop&g=0&l=78

After that information is collated into a single document, you make it a markdown file and put
it in chromium/src/docs/speed/benchmark/harnesses/media.md. Reviewers should be crouleau, johnchen,
and nednguyen.

(If Doug can't do this, at least those links are available for me or the next person to do this now :) )

Reducing this to P2 since it isn't blocking any important work that I can tell (If it is, please add that as a blocking bug.)
Labels: -Pri-2 Pri-1
This actually blocks enabled guidance for Chrome-devs who have to deal with benchmark regressions (https://github.com/catapult-project/catapult/issues/3150#issuecomment-399118294), so I put this back to P1.
Cc: sullivan@chromium.org
We should also add documentation about what to do when benchmark regresses. Example: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

Sign in to add a comment