Add documentation for media benchmark harness |
|||||
Issue descriptionWe should add documentation for media benchmark harness. The documentation should have these informations: 1) What is the media benchmark? 2) What does it measure? How to interpret the metrics. 3) How to run the media benchmark 4) How to add more tests
,
Jun 18 2018
I can do this work. Where should I check it in? Where is the repo for it?
,
Jun 18 2018
Ah nevermind. I see you have a link to it.
,
Jun 28 2018
Maybe +dougman can do this. Doug, all the information that Ned needs is already documented, but it is in a few places: A. How to run the tests what the tests are for and lots of links are in go/videostack-sheriff B. Metrics are documented in https://cs.chromium.org/chromium/src/third_party/catapult/tracing/tracing/metrics/media_metric.html?q=media_metric&g=0&l=1 C. Test cases are listed and documented in https://cs.chromium.org/chromium/src/tools/perf/page_sets/media_cases.py D. Benchmark code is in https://cs.chromium.org/chromium/src/tools/perf/benchmarks/media.py?type=cs&q=media.desktop&g=0&l=78 After that information is collated into a single document, you make it a markdown file and put it in chromium/src/docs/speed/benchmark/harnesses/media.md. Reviewers should be crouleau, johnchen, and nednguyen. (If Doug can't do this, at least those links are available for me or the next person to do this now :) ) Reducing this to P2 since it isn't blocking any important work that I can tell (If it is, please add that as a blocking bug.)
,
Jun 29 2018
This actually blocks enabled guidance for Chrome-devs who have to deal with benchmark regressions (https://github.com/catapult-project/catapult/issues/3150#issuecomment-399118294), so I put this back to P1.
,
Jul 2
We should also add documentation about what to do when benchmark regresses. Example: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by nednguyen@chromium.org
, Jun 18 2018