Migrate WebRTC Telemetry tests to timeline-based measurements (TBM) |
||||||||||||
Issue descriptionWebRTC Telemetry tests (https://cs.chromium.org/chromium/src/tools/perf/measurements/webrtc.py?l=14, etc) are currently written in the old style. Memory metrics in the old style are being deprecated, so we need to migrate to TBM (https://www.chromium.org/developers/telemetry/add_a_measurement#TOC-How-about-Timeline-Based-Measurement-).
,
Aug 11 2016
,
Aug 12 2016
Updating priority. It's blocking a P1 bug.
,
Sep 2 2016
Please note we've just announced the deprecation of the old memory metrics by November 4th. Do reach telemetry team in case of questions/concerns about the migration. https://groups.google.com/a/chromium.org/forum/#!topic/telemetry-announce/HSEYuAzHLSQ
,
Sep 5 2016
,
Sep 5 2016
Patrik: We should take a look at this :)
,
Sep 5 2016
Right. I think this won't be too hard so I'll take a look. Our telemetry tests need a bigger overhaul though so we may end up deleting them instead. Either way, that'll solve this problem.
,
Sep 9 2016
Ok, this will need a bit of non-trivial work. We essentially need to re-implement our metrics as tbmv2 metrics. I don't want to rush that, so I'll just remove the mem metrics for now to unblock 632021.
,
Sep 9 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/37df9608746ce0bebf92918e1d011e02b5243528 commit 37df9608746ce0bebf92918e1d011e02b5243528 Author: phoglund <phoglund@chromium.org> Date: Fri Sep 09 16:45:43 2016 Remove memory metrics from WebRTC measurements. We need to rewrite to tbm2 in order to get memory. We don't really look at memory metrics anyway at this point, so remove for now. BUG= 632295 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:android_s5_perf_cq;master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq Review-Url: https://codereview.chromium.org/2325153002 Cr-Commit-Position: refs/heads/master@{#417603} [modify] https://crrev.com/37df9608746ce0bebf92918e1d011e02b5243528/tools/perf/measurements/webrtc.py
,
Sep 16 2016
Edward, maybe you can look at migrating Telemetry to TBMv2 when time permits? It's a non-trivial project so this should probably be planned into Q4 OKRs.
,
Oct 4 2016
,
Oct 4 2016
,
Oct 4 2016
,
Oct 5 2016
,
Nov 30 2016
,
Mar 9 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/30557b1a00d4cf8daca845404ae6951c8089d01c commit 30557b1a00d4cf8daca845404ae6951c8089d01c Author: catapult-deps-roller <catapult-deps-roller@chromium.org> Date: Thu Mar 09 21:55:29 2017 Roll src/third_party/catapult/ 89b70b666..2ae07fc28 (2 commits) https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/89b70b666f2f..2ae07fc28bb3 $ git log 89b70b666..2ae07fc28 --date=short --no-merges --format='%ad %ae %s' 2017-03-09 ehmaldonado Add a TBMv2 webrtc_rendering_metric. 2017-03-09 ashleymarie Changing email from string to list of strings Created with: roll-dep src/third_party/catapult BUG= 632295 , 575318 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, see: http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel TBR=catapult-sheriff@chromium.org Review-Url: https://codereview.chromium.org/2740933003 Cr-Commit-Position: refs/heads/master@{#455875} [modify] https://crrev.com/30557b1a00d4cf8daca845404ae6951c8089d01c/DEPS
,
Mar 15 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e82554d88243ad84ee3329e7e17ddc6c22e8c250 commit e82554d88243ad84ee3329e7e17ddc6c22e8c250 Author: ehmaldonado <ehmaldonado@chromium.org> Date: Wed Mar 15 15:17:09 2017 WebRTC: Add a TBMv2 version of the webrtc.webrtc_smoothness benchmark. This adds the benchmark for the TBMv2 metric created at https://codereview.chromium.org/2711623002/ BUG= chromium:632295 , chromium:671121 Review-Url: https://codereview.chromium.org/2742293002 Cr-Commit-Position: refs/heads/master@{#457085} [modify] https://crrev.com/e82554d88243ad84ee3329e7e17ddc6c22e8c250/testing/buildbot/chromium.perf.fyi.json [modify] https://crrev.com/e82554d88243ad84ee3329e7e17ddc6c22e8c250/testing/buildbot/chromium.perf.json [modify] https://crrev.com/e82554d88243ad84ee3329e7e17ddc6c22e8c250/tools/perf/benchmarks/webrtc.py
,
May 17 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/db225968aa80fc5e947b2aa405555da8413794a9 commit db225968aa80fc5e947b2aa405555da8413794a9 Author: Edward Lemur <ehmaldonado@chromium.org> Date: Wed May 17 20:14:40 2017 Use a single benchmark for WebRTC telemetry tests. BUG= chromium:632295 NOTRY=true Change-Id: Icef34f9078a8b53d4f2023eb1d47365f17bb14b2 Reviewed-on: https://chromium-review.googlesource.com/506017 Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org> Reviewed-by: Henrik Kjellander <kjellander@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#472543} [modify] https://crrev.com/db225968aa80fc5e947b2aa405555da8413794a9/testing/buildbot/chromium.perf.fyi.json [modify] https://crrev.com/db225968aa80fc5e947b2aa405555da8413794a9/testing/buildbot/chromium.perf.json [modify] https://crrev.com/db225968aa80fc5e947b2aa405555da8413794a9/tools/perf/benchmark.csv [modify] https://crrev.com/db225968aa80fc5e947b2aa405555da8413794a9/tools/perf/benchmarks/webrtc.py [modify] https://crrev.com/db225968aa80fc5e947b2aa405555da8413794a9/tools/perf/core/benchmark_sharding_map.json [delete] https://crrev.com/7fe8085f9f4f10d88ec704588adea49392303fbe/tools/perf/measurements/webrtc.py [modify] https://crrev.com/db225968aa80fc5e947b2aa405555da8413794a9/tools/perf/page_sets/webrtc_cases.py [delete] https://crrev.com/7fe8085f9f4f10d88ec704588adea49392303fbe/tools/perf/page_sets/webrtc_track_peerconnections.js [modify] https://crrev.com/db225968aa80fc5e947b2aa405555da8413794a9/tools/perf/unowned_benchmarks.txt
,
Jul 6 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/2337207f30a90468192c0a21aa00a51e1ad3567a commit 2337207f30a90468192c0a21aa00a51e1ad3567a Author: ChromeOS Developer <ehmaldonado@chromium.org> Date: Thu Jul 06 04:54:58 2017 Update the control files for webrtc telemetry tests The benchmarks in Chromium were modified, so that all the webrtc.* benchmarks are part of a single webrtc benchmark now. BUG= chromium:632295 TEST=Not tested yet Change-Id: Idfd0aafd082200d7bd987a06a87d53194da1d8a8 Reviewed-on: https://chromium-review.googlesource.com/558908 Commit-Ready: Edward Lesmes <ehmaldonado@chromium.org> Tested-by: Edward Lesmes <ehmaldonado@chromium.org> Reviewed-by: Shuo-Peng Liao <deanliao@chromium.org> [copy] https://crrev.com/2337207f30a90468192c0a21aa00a51e1ad3567a/server/site_tests/telemetry_Benchmarks/control.webrtc.buddy.kepler [delete] https://crrev.com/240b4672a10cc1621f909bd42248bc15d9ca33ab/server/site_tests/telemetry_Benchmarks/control.webrtc.peerconnection [modify] https://crrev.com/2337207f30a90468192c0a21aa00a51e1ad3567a/server/site_tests/telemetry_Benchmarks/generate_controlfiles.py [rename] https://crrev.com/2337207f30a90468192c0a21aa00a51e1ad3567a/server/site_tests/telemetry_Benchmarks/control.webrtc [delete] https://crrev.com/240b4672a10cc1621f909bd42248bc15d9ca33ab/server/site_tests/telemetry_Benchmarks/control.webrtc.stress.buddy.kepler
,
Jan 24 2018
,
Aug 28
|
||||||||||||
►
Sign in to add a comment |
||||||||||||
Comment 1 by tkonch...@chromium.org
, Aug 5 2016