Upload perf results to cloud storage and surface links in buildbot |
|||
Issue descriptionWe need a way to access perf dashboard results once all of our benchmarks are running under one buildbot step. In buildbot today you can see links to the json.ouput under the Dashboard Upload step: https://uberchromegw.corp.google.com/i/chromium.perf/builders/Mac%20Pro%2010.11%20Perf/builds/2151 We do this using api.json.output() in steps.py: https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/chromium_tests/steps.py?q=chromium_tests/steps&sq=package:chromium&l=1883 Now that we are uploading all benchmarks at once during swarming collect step, we need to surface the dashboard results back to buildbot so they can be consumed for debugging. Steps are as follows: 1) Upload all results to cloud storage (chrome-telemetry-bucket) or generate logdog streams for each one. More investigation has to be done here to determine what is the best mechanism for storing these results. 2) Generate a json mapping from benchmark to both results url and the uploaded perf results data. Create a logdog stream of this. 3) Pass this back through in a links section of the collect step: https://cs.chromium.org/chromium/src/third_party/skia/infra/bots/recipe_modules/swarming/resources/collect_task.py?q=%22No+shard+json+files+found+in%22&sq=package:chromium&dr=C&l=48 Users will now have to index into this to find the results link of the perf dashboard data.
,
Feb 26 2018
Do we really want to store all of our perf results in cloud storage though? It seems like logdog streams might be a wiser choice for temporary storage.
,
Feb 26 2018
Ah, I just point that out there as an escape hatch option in case it's unclear how to integrate with logdog stream. I agree that logdog stream is the better option :-)
,
Mar 28 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/64fca1a28c7252db176f661c39f0bd331aa7f0ca commit 64fca1a28c7252db176f661c39f0bd331aa7f0ca Author: Ashley Enstad <ashleymarie@chromium.org> Date: Wed Mar 28 22:02:24 2018 Generate logdog streams for perf results 1) Generate logdog stream for each perf result 2) Generate a json mapping from benchmark to results url and uploaded perf results data 3) Add link to 2) to the links section of the collect step Bug: 816458 Change-Id: I1b57121881a81269d75202c88f3806fd85b0dcb6 Reviewed-on: https://chromium-review.googlesource.com/978543 Commit-Queue: Ashley Enstad <ashleymarie@chromium.org> Reviewed-by: Emily Hanley <eyaich@chromium.org> Cr-Commit-Position: refs/heads/master@{#546613} [modify] https://crrev.com/64fca1a28c7252db176f661c39f0bd331aa7f0ca/tools/perf/core/results_dashboard.py [modify] https://crrev.com/64fca1a28c7252db176f661c39f0bd331aa7f0ca/tools/perf/core/upload_results_to_perf_dashboard.py [modify] https://crrev.com/64fca1a28c7252db176f661c39f0bd331aa7f0ca/tools/perf/process_perf_results.py
,
Mar 30 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8e4effc87ca4f285ee6f34b2bd37ede0f17f87da commit 8e4effc87ca4f285ee6f34b2bd37ede0f17f87da Author: Ashley Enstad <ashleymarie@chromium.org> Date: Fri Mar 30 12:29:59 2018 Marking which test had a failed upload in the results Bug: 816458 Change-Id: I279266201ecb7f7ec94cef378816c9d55d527bc7 Reviewed-on: https://chromium-review.googlesource.com/985957 Reviewed-by: Emily Hanley <eyaich@chromium.org> Commit-Queue: Emily Hanley <eyaich@chromium.org> Cr-Commit-Position: refs/heads/master@{#547145} [modify] https://crrev.com/8e4effc87ca4f285ee6f34b2bd37ede0f17f87da/tools/perf/process_perf_results.py
,
Apr 10 2018
I was getting ready to close this bug and found an interesting issue! https://ci.chromium.org/buildbot/chromium.perf.fyi/One%20Buildbot%20Step%20Test%20Builder/4868 Looks like we can't reuse the same logdog file name (Results_Dashboard) if there are multiple steps since the url generated is the exact same https://logs.chromium.org/v/?s=chrome%2Fbb%2Fchromium.perf.fyi%2FOne_Buildbot_Step_Test_Builder%2F4868%2F%2B%2FResults_Dashboard https://logs.chromium.org/v/?s=chrome%2Fbb%2Fchromium.perf.fyi%2FOne_Buildbot_Step_Test_Builder%2F4868%2F%2B%2FResults_Dashboard I'll update this naming now and see if I can get those fully functional :)
,
Apr 12 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f44d34181cd05685ddab0d813414c21d8d151dfc commit f44d34181cd05685ddab0d813414c21d8d151dfc Author: Ashley Enstad <ashleymarie@chromium.org> Date: Thu Apr 12 15:15:50 2018 Adding unique id to Dashboard Url upload Bug: 816458 Change-Id: Iba29c656f02789baa8d0f9153bfaabe0fc1121e8 Reviewed-on: https://chromium-review.googlesource.com/1005032 Commit-Queue: Ashley Enstad <ashleymarie@chromium.org> Reviewed-by: Emily Hanley <eyaich@chromium.org> Cr-Commit-Position: refs/heads/master@{#550205} [modify] https://crrev.com/f44d34181cd05685ddab0d813414c21d8d151dfc/tools/perf/process_perf_results.py
,
Apr 17 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f44d34181cd05685ddab0d813414c21d8d151dfc commit f44d34181cd05685ddab0d813414c21d8d151dfc Author: Ashley Enstad <ashleymarie@chromium.org> Date: Thu Apr 12 15:15:50 2018 Adding unique id to Dashboard Url upload Bug: 816458 Change-Id: Iba29c656f02789baa8d0f9153bfaabe0fc1121e8 Reviewed-on: https://chromium-review.googlesource.com/1005032 Commit-Queue: Ashley Enstad <ashleymarie@chromium.org> Reviewed-by: Emily Hanley <eyaich@chromium.org> Cr-Commit-Position: refs/heads/master@{#550205} [modify] https://crrev.com/f44d34181cd05685ddab0d813414c21d8d151dfc/tools/perf/process_perf_results.py
,
Aug 3
This bug has an owner, thus, it's been triaged. Changing status to "assigned". |
|||
►
Sign in to add a comment |
|||
Comment 1 by nednguyen@chromium.org
, Feb 26 2018