Make sure benchmarks documentation are surfaced on the perf regression/pinpoint reports |
|||||
Issue descriptionNow some of the benchmarks are properly documented (see https://docs.google.com/spreadsheets/d/1xaAo0_SU3iDfGdqDJZX_jRV0QtkufwHUKH3kQKF3YQs/edit#gid=0), we should make sure that perf dashboard & pinpoint surface them in the regression report.
,
Jul 16
Where do we want to surface these, just in the bug comment or also on the Pinpoint page?
,
Jul 16
I think we can start with the bug comment first.
,
Jul 16
+agrieve Note that We'd probably want to put these in comment #1 when the bug is filed, so that they work for binary size regressions which generally are not bisected. It would be fine to be redundant here and also put a comment in the completed bisect like "this regression was detected in the system_health.memory_mobile benchmark. For details on debugging this benchmark, see <link>"
,
Jul 17
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/592cb67990bc59f6bcb7d48a8ef23326afbd7168 commit 592cb67990bc59f6bcb7d48a8ef23326afbd7168 Author: Simon <simonhatch@chromium.org> Date: Tue Jul 17 19:22:39 2018 Dashboard - Whitelist documentation urls in /add_histograms Will need to follow up with actually using them, but for now at least save them. Bug: chromium:863861 Change-Id: I8a0305f2e5b9805af0cf71e01ceade4e40226003 Reviewed-on: https://chromium-review.googlesource.com/1140778 Reviewed-by: Ethan Kuefner <eakuefner@chromium.org> Commit-Queue: Simon Hatch <simonhatch@chromium.org> [modify] https://crrev.com/592cb67990bc59f6bcb7d48a8ef23326afbd7168/dashboard/dashboard/add_histograms.py
,
Jul 18
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ee1d7d6a10afc8d71253cb849ed508f706cf92a9 commit ee1d7d6a10afc8d71253cb849ed508f706cf92a9 Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Wed Jul 18 07:21:37 2018 Roll src/third_party/catapult f5342c4cf3d3..e8dc05ccba6b (9 commits) https://chromium.googlesource.com/catapult.git/+log/f5342c4cf3d3..e8dc05ccba6b git log f5342c4cf3d3..e8dc05ccba6b --date=short --no-merges --format='%ad %ae %s' 2018-07-18 nednguyen@google.com Revert "Handle stale directories when pushing files" 2018-07-18 nednguyen@google.com Implement screenshot capturing at system level on Linux 2018-07-17 dtu@chromium.org [pinpoint] Case-insensitive compare for Webview. 2018-07-17 simonhatch@chromium.org Dashboard - Whitelist documentation urls in /add_histograms 2018-07-17 cphlipot0@gmail.com [Systrace] Raise error when specified device is not found 2018-07-17 nednguyen@google.com Add test that verifies the way Telemetry generate skipped entries for filtered stories when benchmark is interrupted 2018-07-17 xiaochengh@chromium.org Disable WebPageReplayGoServerTest.testSmokeStartingWebPageReplayGoServer on Windows 2018-07-16 dtu@chromium.org [pinpoint] Update list of bots available in Pinpoint. 2018-07-16 bsheedy@chromium.org Handle stale directories when pushing files Created with: gclient setdep -r src/third_party/catapult@e8dc05ccba6b The AutoRoll server is located here: https://catapult-roll.skia.org Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG= chromium:862845 , chromium:862996 , chromium:863861 , chromium:860091 ,chromium:864294, chromium:863995 , chromium:862845 TBR=sullivan@chromium.org Change-Id: I5cab2a67db2dfee8c71f9ed7e445b82a9df41aec Reviewed-on: https://chromium-review.googlesource.com/1141465 Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#575965} [modify] https://crrev.com/ee1d7d6a10afc8d71253cb849ed508f706cf92a9/DEPS
,
Jul 25
,
Aug 22
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/6743e7b08a0d395192bceb46dd4174e8c4c222d9 commit 6743e7b08a0d395192bceb46dd4174e8c4c222d9 Author: Simon <simonhatch@chromium.org> Date: Wed Aug 22 22:34:04 2018 Pinpoint - Include doc link in bug comment. Bug: chromium:863861 Change-Id: I4b40eeabd188acf79305f2acb28a583207cb3d16 Reviewed-on: https://chromium-review.googlesource.com/1183986 Commit-Queue: Simon Hatch <simonhatch@chromium.org> Reviewed-by: David Tu <dtu@chromium.org> [modify] https://crrev.com/6743e7b08a0d395192bceb46dd4174e8c4c222d9/dashboard/dashboard/pinpoint/models/job.py [modify] https://crrev.com/6743e7b08a0d395192bceb46dd4174e8c4c222d9/dashboard/dashboard/pinpoint/models/job_test.py
,
Aug 23
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/d95dde6e066fe1b0ef88d3e0e5ccb5cae48b05a9 commit d95dde6e066fe1b0ef88d3e0e5ccb5cae48b05a9 Author: Simon <simonhatch@chromium.org> Date: Thu Aug 23 00:56:31 2018 Dashboard - List out docs for alerts. This prints a doc link for each suite in the list of alerts when a bug is filed. Will append a footer to the existing comment like: system_health.memory_mobile - Documentation links: http://system_health_docs loading.benchmark - Documentation links: http://foo/loading etc Additionally, have another cl up for Pinpoint to add a similar comment when it finishes a job, to include the specific docs for the suite it bisected. Bug: chromium:863861 Change-Id: I7d5b1e20515323e21a40521832e4a6e9533dc480 Reviewed-on: https://chromium-review.googlesource.com/1185206 Commit-Queue: Simon Hatch <simonhatch@chromium.org> Reviewed-by: Annie Sullivan <sullivan@chromium.org> [modify] https://crrev.com/d95dde6e066fe1b0ef88d3e0e5ccb5cae48b05a9/dashboard/dashboard/file_bug_test.py [modify] https://crrev.com/d95dde6e066fe1b0ef88d3e0e5ccb5cae48b05a9/dashboard/dashboard/file_bug.py
,
Aug 23
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c35cb7b095f3b20318cf38aa5ff0f035dd6f488e commit c35cb7b095f3b20318cf38aa5ff0f035dd6f488e Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Thu Aug 23 03:20:32 2018 Roll src/third_party/catapult c67a80df2679..d95dde6e066f (5 commits) https://chromium.googlesource.com/catapult.git/+log/c67a80df2679..d95dde6e066f git log c67a80df2679..d95dde6e066f --date=short --no-merges --format='%ad %ae %s' 2018-08-23 simonhatch@chromium.org Dashboard - List out docs for alerts. 2018-08-22 benjhayden@chromium.org Remove SparseDiagnostic.GetMostRecentValuesByNames. 2018-08-22 simonhatch@chromium.org Pinpoint - Include doc link in bug comment. 2018-08-22 benjhayden@chromium.org Set default_expiration: 20h in v2spa.yaml. 2018-08-22 benjhayden@chromium.org Allow passing custom version to deploy_v2spa. Created with: gclient setdep -r src/third_party/catapult@d95dde6e066f The AutoRoll server is located here: https://catapult-roll.skia.org Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG= chromium:863861 , chromium:876082 , chromium:863861 TBR=sullivan@chromium.org Change-Id: Ib326be155ae9fb40b064de9512abbef02e4a23a7 Reviewed-on: https://chromium-review.googlesource.com/1185691 Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#585384} [modify] https://crrev.com/c35cb7b095f3b20318cf38aa5ff0f035dd6f488e/DEPS
,
Aug 27
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/5d0291214a16b137a2c1ba9e67044402ffd71905 commit 5d0291214a16b137a2c1ba9e67044402ffd71905 Author: Dave Tu <dtu@chromium.org> Date: Mon Aug 27 23:01:17 2018 Revert "Pinpoint - Include doc link in bug comment." This reverts commit 6743e7b08a0d395192bceb46dd4174e8c4c222d9. Reason for revert: CL doesn't work in production, since histogram.py depends on jinja2. Original change's description: > Pinpoint - Include doc link in bug comment. > > Bug: chromium:863861 > Change-Id: I4b40eeabd188acf79305f2acb28a583207cb3d16 > Reviewed-on: https://chromium-review.googlesource.com/1183986 > Commit-Queue: Simon Hatch <simonhatch@chromium.org> > Reviewed-by: David Tu <dtu@chromium.org> TBR=dtu@chromium.org,simonhatch@chromium.org Bug: chromium:863861 Change-Id: I4b76d0ca470a30c8952072a49dda29ff79c2cd78 Reviewed-on: https://chromium-review.googlesource.com/1191983 Commit-Queue: Dave Tu <dtu@chromium.org> Reviewed-by: Dave Tu <dtu@chromium.org> [modify] https://crrev.com/5d0291214a16b137a2c1ba9e67044402ffd71905/dashboard/dashboard/pinpoint/models/job.py [modify] https://crrev.com/5d0291214a16b137a2c1ba9e67044402ffd71905/dashboard/dashboard/pinpoint/models/job_test.py
,
Aug 28
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/22476f41a289c3ef20f8113b5e7e7c96f5796188 commit 22476f41a289c3ef20f8113b5e7e7c96f5796188 Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Tue Aug 28 04:56:49 2018 Roll src/third_party/catapult d0d714d0f2fc..5d0291214a16 (1 commits) https://chromium.googlesource.com/catapult.git/+log/d0d714d0f2fc..5d0291214a16 git log d0d714d0f2fc..5d0291214a16 --date=short --no-merges --format='%ad %ae %s' 2018-08-27 dtu@chromium.org Revert "Pinpoint - Include doc link in bug comment." Created with: gclient setdep -r src/third_party/catapult@5d0291214a16 The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG= chromium:863861 TBR=sullivan@chromium.org Change-Id: I15c970c923616ebb07b0cb230d9b2334c1303184 Reviewed-on: https://chromium-review.googlesource.com/1192251 Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#586583} [modify] https://crrev.com/22476f41a289c3ef20f8113b5e7e7c96f5796188/DEPS
,
Aug 28
,
Aug 29
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/50485c5d9fd22a386c0632c6b6731ae0b2b080f2 commit 50485c5d9fd22a386c0632c6b6731ae0b2b080f2 Author: Simon <simonhatch@chromium.org> Date: Wed Aug 29 18:53:13 2018 Reland "Pinpoint - Include doc link in bug comment." This is a reland of 6743e7b08a0d395192bceb46dd4174e8c4c222d9 Original change's description: > Pinpoint - Include doc link in bug comment. > > Bug: chromium:863861 > Change-Id: I4b40eeabd188acf79305f2acb28a583207cb3d16 > Reviewed-on: https://chromium-review.googlesource.com/1183986 > Commit-Queue: Simon Hatch <simonhatch@chromium.org> > Reviewed-by: David Tu <dtu@chromium.org> Bug: chromium:863861 Change-Id: I2864caad8199cb515d47995bed3d511647cc58ad Reviewed-on: https://chromium-review.googlesource.com/1194909 Commit-Queue: Simon Hatch <simonhatch@chromium.org> Reviewed-by: Simon Hatch <simonhatch@chromium.org> Reviewed-by: Dave Tu <dtu@chromium.org> [modify] https://crrev.com/50485c5d9fd22a386c0632c6b6731ae0b2b080f2/dashboard/dashboard/pinpoint/models/job.py [modify] https://crrev.com/50485c5d9fd22a386c0632c6b6731ae0b2b080f2/dashboard/dashboard/pinpoint/models/job_test.py
,
Aug 29
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/3bdd397e4385c543178d8e7b4abd0eb7bb03b987 commit 3bdd397e4385c543178d8e7b4abd0eb7bb03b987 Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Wed Aug 29 22:05:32 2018 Roll src/third_party/catapult 7dfd811ec106..8972cd776e13 (6 commits) https://chromium.googlesource.com/catapult.git/+log/7dfd811ec106..8972cd776e13 git log 7dfd811ec106..8972cd776e13 --date=short --no-merges --format='%ad %ae %s' 2018-08-29 dtu@chromium.org [pinpoint] Add temporary mapping from Buildbot to LUCI builders. 2018-08-29 simonhatch@chromium.org Pinpoint - Escape names when creating test path 2018-08-29 simonhatch@chromium.org Reland "Pinpoint - Include doc link in bug comment." 2018-08-29 pasko@chromium.org dashboard: replace assert_called_once() 2018-08-29 zmo@chromium.org Remove node_runner from telemetry isolate 2018-08-29 dtu@chromium.org [pinpoint] d3-based heatmap and histogram. Created with: gclient setdep -r src/third_party/catapult@8972cd776e13 The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG=chromium:790075, chromium:863861 ,chromium:878777, chromium:872521 TBR=sullivan@chromium.org Change-Id: I17642df1db4953b723db72b7b52caaf3a97d8538 Reviewed-on: https://chromium-review.googlesource.com/1195862 Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#587303} [modify] https://crrev.com/3bdd397e4385c543178d8e7b4abd0eb7bb03b987/DEPS
,
Oct 4
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by nednguyen@chromium.org
, Jul 16