New issue
Advanced search Search tips

Issue 863861 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Oct 4
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocked on:
issue 878397

Blocking:
issue 859524



Sign in to add a comment

Make sure benchmarks documentation are surfaced on the perf regression/pinpoint reports

Project Member Reported by nednguyen@chromium.org, Jul 16

Issue description

Now some of the benchmarks are properly documented (see https://docs.google.com/spreadsheets/d/1xaAo0_SU3iDfGdqDJZX_jRV0QtkufwHUKH3kQKF3YQs/edit#gid=0), we should make sure that perf dashboard & pinpoint surface them in the regression report.
 
Blocking: 859524
Where do we want to surface these, just in the bug comment or also on the Pinpoint page?
I think we can start with the bug comment first. 
Cc: agrieve@chromium.org
+agrieve

Note that We'd probably want to put these in comment #1 when the bug is filed, so that they work for binary size regressions which generally are not bisected.

It would be fine to be redundant here and also put a comment in the completed bisect like "this regression was detected in the system_health.memory_mobile benchmark. For details on debugging this benchmark, see <link>"
Project Member

Comment 5 by bugdroid1@chromium.org, Jul 17

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/592cb67990bc59f6bcb7d48a8ef23326afbd7168

commit 592cb67990bc59f6bcb7d48a8ef23326afbd7168
Author: Simon <simonhatch@chromium.org>
Date: Tue Jul 17 19:22:39 2018

Dashboard - Whitelist documentation urls in /add_histograms

Will need to follow up with actually using them, but for now at least
save them.

Bug:  chromium:863861 
Change-Id: I8a0305f2e5b9805af0cf71e01ceade4e40226003
Reviewed-on: https://chromium-review.googlesource.com/1140778
Reviewed-by: Ethan Kuefner <eakuefner@chromium.org>
Commit-Queue: Simon Hatch <simonhatch@chromium.org>

[modify] https://crrev.com/592cb67990bc59f6bcb7d48a8ef23326afbd7168/dashboard/dashboard/add_histograms.py

Project Member

Comment 6 by bugdroid1@chromium.org, Jul 18

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ee1d7d6a10afc8d71253cb849ed508f706cf92a9

commit ee1d7d6a10afc8d71253cb849ed508f706cf92a9
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Wed Jul 18 07:21:37 2018

Roll src/third_party/catapult f5342c4cf3d3..e8dc05ccba6b (9 commits)

https://chromium.googlesource.com/catapult.git/+log/f5342c4cf3d3..e8dc05ccba6b


git log f5342c4cf3d3..e8dc05ccba6b --date=short --no-merges --format='%ad %ae %s'
2018-07-18 nednguyen@google.com Revert "Handle stale directories when pushing files"
2018-07-18 nednguyen@google.com Implement screenshot capturing at system level on Linux
2018-07-17 dtu@chromium.org [pinpoint] Case-insensitive compare for Webview.
2018-07-17 simonhatch@chromium.org Dashboard - Whitelist documentation urls in /add_histograms
2018-07-17 cphlipot0@gmail.com [Systrace] Raise error when specified device is not found
2018-07-17 nednguyen@google.com Add test that verifies the way Telemetry generate skipped entries for filtered stories when benchmark is interrupted
2018-07-17 xiaochengh@chromium.org Disable WebPageReplayGoServerTest.testSmokeStartingWebPageReplayGoServer on Windows
2018-07-16 dtu@chromium.org [pinpoint] Update list of bots available in Pinpoint.
2018-07-16 bsheedy@chromium.org Handle stale directories when pushing files


Created with:
  gclient setdep -r src/third_party/catapult@e8dc05ccba6b

The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:862845 , chromium:862996 , chromium:863861 , chromium:860091 ,chromium:864294, chromium:863995 , chromium:862845 
TBR=sullivan@chromium.org

Change-Id: I5cab2a67db2dfee8c71f9ed7e445b82a9df41aec
Reviewed-on: https://chromium-review.googlesource.com/1141465
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#575965}
[modify] https://crrev.com/ee1d7d6a10afc8d71253cb849ed508f706cf92a9/DEPS

Status: Started (was: Untriaged)
Project Member

Comment 8 by bugdroid1@chromium.org, Aug 22

Project Member

Comment 9 by bugdroid1@chromium.org, Aug 23

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/d95dde6e066fe1b0ef88d3e0e5ccb5cae48b05a9

commit d95dde6e066fe1b0ef88d3e0e5ccb5cae48b05a9
Author: Simon <simonhatch@chromium.org>
Date: Thu Aug 23 00:56:31 2018

Dashboard - List out docs for alerts.

This prints a doc link for each suite in the list of alerts when a bug
is filed.

Will append a footer to the existing comment like:

system_health.memory_mobile - Documentation links:
  http://system_health_docs

loading.benchmark - Documentation links:
  http://foo/loading

etc


Additionally, have another cl up for Pinpoint to add a similar comment
when it finishes a job, to include the specific docs for the suite it
bisected.

Bug:  chromium:863861 
Change-Id: I7d5b1e20515323e21a40521832e4a6e9533dc480
Reviewed-on: https://chromium-review.googlesource.com/1185206
Commit-Queue: Simon Hatch <simonhatch@chromium.org>
Reviewed-by: Annie Sullivan <sullivan@chromium.org>

[modify] https://crrev.com/d95dde6e066fe1b0ef88d3e0e5ccb5cae48b05a9/dashboard/dashboard/file_bug_test.py
[modify] https://crrev.com/d95dde6e066fe1b0ef88d3e0e5ccb5cae48b05a9/dashboard/dashboard/file_bug.py

Project Member

Comment 10 by bugdroid1@chromium.org, Aug 23

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c35cb7b095f3b20318cf38aa5ff0f035dd6f488e

commit c35cb7b095f3b20318cf38aa5ff0f035dd6f488e
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Thu Aug 23 03:20:32 2018

Roll src/third_party/catapult c67a80df2679..d95dde6e066f (5 commits)

https://chromium.googlesource.com/catapult.git/+log/c67a80df2679..d95dde6e066f


git log c67a80df2679..d95dde6e066f --date=short --no-merges --format='%ad %ae %s'
2018-08-23 simonhatch@chromium.org Dashboard - List out docs for alerts.
2018-08-22 benjhayden@chromium.org Remove SparseDiagnostic.GetMostRecentValuesByNames.
2018-08-22 simonhatch@chromium.org Pinpoint - Include doc link in bug comment.
2018-08-22 benjhayden@chromium.org Set default_expiration: 20h in v2spa.yaml.
2018-08-22 benjhayden@chromium.org Allow passing custom version to deploy_v2spa.


Created with:
  gclient setdep -r src/third_party/catapult@d95dde6e066f

The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:863861 , chromium:876082 , chromium:863861 
TBR=sullivan@chromium.org

Change-Id: Ib326be155ae9fb40b064de9512abbef02e4a23a7
Reviewed-on: https://chromium-review.googlesource.com/1185691
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#585384}
[modify] https://crrev.com/c35cb7b095f3b20318cf38aa5ff0f035dd6f488e/DEPS

Project Member

Comment 11 by bugdroid1@chromium.org, Aug 27

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/5d0291214a16b137a2c1ba9e67044402ffd71905

commit 5d0291214a16b137a2c1ba9e67044402ffd71905
Author: Dave Tu <dtu@chromium.org>
Date: Mon Aug 27 23:01:17 2018

Revert "Pinpoint - Include doc link in bug comment."

This reverts commit 6743e7b08a0d395192bceb46dd4174e8c4c222d9.

Reason for revert: CL doesn't work in production, since histogram.py depends on jinja2.

Original change's description:
> Pinpoint - Include doc link in bug comment.
>
> Bug:  chromium:863861 
> Change-Id: I4b40eeabd188acf79305f2acb28a583207cb3d16
> Reviewed-on: https://chromium-review.googlesource.com/1183986
> Commit-Queue: Simon Hatch <simonhatch@chromium.org>
> Reviewed-by: David Tu <dtu@chromium.org>

TBR=dtu@chromium.org,simonhatch@chromium.org

Bug:  chromium:863861 
Change-Id: I4b76d0ca470a30c8952072a49dda29ff79c2cd78
Reviewed-on: https://chromium-review.googlesource.com/1191983
Commit-Queue: Dave Tu <dtu@chromium.org>
Reviewed-by: Dave Tu <dtu@chromium.org>

[modify] https://crrev.com/5d0291214a16b137a2c1ba9e67044402ffd71905/dashboard/dashboard/pinpoint/models/job.py
[modify] https://crrev.com/5d0291214a16b137a2c1ba9e67044402ffd71905/dashboard/dashboard/pinpoint/models/job_test.py

Project Member

Comment 12 by bugdroid1@chromium.org, Aug 28

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/22476f41a289c3ef20f8113b5e7e7c96f5796188

commit 22476f41a289c3ef20f8113b5e7e7c96f5796188
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Tue Aug 28 04:56:49 2018

Roll src/third_party/catapult d0d714d0f2fc..5d0291214a16 (1 commits)

https://chromium.googlesource.com/catapult.git/+log/d0d714d0f2fc..5d0291214a16


git log d0d714d0f2fc..5d0291214a16 --date=short --no-merges --format='%ad %ae %s'
2018-08-27 dtu@chromium.org Revert "Pinpoint - Include doc link in bug comment."


Created with:
  gclient setdep -r src/third_party/catapult@5d0291214a16

The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:863861 
TBR=sullivan@chromium.org

Change-Id: I15c970c923616ebb07b0cb230d9b2334c1303184
Reviewed-on: https://chromium-review.googlesource.com/1192251
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#586583}
[modify] https://crrev.com/22476f41a289c3ef20f8113b5e7e7c96f5796188/DEPS

Blockedon: 878397
Project Member

Comment 14 by bugdroid1@chromium.org, Aug 29

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/50485c5d9fd22a386c0632c6b6731ae0b2b080f2

commit 50485c5d9fd22a386c0632c6b6731ae0b2b080f2
Author: Simon <simonhatch@chromium.org>
Date: Wed Aug 29 18:53:13 2018

Reland "Pinpoint - Include doc link in bug comment."

This is a reland of 6743e7b08a0d395192bceb46dd4174e8c4c222d9

Original change's description:
> Pinpoint - Include doc link in bug comment.
> 
> Bug:  chromium:863861 
> Change-Id: I4b40eeabd188acf79305f2acb28a583207cb3d16
> Reviewed-on: https://chromium-review.googlesource.com/1183986
> Commit-Queue: Simon Hatch <simonhatch@chromium.org>
> Reviewed-by: David Tu <dtu@chromium.org>

Bug:  chromium:863861 
Change-Id: I2864caad8199cb515d47995bed3d511647cc58ad
Reviewed-on: https://chromium-review.googlesource.com/1194909
Commit-Queue: Simon Hatch <simonhatch@chromium.org>
Reviewed-by: Simon Hatch <simonhatch@chromium.org>
Reviewed-by: Dave Tu <dtu@chromium.org>

[modify] https://crrev.com/50485c5d9fd22a386c0632c6b6731ae0b2b080f2/dashboard/dashboard/pinpoint/models/job.py
[modify] https://crrev.com/50485c5d9fd22a386c0632c6b6731ae0b2b080f2/dashboard/dashboard/pinpoint/models/job_test.py

Project Member

Comment 15 by bugdroid1@chromium.org, Aug 29

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3bdd397e4385c543178d8e7b4abd0eb7bb03b987

commit 3bdd397e4385c543178d8e7b4abd0eb7bb03b987
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Wed Aug 29 22:05:32 2018

Roll src/third_party/catapult 7dfd811ec106..8972cd776e13 (6 commits)

https://chromium.googlesource.com/catapult.git/+log/7dfd811ec106..8972cd776e13


git log 7dfd811ec106..8972cd776e13 --date=short --no-merges --format='%ad %ae %s'
2018-08-29 dtu@chromium.org [pinpoint] Add temporary mapping from Buildbot to LUCI builders.
2018-08-29 simonhatch@chromium.org Pinpoint - Escape names when creating test path
2018-08-29 simonhatch@chromium.org Reland "Pinpoint - Include doc link in bug comment."
2018-08-29 pasko@chromium.org dashboard: replace assert_called_once()
2018-08-29 zmo@chromium.org Remove node_runner from telemetry isolate
2018-08-29 dtu@chromium.org [pinpoint] d3-based heatmap and histogram.


Created with:
  gclient setdep -r src/third_party/catapult@8972cd776e13

The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG=chromium:790075, chromium:863861 ,chromium:878777, chromium:872521 
TBR=sullivan@chromium.org

Change-Id: I17642df1db4953b723db72b7b52caaf3a97d8538
Reviewed-on: https://chromium-review.googlesource.com/1195862
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#587303}
[modify] https://crrev.com/3bdd397e4385c543178d8e7b4abd0eb7bb03b987/DEPS

Status: Fixed (was: Started)

Sign in to add a comment