New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 872521 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Aug 27
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug

Blocking:
issue 867155



Sign in to add a comment

Reduce telemetry based tests isolate size

Project Member Reported by zmo@chromium.org, Aug 9

Issue description

Right now in tools/perf/chrome_telemetry_build/BUILD.gn, the target "telemetry_chrome_test_without_chrome" includes the entire third_party/catapult.

This zipped to a 200M file on Linux. Similar on other platforms.

On Windows, extracting such data took quite a while.

Looking into the catapult folder, many data are not required to run telemetry tests. By some rough experiments, we can reduce it on Linux to around 62M and the tests run fine.

This will make wrapping a test on one machine and running it in another much easier. In Chromium's testing, we separate builders from testers and use cloud storage to store isolates. So this reduction could also benefit that quite a bit.
 
Blocking: 867155
Status: Started (was: Assigned)
CL uploaded for review: https://chromium-review.googlesource.com/c/chromium/src/+/1166261
Project Member

Comment 3 by bugdroid1@chromium.org, Aug 11

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1715e3f972fe31826c3aaa4eaef20d0aeb7794c6

commit 1715e3f972fe31826c3aaa4eaef20d0aeb7794c6
Author: Zhenyao Mo <zmo@chromium.org>
Date: Sat Aug 11 00:19:47 2018

Reduce telemetry based isolate size

Excluding some large size data in third_party/catapult/ that are not needed
in running the telemetry tests.

R=kbr@chromium.org,nednguyen@chromium.org

Change-Id: I10a1968b5a9b02415f5d7f66b33a247d64791578
Bug:  872521 
Reviewed-on: https://chromium-review.googlesource.com/1166261
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Zhenyao Mo <zmo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#582397}
[modify] https://crrev.com/1715e3f972fe31826c3aaa4eaef20d0aeb7794c6/chrome/test/BUILD.gn
[modify] https://crrev.com/1715e3f972fe31826c3aaa4eaef20d0aeb7794c6/tools/perf/chrome_telemetry_build/BUILD.gn

Now this CL landed, Per discussion with kbr, a more maintainable method to to do blacklist instead of whitelist. i.e., define a list of folders/files we want to exclude from the scripts target, and write a python script to scan all catapult data and auto-generate a BUILD.gn file on catapult side.

Also, in order to make sure every time some folders/files are added/removed from catapult and we need to re-generate the BUILD.gn, we add a PRESUBMIT.py which will generate the BUILD.gn on the spot and check if it's the same as the current version on the workspace. If not, fail and warn the author.
Project Member

Comment 5 by bugdroid1@chromium.org, Aug 24

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/72712b0d60789853266a87daa8f4ed0f1e7c4c63

commit 72712b0d60789853266a87daa8f4ed0f1e7c4c63
Author: Zhenyao Mo <zmo@chromium.org>
Date: Fri Aug 24 17:52:50 2018

Add a script to auto generate telemetry_chrome_test_support build target

So we can list a few folders/files we want to exclude because:
 1) they are not needed to run telemetry chrome test
 2) their size are huge

Note we haven't hooked this up with PRESUBMIT yet. Wait until this lands and
chromium side switches to this new target and everything runs fine.

Bug:  chromium:872521 
Change-Id: Ic6b623ac64759a3e1088b66c84de1862a503716d
R=kbr@chromium.org,nednguyen@chromium.org
Reviewed-on: https://chromium-review.googlesource.com/1186132
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Zhenyao Mo <zmo@chromium.org>

[add] https://crrev.com/72712b0d60789853266a87daa8f4ed0f1e7c4c63/BUILD.gn
[add] https://crrev.com/72712b0d60789853266a87daa8f4ed0f1e7c4c63/generate_telemetry_build.py

Project Member

Comment 6 by bugdroid1@chromium.org, Aug 24

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/bd4bed873886cf2edc11c015d7447f448dd0fd4d

commit bd4bed873886cf2edc11c015d7447f448dd0fd4d
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Fri Aug 24 22:19:43 2018

Roll src/third_party/catapult 1e44d066259f..72712b0d6078 (6 commits)

https://chromium.googlesource.com/catapult.git/+log/1e44d066259f..72712b0d6078


git log 1e44d066259f..72712b0d6078 --date=short --no-merges --format='%ad %ae %s'
2018-08-24 zmo@chromium.org Add a script to auto generate telemetry_chrome_test_support build target
2018-08-24 benjhayden@chromium.org Remove bot_whitelist.
2018-08-24 simonhatch@chromium.org Pinpoint - Fix flaky migrate tests
2018-08-24 anthonyalridge@google.com Use real metric data for calculating significant changes.
2018-08-24 anthonyalridge@google.com Introduce display region for statistical test results.
2018-08-24 pasko@chromium.org perf_control.py: Remove TODO for a fixed bug


Created with:
  gclient setdep -r src/third_party/catapult@72712b0d6078

The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:872521 , chromium:868512 , chromium:877074 ,chromium:866423,chromium:866423,chromium:397118
TBR=sullivan@chromium.org

Change-Id: I2a744233b5777b9b640ea4fbaff5c9fba95114e6
Reviewed-on: https://chromium-review.googlesource.com/1188518
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: Zhenyao Mo <zmo@chromium.org>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#586021}
[modify] https://crrev.com/bd4bed873886cf2edc11c015d7447f448dd0fd4d/DEPS

Project Member

Comment 7 by bugdroid1@chromium.org, Aug 24

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/bc2c0a9307285fa36e03e7cdb6bf8623390ff855

commit bc2c0a9307285fa36e03e7cdb6bf8623390ff855
Author: Zhenyao Mo <zmo@chromium.org>
Date: Fri Aug 24 22:37:05 2018

Exclude a few more folders with large data from telemetry data set

Bug:  chromium:872521 
Change-Id: Ib6247c33887bc8ce349194510e157f525ef164e4
R=nednguyen@chromium.org
Reviewed-on: https://chromium-review.googlesource.com/1188386
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Zhenyao Mo <zmo@chromium.org>

[modify] https://crrev.com/bc2c0a9307285fa36e03e7cdb6bf8623390ff855/BUILD.gn
[modify] https://crrev.com/bc2c0a9307285fa36e03e7cdb6bf8623390ff855/generate_telemetry_build.py

Project Member

Comment 8 by bugdroid1@chromium.org, Aug 25

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6e14f68e47751dabc49067c4ac4004868fbb3907

commit 6e14f68e47751dabc49067c4ac4004868fbb3907
Author: Zhenyao Mo <zmo@chromium.org>
Date: Sat Aug 25 00:07:03 2018

Switch chromium side to use the catapult's telemetry_chrome_test_support

remove the manual source list and use the auto-generated one.

BUG= 872521 
TEST=bots
R=nednguyen@chromium.org,kbr@chromium.org,nednguyen@google.com

Change-Id: Ic5d00299edc39af23802db71c43ad0716669837d
Reviewed-on: https://chromium-review.googlesource.com/1188901
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Zhenyao Mo <zmo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#586069}
[modify] https://crrev.com/6e14f68e47751dabc49067c4ac4004868fbb3907/chrome/test/BUILD.gn
[modify] https://crrev.com/6e14f68e47751dabc49067c4ac4004868fbb3907/tools/perf/chrome_telemetry_build/BUILD.gn

Project Member

Comment 9 by bugdroid1@chromium.org, Aug 25

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/479925f4f66c902ab2f02a9fc0a04eba09bd13b1

commit 479925f4f66c902ab2f02a9fc0a04eba09bd13b1
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Sat Aug 25 01:11:14 2018

Roll src/third_party/catapult 72712b0d6078..bc2c0a930728 (1 commits)

https://chromium.googlesource.com/catapult.git/+log/72712b0d6078..bc2c0a930728


git log 72712b0d6078..bc2c0a930728 --date=short --no-merges --format='%ad %ae %s'
2018-08-24 zmo@chromium.org Exclude a few more folders with large data from telemetry data set


Created with:
  gclient setdep -r src/third_party/catapult@bc2c0a930728

The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:872521 
TBR=sullivan@chromium.org

Change-Id: I905c391cc4d42a8e3d7334203aa1af701a1bbc66
Reviewed-on: https://chromium-review.googlesource.com/1188931
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#586091}
[modify] https://crrev.com/479925f4f66c902ab2f02a9fc0a04eba09bd13b1/DEPS

Project Member

Comment 10 by bugdroid1@chromium.org, Aug 27

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/074cebc28321a8278d392e282b16891bf36fc445

commit 074cebc28321a8278d392e282b16891bf36fc445
Author: Zhenyao Mo <zmo@chromium.org>
Date: Mon Aug 27 19:11:34 2018

Hook up generate_telemetry_build.py to PRESUBMIT.py

Bug:  chromium:872521 
Change-Id: I5f4952bfca5fca6dd3c93a00845e7c5fe15234be
R=kbr@chromium.org,nednguyen@google.com
Reviewed-on: https://chromium-review.googlesource.com/1188391
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Zhenyao Mo <zmo@chromium.org>

[modify] https://crrev.com/074cebc28321a8278d392e282b16891bf36fc445/PRESUBMIT.py
[modify] https://crrev.com/074cebc28321a8278d392e282b16891bf36fc445/telemetry/BUILD.gn
[modify] https://crrev.com/074cebc28321a8278d392e282b16891bf36fc445/generate_telemetry_build.py

Status: Fixed (was: Started)
I am sure we can further exclude more dirs and further reduce the size, but it looks to me the largest unnecessary dirs have already been excluded, so I'll call this fixed.
Project Member

Comment 12 by bugdroid1@chromium.org, Aug 28

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5d4e3adf79b5ab9fc063c7e10dd6230bd522ed78

commit 5d4e3adf79b5ab9fc063c7e10dd6230bd522ed78
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Tue Aug 28 00:09:06 2018

Roll src/third_party/catapult 9b4fbc6e4341..d0d714d0f2fc (4 commits)

https://chromium.googlesource.com/catapult.git/+log/9b4fbc6e4341..d0d714d0f2fc


git log 9b4fbc6e4341..d0d714d0f2fc --date=short --no-merges --format='%ad %ae %s'
2018-08-27 benjhayden@chromium.org Define CSS vars for V2SPA.
2018-08-27 dtu@chromium.org [pinpoint] Ignore comparison_magnitude == 'None' for patch jobs.
2018-08-27 zmo@chromium.org Hook up generate_telemetry_build.py to PRESUBMIT.py
2018-08-27 benjhayden@chromium.org Add web-component-tester to third_party.


Created with:
  gclient setdep -r src/third_party/catapult@d0d714d0f2fc

The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG=chromium:877096, chromium:872521 , chromium:877660 
TBR=sullivan@chromium.org

Change-Id: Iac0ca8d5f207af61bf173c68d57778ce1c6b91b7
Reviewed-on: https://chromium-review.googlesource.com/1192244
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#586511}
[modify] https://crrev.com/5d4e3adf79b5ab9fc063c7e10dd6230bd522ed78/DEPS

Project Member

Comment 13 by bugdroid1@chromium.org, Aug 29

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/b18e8eb867ec6b691dd9fe13984316f91dd15230

commit b18e8eb867ec6b691dd9fe13984316f91dd15230
Author: Zhenyao Mo <zmo@chromium.org>
Date: Wed Aug 29 02:25:41 2018

Order input file/dir list, not the output lists, in generate_telemetry_build.py

Bug:  chromium:872521 
Change-Id: Ia48475154965aa55cac76ee0a47d677d76873dc4
R=kbr@chromium.org,nednguyen@google.com
Reviewed-on: https://chromium-review.googlesource.com/1195004
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Zhenyao Mo <zmo@chromium.org>

[modify] https://crrev.com/b18e8eb867ec6b691dd9fe13984316f91dd15230/BUILD.gn
[modify] https://crrev.com/b18e8eb867ec6b691dd9fe13984316f91dd15230/generate_telemetry_build.py

Project Member

Comment 14 by bugdroid1@chromium.org, Aug 29

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3cee42cd6d8170fa1dae35a2de3f6dcf02c0925e

commit 3cee42cd6d8170fa1dae35a2de3f6dcf02c0925e
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Wed Aug 29 03:43:52 2018

Roll src/third_party/catapult 3a64f6fe7147..b18e8eb867ec (1 commits)

https://chromium.googlesource.com/catapult.git/+log/3a64f6fe7147..b18e8eb867ec


git log 3a64f6fe7147..b18e8eb867ec --date=short --no-merges --format='%ad %ae %s'
2018-08-29 zmo@chromium.org Order input file/dir list, not the output lists, in generate_telemetry_build.py


Created with:
  gclient setdep -r src/third_party/catapult@b18e8eb867ec

The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:872521 
TBR=sullivan@chromium.org

Change-Id: I682a58c1171c04e631a94f4fd9a2823a3adfa5a4
Reviewed-on: https://chromium-review.googlesource.com/1195107
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#587006}
[modify] https://crrev.com/3cee42cd6d8170fa1dae35a2de3f6dcf02c0925e/DEPS

Project Member

Comment 15 by bugdroid1@chromium.org, Aug 29

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/c54a6eb1ade8ad9e769eaa17ce47ef811ab66a19

commit c54a6eb1ade8ad9e769eaa17ce47ef811ab66a19
Author: Zhenyao Mo <zmo@chromium.org>
Date: Wed Aug 29 17:45:02 2018

Remove node_runner from telemetry isolate

Bug:  chromium:872521 
Change-Id: I3c3fe1e2eac639c7e5adb9095deb2c9a32af1130
R=nednguyen@google.com
Reviewed-on: https://chromium-review.googlesource.com/1195008
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Zhenyao Mo <zmo@chromium.org>

[modify] https://crrev.com/c54a6eb1ade8ad9e769eaa17ce47ef811ab66a19/BUILD.gn
[modify] https://crrev.com/c54a6eb1ade8ad9e769eaa17ce47ef811ab66a19/generate_telemetry_build.py

Project Member

Comment 16 by bugdroid1@chromium.org, Aug 29

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3bdd397e4385c543178d8e7b4abd0eb7bb03b987

commit 3bdd397e4385c543178d8e7b4abd0eb7bb03b987
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Wed Aug 29 22:05:32 2018

Roll src/third_party/catapult 7dfd811ec106..8972cd776e13 (6 commits)

https://chromium.googlesource.com/catapult.git/+log/7dfd811ec106..8972cd776e13


git log 7dfd811ec106..8972cd776e13 --date=short --no-merges --format='%ad %ae %s'
2018-08-29 dtu@chromium.org [pinpoint] Add temporary mapping from Buildbot to LUCI builders.
2018-08-29 simonhatch@chromium.org Pinpoint - Escape names when creating test path
2018-08-29 simonhatch@chromium.org Reland "Pinpoint - Include doc link in bug comment."
2018-08-29 pasko@chromium.org dashboard: replace assert_called_once()
2018-08-29 zmo@chromium.org Remove node_runner from telemetry isolate
2018-08-29 dtu@chromium.org [pinpoint] d3-based heatmap and histogram.


Created with:
  gclient setdep -r src/third_party/catapult@8972cd776e13

The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG=chromium:790075, chromium:863861 ,chromium:878777, chromium:872521 
TBR=sullivan@chromium.org

Change-Id: I17642df1db4953b723db72b7b52caaf3a97d8538
Reviewed-on: https://chromium-review.googlesource.com/1195862
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#587303}
[modify] https://crrev.com/3bdd397e4385c543178d8e7b4abd0eb7bb03b987/DEPS

Sign in to add a comment