Reduce telemetry based tests isolate size |
|||
Issue descriptionRight now in tools/perf/chrome_telemetry_build/BUILD.gn, the target "telemetry_chrome_test_without_chrome" includes the entire third_party/catapult. This zipped to a 200M file on Linux. Similar on other platforms. On Windows, extracting such data took quite a while. Looking into the catapult folder, many data are not required to run telemetry tests. By some rough experiments, we can reduce it on Linux to around 62M and the tests run fine. This will make wrapping a test on one machine and running it in another much easier. In Chromium's testing, we separate builders from testers and use cloud storage to store isolates. So this reduction could also benefit that quite a bit.
,
Aug 9
CL uploaded for review: https://chromium-review.googlesource.com/c/chromium/src/+/1166261
,
Aug 11
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/1715e3f972fe31826c3aaa4eaef20d0aeb7794c6 commit 1715e3f972fe31826c3aaa4eaef20d0aeb7794c6 Author: Zhenyao Mo <zmo@chromium.org> Date: Sat Aug 11 00:19:47 2018 Reduce telemetry based isolate size Excluding some large size data in third_party/catapult/ that are not needed in running the telemetry tests. R=kbr@chromium.org,nednguyen@chromium.org Change-Id: I10a1968b5a9b02415f5d7f66b33a247d64791578 Bug: 872521 Reviewed-on: https://chromium-review.googlesource.com/1166261 Reviewed-by: Ned Nguyen <nednguyen@google.com> Reviewed-by: Kenneth Russell <kbr@chromium.org> Commit-Queue: Zhenyao Mo <zmo@chromium.org> Cr-Commit-Position: refs/heads/master@{#582397} [modify] https://crrev.com/1715e3f972fe31826c3aaa4eaef20d0aeb7794c6/chrome/test/BUILD.gn [modify] https://crrev.com/1715e3f972fe31826c3aaa4eaef20d0aeb7794c6/tools/perf/chrome_telemetry_build/BUILD.gn
,
Aug 11
Now this CL landed, Per discussion with kbr, a more maintainable method to to do blacklist instead of whitelist. i.e., define a list of folders/files we want to exclude from the scripts target, and write a python script to scan all catapult data and auto-generate a BUILD.gn file on catapult side. Also, in order to make sure every time some folders/files are added/removed from catapult and we need to re-generate the BUILD.gn, we add a PRESUBMIT.py which will generate the BUILD.gn on the spot and check if it's the same as the current version on the workspace. If not, fail and warn the author.
,
Aug 24
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/72712b0d60789853266a87daa8f4ed0f1e7c4c63 commit 72712b0d60789853266a87daa8f4ed0f1e7c4c63 Author: Zhenyao Mo <zmo@chromium.org> Date: Fri Aug 24 17:52:50 2018 Add a script to auto generate telemetry_chrome_test_support build target So we can list a few folders/files we want to exclude because: 1) they are not needed to run telemetry chrome test 2) their size are huge Note we haven't hooked this up with PRESUBMIT yet. Wait until this lands and chromium side switches to this new target and everything runs fine. Bug: chromium:872521 Change-Id: Ic6b623ac64759a3e1088b66c84de1862a503716d R=kbr@chromium.org,nednguyen@chromium.org Reviewed-on: https://chromium-review.googlesource.com/1186132 Reviewed-by: Ned Nguyen <nednguyen@google.com> Reviewed-by: Kenneth Russell <kbr@chromium.org> Commit-Queue: Zhenyao Mo <zmo@chromium.org> [add] https://crrev.com/72712b0d60789853266a87daa8f4ed0f1e7c4c63/BUILD.gn [add] https://crrev.com/72712b0d60789853266a87daa8f4ed0f1e7c4c63/generate_telemetry_build.py
,
Aug 24
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/bd4bed873886cf2edc11c015d7447f448dd0fd4d commit bd4bed873886cf2edc11c015d7447f448dd0fd4d Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Fri Aug 24 22:19:43 2018 Roll src/third_party/catapult 1e44d066259f..72712b0d6078 (6 commits) https://chromium.googlesource.com/catapult.git/+log/1e44d066259f..72712b0d6078 git log 1e44d066259f..72712b0d6078 --date=short --no-merges --format='%ad %ae %s' 2018-08-24 zmo@chromium.org Add a script to auto generate telemetry_chrome_test_support build target 2018-08-24 benjhayden@chromium.org Remove bot_whitelist. 2018-08-24 simonhatch@chromium.org Pinpoint - Fix flaky migrate tests 2018-08-24 anthonyalridge@google.com Use real metric data for calculating significant changes. 2018-08-24 anthonyalridge@google.com Introduce display region for statistical test results. 2018-08-24 pasko@chromium.org perf_control.py: Remove TODO for a fixed bug Created with: gclient setdep -r src/third_party/catapult@72712b0d6078 The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG= chromium:872521 , chromium:868512 , chromium:877074 ,chromium:866423,chromium:866423,chromium:397118 TBR=sullivan@chromium.org Change-Id: I2a744233b5777b9b640ea4fbaff5c9fba95114e6 Reviewed-on: https://chromium-review.googlesource.com/1188518 Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: Zhenyao Mo <zmo@chromium.org> Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#586021} [modify] https://crrev.com/bd4bed873886cf2edc11c015d7447f448dd0fd4d/DEPS
,
Aug 24
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/bc2c0a9307285fa36e03e7cdb6bf8623390ff855 commit bc2c0a9307285fa36e03e7cdb6bf8623390ff855 Author: Zhenyao Mo <zmo@chromium.org> Date: Fri Aug 24 22:37:05 2018 Exclude a few more folders with large data from telemetry data set Bug: chromium:872521 Change-Id: Ib6247c33887bc8ce349194510e157f525ef164e4 R=nednguyen@chromium.org Reviewed-on: https://chromium-review.googlesource.com/1188386 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Zhenyao Mo <zmo@chromium.org> [modify] https://crrev.com/bc2c0a9307285fa36e03e7cdb6bf8623390ff855/BUILD.gn [modify] https://crrev.com/bc2c0a9307285fa36e03e7cdb6bf8623390ff855/generate_telemetry_build.py
,
Aug 25
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6e14f68e47751dabc49067c4ac4004868fbb3907 commit 6e14f68e47751dabc49067c4ac4004868fbb3907 Author: Zhenyao Mo <zmo@chromium.org> Date: Sat Aug 25 00:07:03 2018 Switch chromium side to use the catapult's telemetry_chrome_test_support remove the manual source list and use the auto-generated one. BUG= 872521 TEST=bots R=nednguyen@chromium.org,kbr@chromium.org,nednguyen@google.com Change-Id: Ic5d00299edc39af23802db71c43ad0716669837d Reviewed-on: https://chromium-review.googlesource.com/1188901 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Zhenyao Mo <zmo@chromium.org> Cr-Commit-Position: refs/heads/master@{#586069} [modify] https://crrev.com/6e14f68e47751dabc49067c4ac4004868fbb3907/chrome/test/BUILD.gn [modify] https://crrev.com/6e14f68e47751dabc49067c4ac4004868fbb3907/tools/perf/chrome_telemetry_build/BUILD.gn
,
Aug 25
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/479925f4f66c902ab2f02a9fc0a04eba09bd13b1 commit 479925f4f66c902ab2f02a9fc0a04eba09bd13b1 Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Sat Aug 25 01:11:14 2018 Roll src/third_party/catapult 72712b0d6078..bc2c0a930728 (1 commits) https://chromium.googlesource.com/catapult.git/+log/72712b0d6078..bc2c0a930728 git log 72712b0d6078..bc2c0a930728 --date=short --no-merges --format='%ad %ae %s' 2018-08-24 zmo@chromium.org Exclude a few more folders with large data from telemetry data set Created with: gclient setdep -r src/third_party/catapult@bc2c0a930728 The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG= chromium:872521 TBR=sullivan@chromium.org Change-Id: I905c391cc4d42a8e3d7334203aa1af701a1bbc66 Reviewed-on: https://chromium-review.googlesource.com/1188931 Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#586091} [modify] https://crrev.com/479925f4f66c902ab2f02a9fc0a04eba09bd13b1/DEPS
,
Aug 27
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/074cebc28321a8278d392e282b16891bf36fc445 commit 074cebc28321a8278d392e282b16891bf36fc445 Author: Zhenyao Mo <zmo@chromium.org> Date: Mon Aug 27 19:11:34 2018 Hook up generate_telemetry_build.py to PRESUBMIT.py Bug: chromium:872521 Change-Id: I5f4952bfca5fca6dd3c93a00845e7c5fe15234be R=kbr@chromium.org,nednguyen@google.com Reviewed-on: https://chromium-review.googlesource.com/1188391 Reviewed-by: Ned Nguyen <nednguyen@google.com> Reviewed-by: Kenneth Russell <kbr@chromium.org> Commit-Queue: Zhenyao Mo <zmo@chromium.org> [modify] https://crrev.com/074cebc28321a8278d392e282b16891bf36fc445/PRESUBMIT.py [modify] https://crrev.com/074cebc28321a8278d392e282b16891bf36fc445/telemetry/BUILD.gn [modify] https://crrev.com/074cebc28321a8278d392e282b16891bf36fc445/generate_telemetry_build.py
,
Aug 27
I am sure we can further exclude more dirs and further reduce the size, but it looks to me the largest unnecessary dirs have already been excluded, so I'll call this fixed.
,
Aug 28
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/5d4e3adf79b5ab9fc063c7e10dd6230bd522ed78 commit 5d4e3adf79b5ab9fc063c7e10dd6230bd522ed78 Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Tue Aug 28 00:09:06 2018 Roll src/third_party/catapult 9b4fbc6e4341..d0d714d0f2fc (4 commits) https://chromium.googlesource.com/catapult.git/+log/9b4fbc6e4341..d0d714d0f2fc git log 9b4fbc6e4341..d0d714d0f2fc --date=short --no-merges --format='%ad %ae %s' 2018-08-27 benjhayden@chromium.org Define CSS vars for V2SPA. 2018-08-27 dtu@chromium.org [pinpoint] Ignore comparison_magnitude == 'None' for patch jobs. 2018-08-27 zmo@chromium.org Hook up generate_telemetry_build.py to PRESUBMIT.py 2018-08-27 benjhayden@chromium.org Add web-component-tester to third_party. Created with: gclient setdep -r src/third_party/catapult@d0d714d0f2fc The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG=chromium:877096, chromium:872521 , chromium:877660 TBR=sullivan@chromium.org Change-Id: Iac0ca8d5f207af61bf173c68d57778ce1c6b91b7 Reviewed-on: https://chromium-review.googlesource.com/1192244 Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#586511} [modify] https://crrev.com/5d4e3adf79b5ab9fc063c7e10dd6230bd522ed78/DEPS
,
Aug 29
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/b18e8eb867ec6b691dd9fe13984316f91dd15230 commit b18e8eb867ec6b691dd9fe13984316f91dd15230 Author: Zhenyao Mo <zmo@chromium.org> Date: Wed Aug 29 02:25:41 2018 Order input file/dir list, not the output lists, in generate_telemetry_build.py Bug: chromium:872521 Change-Id: Ia48475154965aa55cac76ee0a47d677d76873dc4 R=kbr@chromium.org,nednguyen@google.com Reviewed-on: https://chromium-review.googlesource.com/1195004 Reviewed-by: Kenneth Russell <kbr@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Zhenyao Mo <zmo@chromium.org> [modify] https://crrev.com/b18e8eb867ec6b691dd9fe13984316f91dd15230/BUILD.gn [modify] https://crrev.com/b18e8eb867ec6b691dd9fe13984316f91dd15230/generate_telemetry_build.py
,
Aug 29
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/3cee42cd6d8170fa1dae35a2de3f6dcf02c0925e commit 3cee42cd6d8170fa1dae35a2de3f6dcf02c0925e Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Wed Aug 29 03:43:52 2018 Roll src/third_party/catapult 3a64f6fe7147..b18e8eb867ec (1 commits) https://chromium.googlesource.com/catapult.git/+log/3a64f6fe7147..b18e8eb867ec git log 3a64f6fe7147..b18e8eb867ec --date=short --no-merges --format='%ad %ae %s' 2018-08-29 zmo@chromium.org Order input file/dir list, not the output lists, in generate_telemetry_build.py Created with: gclient setdep -r src/third_party/catapult@b18e8eb867ec The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG= chromium:872521 TBR=sullivan@chromium.org Change-Id: I682a58c1171c04e631a94f4fd9a2823a3adfa5a4 Reviewed-on: https://chromium-review.googlesource.com/1195107 Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#587006} [modify] https://crrev.com/3cee42cd6d8170fa1dae35a2de3f6dcf02c0925e/DEPS
,
Aug 29
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/c54a6eb1ade8ad9e769eaa17ce47ef811ab66a19 commit c54a6eb1ade8ad9e769eaa17ce47ef811ab66a19 Author: Zhenyao Mo <zmo@chromium.org> Date: Wed Aug 29 17:45:02 2018 Remove node_runner from telemetry isolate Bug: chromium:872521 Change-Id: I3c3fe1e2eac639c7e5adb9095deb2c9a32af1130 R=nednguyen@google.com Reviewed-on: https://chromium-review.googlesource.com/1195008 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Zhenyao Mo <zmo@chromium.org> [modify] https://crrev.com/c54a6eb1ade8ad9e769eaa17ce47ef811ab66a19/BUILD.gn [modify] https://crrev.com/c54a6eb1ade8ad9e769eaa17ce47ef811ab66a19/generate_telemetry_build.py
,
Aug 29
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/3bdd397e4385c543178d8e7b4abd0eb7bb03b987 commit 3bdd397e4385c543178d8e7b4abd0eb7bb03b987 Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Wed Aug 29 22:05:32 2018 Roll src/third_party/catapult 7dfd811ec106..8972cd776e13 (6 commits) https://chromium.googlesource.com/catapult.git/+log/7dfd811ec106..8972cd776e13 git log 7dfd811ec106..8972cd776e13 --date=short --no-merges --format='%ad %ae %s' 2018-08-29 dtu@chromium.org [pinpoint] Add temporary mapping from Buildbot to LUCI builders. 2018-08-29 simonhatch@chromium.org Pinpoint - Escape names when creating test path 2018-08-29 simonhatch@chromium.org Reland "Pinpoint - Include doc link in bug comment." 2018-08-29 pasko@chromium.org dashboard: replace assert_called_once() 2018-08-29 zmo@chromium.org Remove node_runner from telemetry isolate 2018-08-29 dtu@chromium.org [pinpoint] d3-based heatmap and histogram. Created with: gclient setdep -r src/third_party/catapult@8972cd776e13 The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG=chromium:790075, chromium:863861 ,chromium:878777, chromium:872521 TBR=sullivan@chromium.org Change-Id: I17642df1db4953b723db72b7b52caaf3a97d8538 Reviewed-on: https://chromium-review.googlesource.com/1195862 Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#587303} [modify] https://crrev.com/3bdd397e4385c543178d8e7b4abd0eb7bb03b987/DEPS |
|||
►
Sign in to add a comment |
|||
Comment 1 by zmo@chromium.org
, Aug 9