New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 869762 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Aug 20
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Only download necessary wprgo files when running telemetry benchmark

Project Member Reported by vovoy@chromium.org, Aug 1

Issue description

When we are running telemetry benchmark, some unnecessary wprgo files are downloaded.

For example, tab_switching.typical_25 only needs system_health_desktop_049.wprgo. But executing "run_benchmark tab_switching.typical_25" would download all wprgo files in page_sets/data/system_health_desktop.json .

$ ./fetch_benchmark_deps.py --output-deps /dev/stdout tab_switching.typical_25
{"tab_switching.typical_25": ["tools/perf/page_sets/data/system_health_desktop_049.wprgo"]}

 
When we are running CrOS telemetry autotest[1], it replies on fetch_benchmark_deps.py to get the wprgo file list to upload to DUT(device under test) so that we don't have to setup gsutil on DUT. If running a telemetry benchmark requires wprgo files not listed by fetch_benchmark_deps.py, the CrOS telemetry autotest would fail.

[1]: https://cs.corp.google.com/chromeos_public/src/third_party/autotest/files/server/site_tests/telemetry_Benchmarks/telemetry_Benchmarks.py
Project Member

Comment 2 by bugdroid1@chromium.org, Aug 3

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/f6027ed1df0b8fb281e5d2bd03846907764bf6cf

commit f6027ed1df0b8fb281e5d2bd03846907764bf6cf
Author: Kuo-Hsin Yang <vovoy@chromium.org>
Date: Fri Aug 03 16:13:09 2018

Only download necessary wprgo files in story_runner

DownloadArchivesIfNeeded() downloads all the wprgo files in the archive,
add an optional parameter stories to download only the wprgo files used
by these stories.

Bug:  chromium:869762 
Change-Id: I78c79c2cf166426be89476ae2c7fa5376de8c1f2
Reviewed-on: https://chromium-review.googlesource.com/1158115
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org>
Commit-Queue: Vovo Yang <vovoy@chromium.org>

[modify] https://crrev.com/f6027ed1df0b8fb281e5d2bd03846907764bf6cf/telemetry/telemetry/internal/story_runner.py
[modify] https://crrev.com/f6027ed1df0b8fb281e5d2bd03846907764bf6cf/telemetry/telemetry/wpr/archive_info_unittest.py
[modify] https://crrev.com/f6027ed1df0b8fb281e5d2bd03846907764bf6cf/telemetry/telemetry/wpr/archive_info.py

Project Member

Comment 3 by bugdroid1@chromium.org, Aug 3

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9aad8e5c135e447e9ddd625a520b0c78fb52538c

commit 9aad8e5c135e447e9ddd625a520b0c78fb52538c
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Fri Aug 03 17:29:55 2018

Roll src/third_party/catapult d38bbdff043d..f6027ed1df0b (1 commits)

https://chromium.googlesource.com/catapult.git/+log/d38bbdff043d..f6027ed1df0b


git log d38bbdff043d..f6027ed1df0b --date=short --no-merges --format='%ad %ae %s'
2018-08-03 vovoy@chromium.org Only download necessary wprgo files in story_runner


Created with:
  gclient setdep -r src/third_party/catapult@f6027ed1df0b

The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:869762 
TBR=sullivan@chromium.org

Change-Id: Ic657f700e58d458dfdf90ae2f8a3ccaeed5f76c6
Reviewed-on: https://chromium-review.googlesource.com/1162113
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#580583}
[modify] https://crrev.com/9aad8e5c135e447e9ddd625a520b0c78fb52538c/DEPS

Project Member

Comment 4 by bugdroid1@chromium.org, Aug 6

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ea304b716669f45525eb1feb6a2d0223ee1c1c37

commit ea304b716669f45525eb1feb6a2d0223ee1c1c37
Author: Kuo-Hsin Yang <vovoy@chromium.org>
Date: Mon Aug 06 17:48:38 2018

Only download necessary wprgo files in perf scripts

Bug:  chromium:869762 
Change-Id: I85a0b5f35465c5e2974248426dfb874e13259c8e
Reviewed-on: https://chromium-review.googlesource.com/1163045
Commit-Queue: Vovo Yang <vovoy@chromium.org>
Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#580913}
[modify] https://crrev.com/ea304b716669f45525eb1feb6a2d0223ee1c1c37/tools/perf/benchmarks/system_health_smoke_test.py
[modify] https://crrev.com/ea304b716669f45525eb1feb6a2d0223ee1c1c37/tools/perf/fetch_benchmark_deps.py

Blockedon: 874386
Status: Fixed (was: Assigned)
Blockedon: -874386

Sign in to add a comment