Chrome has as large amount of existing Telemetry performance tests. A lot of these are relevant for WebVR and/or the VR browser, such as scrolling smoothness, so porting them to work with VR makes sense.
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/353eaeaa6dd940ade0dd83fe9a962edbf955cb6a commit 353eaeaa6dd940ade0dd83fe9a962edbf955cb6a Author: bsheedy <bsheedy@chromium.org> Date: Thu Jun 01 17:59:29 2017 Add APK to simulate VR NFC scan Adds a standalone APK that, when run, simulates a Daydream View NFC tag scan. This is so that Telemetry can enter the VR browser without making any modifications to Chrome, e.g. a flag. Bug: 726906 Change-Id: I37a430c98b08dc41363fdc02b3486193b9e26fa6 Reviewed-on: https://chromium-review.googlesource.com/517682 Reviewed-by: David Trainor <dtrainor@chromium.org> Reviewed-by: Michael Thiessen <mthiesse@chromium.org> Commit-Queue: Brian Sheedy <bsheedy@chromium.org> Cr-Commit-Position: refs/heads/master@{#476348} [modify] https://crrev.com/353eaeaa6dd940ade0dd83fe9a962edbf955cb6a/chrome/android/BUILD.gn [add] https://crrev.com/353eaeaa6dd940ade0dd83fe9a962edbf955cb6a/chrome/android/javatests/src/org/chromium/chrome/browser/vr_shell/NfcSimUtils.java [modify] https://crrev.com/353eaeaa6dd940ade0dd83fe9a962edbf955cb6a/chrome/android/javatests/src/org/chromium/chrome/browser/vr_shell/VrUtils.java [add] https://crrev.com/353eaeaa6dd940ade0dd83fe9a962edbf955cb6a/chrome/android/javatests/src/org/chromium/chrome/browser/vr_shell/nfc_apk/AndroidManifest.xml [add] https://crrev.com/353eaeaa6dd940ade0dd83fe9a962edbf955cb6a/chrome/android/javatests/src/org/chromium/chrome/browser/vr_shell/nfc_apk/SimNfcActivity.java
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2755196ca8173187e99f0141ac1de6644a257a08 commit 2755196ca8173187e99f0141ac1de6644a257a08 Author: bsheedy <bsheedy@chromium.org> Date: Mon Jun 05 23:40:34 2017 Remove extra VR test library target Follow-up to https://chromium-review.googlesource.com/c/517682/6. As pointed out by torne@ in https://chromium-review.googlesource.com/c/522082/, the dex errors were caused by the inclusion of JSR305 annotations in the controller test library which conflicted with the annotations in base. For some unknown reason, this conflict doesn't affect the building of chrome_public_test_vr_apk (instrumentation_test_apk), but does affect vr_nfc_simulator_apk (android_apk). This CL adds an updated version of the library with the annotations removed and combines the vr_nfc_simulator_java library target with chrome_test_vr_java. Bug: 726906 Change-Id: Id352a5f636aeed0f9c9fc2d1a228d285ca4634b5 Reviewed-on: https://chromium-review.googlesource.com/523342 Commit-Queue: Brian Sheedy <bsheedy@chromium.org> Reviewed-by: David Trainor <dtrainor@chromium.org> Reviewed-by: Brandon Jones <bajones@chromium.org> Cr-Commit-Position: refs/heads/master@{#477119} [modify] https://crrev.com/2755196ca8173187e99f0141ac1de6644a257a08/chrome/android/BUILD.gn [modify] https://crrev.com/2755196ca8173187e99f0141ac1de6644a257a08/third_party/gvr-android-sdk/test-libraries/controller_test_api.aar.sha1
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/079d646f4f177e4a03c1062c7b251bdb191bc813 commit 079d646f4f177e4a03c1062c7b251bdb191bc813 Author: bsheedy <bsheedy@chromium.org> Date: Sat Jul 01 01:08:32 2017 Add tools/perf/config directory for VR Adds a new directory to tools/perf/config for VR Telemetry tests. As per tools/perf/contrib/README.md, the first CL should only include the OWNERS file. A number of existing metrics will be measured in VR, but since we: 1. Don't want to run these tests on the perf waterfall (at least not yet) 2. Need to perform non-standard setup before running benchmarks (additional APK installation and configuration) We need to create our own benchmarks instead of re-using existing ones. Bug: 726906 Change-Id: I380d9bbcafa2388223edd24a84dc848d229bb353 Reviewed-on: https://chromium-review.googlesource.com/558524 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#483884} [add] https://crrev.com/079d646f4f177e4a03c1062c7b251bdb191bc813/tools/perf/contrib/vr_benchmarks/OWNERS
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/9c9d952b8494f108db1efa4c00ddc81b9f485223 commit 9c9d952b8494f108db1efa4c00ddc81b9f485223 Author: bsheedy <bsheedy@chromium.org> Date: Thu Jul 06 00:42:38 2017 Move shared preference settings file modification code to pylib utils Moves the code for modifying an app's shared preference settings file out of local_device_instrumentation_test and instrumentation_test_instance into a new file in pylib/utils. This is being done because we are in the process of adding Telemetry test support for VR, which will require configuring an APK like VR instrumentation tests currently do. Moving the code into pylib/utils allows us to re-use the existing code in Telemetry. Bug: 726906 Change-Id: I531ca33b69a2f418d7d5d82752577ef9799e0f63 Reviewed-on: https://chromium-review.googlesource.com/558305 Commit-Queue: Brian Sheedy <bsheedy@chromium.org> Reviewed-by: John Budorick <jbudorick@chromium.org> Cr-Commit-Position: refs/heads/master@{#484423} [modify] https://crrev.com/9c9d952b8494f108db1efa4c00ddc81b9f485223/build/android/pylib/instrumentation/instrumentation_test_instance.py [modify] https://crrev.com/9c9d952b8494f108db1efa4c00ddc81b9f485223/build/android/pylib/local/device/local_device_instrumentation_test_run.py [add] https://crrev.com/9c9d952b8494f108db1efa4c00ddc81b9f485223/build/android/pylib/utils/shared_preference_utils.py [modify] https://crrev.com/9c9d952b8494f108db1efa4c00ddc81b9f485223/build/android/test_runner.pydeps
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/5c70e92361076651ca1e2c0f6653b367229d41d8 commit 5c70e92361076651ca1e2c0f6653b367229d41d8 Author: bsheedy <bsheedy@chromium.org> Date: Mon Jul 10 22:22:27 2017 Add VR Telemetry benchmark runner Adds a benchmark runner for VR Telemetry tests that installs and configures the necessary APKs before running the benchmark. VR benchmarks, pages, actions, etc. will be added in follow-up CLs. Bug: 738074 , 726906 Change-Id: I88feccafbee51e83114ca0182e394d25a7c7f936 Reviewed-on: https://chromium-review.googlesource.com/563761 Commit-Queue: Brian Sheedy <bsheedy@chromium.org> Reviewed-by: Andrew Grieve <agrieve@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#485421} [modify] https://crrev.com/5c70e92361076651ca1e2c0f6653b367229d41d8/BUILD.gn [add] https://crrev.com/5c70e92361076651ca1e2c0f6653b367229d41d8/tools/perf/contrib/vr_benchmarks/BUILD.gn [add] https://crrev.com/5c70e92361076651ca1e2c0f6653b367229d41d8/tools/perf/contrib/vr_benchmarks/__init__.py [add] https://crrev.com/5c70e92361076651ca1e2c0f6653b367229d41d8/tools/perf/contrib/vr_benchmarks/run_benchmark [add] https://crrev.com/5c70e92361076651ca1e2c0f6653b367229d41d8/tools/perf/contrib/vr_benchmarks/vr_config.py
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/01d9394784248a17ee7dbe436898ae42a1840095 commit 01d9394784248a17ee7dbe436898ae42a1840095 Author: bsheedy <bsheedy@chromium.org> Date: Thu Jul 27 22:49:38 2017 Add webvr.info repo to DEPS Adds the mirrored repo for webvr.info to the DEPS file. The repo contains a number of WebVR samples that we plan on using for various performance tests. Repo was mirrored in https://bugs.chromium.org/p/chromium/issues/detail?id=748086 Bug: 726906 Change-Id: Icdf4697d2c65bccfd1a0c34004bb27c27b69ef07 Reviewed-on: https://chromium-review.googlesource.com/590587 Reviewed-by: Brandon Jones <bajones@chromium.org> Commit-Queue: Brian Sheedy <bsheedy@chromium.org> Cr-Commit-Position: refs/heads/master@{#490036} [modify] https://crrev.com/01d9394784248a17ee7dbe436898ae42a1840095/.gitignore [modify] https://crrev.com/01d9394784248a17ee7dbe436898ae42a1840095/DEPS
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e556039a98c5665a37be8f1cb965f429518b9456 commit e556039a98c5665a37be8f1cb965f429518b9456 Author: bsheedy <bsheedy@chromium.org> Date: Tue Aug 08 19:07:53 2017 Add initial VR Telemetry memory benchmark Adds a WebVR memory benchmark. Memory usage isn't as important as a number of other metrics (e.g. FPS), but this acts as a proof of concept for running a WebVR benchmark with an existing metric. Bug: 726906 Change-Id: Ib3e553f51407b9d9585f9df6668b33f5e0b2d4c3 Reviewed-on: https://chromium-review.googlesource.com/567027 Reviewed-by: Ned Nguyen <nednguyen@google.com> Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org> Commit-Queue: Brian Sheedy <bsheedy@chromium.org> Cr-Commit-Position: refs/heads/master@{#492718} [modify] https://crrev.com/e556039a98c5665a37be8f1cb965f429518b9456/tools/perf/benchmarks/memory.py [modify] https://crrev.com/e556039a98c5665a37be8f1cb965f429518b9456/tools/perf/contrib/vr_benchmarks/BUILD.gn [modify] https://crrev.com/e556039a98c5665a37be8f1cb965f429518b9456/tools/perf/contrib/vr_benchmarks/run_benchmark [add] https://crrev.com/e556039a98c5665a37be8f1cb965f429518b9456/tools/perf/contrib/vr_benchmarks/vr_benchmarks/__init__.py [add] https://crrev.com/e556039a98c5665a37be8f1cb965f429518b9456/tools/perf/contrib/vr_benchmarks/vr_benchmarks/vr_memory.py [modify] https://crrev.com/e556039a98c5665a37be8f1cb965f429518b9456/tools/perf/contrib/vr_benchmarks/vr_config.py [add] https://crrev.com/e556039a98c5665a37be8f1cb965f429518b9456/tools/perf/contrib/vr_benchmarks/vr_page_sets/__init__.py [add] https://crrev.com/e556039a98c5665a37be8f1cb965f429518b9456/tools/perf/contrib/vr_benchmarks/vr_page_sets/webvr_sample_pages.py
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6d0c0474afaf43044106dbb74eeb53df1b99bab5 commit 6d0c0474afaf43044106dbb74eeb53df1b99bab5 Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org> Date: Thu Aug 10 05:32:41 2017 Roll src/third_party/catapult/ bd05965e4..af523a172 (1 commit) https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/bd05965e4557..af523a1723e5 $ git log bd05965e4..af523a172 --date=short --no-merges --format='%ad %ae %s' 2017-08-09 bsheedy Add GetSharedPrefs to Android platform Created with: roll-dep src/third_party/catapult BUG= 726906 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, see: http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel TBR=sullivan@chromium.org Change-Id: I71cb03fb9a99e673f3a687ed8f64b87f28c629d4 Reviewed-on: https://chromium-review.googlesource.com/609597 Reviewed-by: <catapult-deps-roller@chromium.org> Commit-Queue: <catapult-deps-roller@chromium.org> Cr-Commit-Position: refs/heads/master@{#493278} [modify] https://crrev.com/6d0c0474afaf43044106dbb74eeb53df1b99bab5/DEPS
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/550f26061a53fd69ada8fafc13170dff06e5e544 commit 550f26061a53fd69ada8fafc13170dff06e5e544 Author: bsheedy <bsheedy@chromium.org> Date: Tue Aug 15 23:36:05 2017 Move some VR Telemetry setup into shared state Moves the VrCore installation and setup out of run_benchmark and into the shared state class used by VR benchmarks. Has some ugly temporary hacks to make presubmit happy that will be removed in a follow-up CL that moves the remaining of setup code out of run_benchmark, allowing us to remove the file completely. This CL relies on the following Catapult CL being submitted beforehand: https://codereview.chromium.org/2995713002/ Bug: 726906 Change-Id: Ib5969b1657aa11b9b640ea4f7420ff746991f0c1 Reviewed-on: https://chromium-review.googlesource.com/608984 Commit-Queue: Brian Sheedy <bsheedy@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Reviewed-by: John Budorick <jbudorick@chromium.org> Cr-Commit-Position: refs/heads/master@{#494603} [modify] https://crrev.com/550f26061a53fd69ada8fafc13170dff06e5e544/build/android/pylib/local/device/local_device_instrumentation_test_run.py [modify] https://crrev.com/550f26061a53fd69ada8fafc13170dff06e5e544/build/android/pylib/utils/shared_preference_utils.py [modify] https://crrev.com/550f26061a53fd69ada8fafc13170dff06e5e544/tools/perf/BUILD.gn [modify] https://crrev.com/550f26061a53fd69ada8fafc13170dff06e5e544/tools/perf/PRESUBMIT.py [modify] https://crrev.com/550f26061a53fd69ada8fafc13170dff06e5e544/tools/perf/contrib/vr_benchmarks/run_benchmark [modify] https://crrev.com/550f26061a53fd69ada8fafc13170dff06e5e544/tools/perf/contrib/vr_benchmarks/vr_benchmarks/vr_memory.py [add] https://crrev.com/550f26061a53fd69ada8fafc13170dff06e5e544/tools/perf/contrib/vr_benchmarks/vr_page_sets/shared_android_vr_page_state.py [modify] https://crrev.com/550f26061a53fd69ada8fafc13170dff06e5e544/tools/perf/contrib/vr_benchmarks/vr_page_sets/webvr_sample_pages.py [modify] https://crrev.com/550f26061a53fd69ada8fafc13170dff06e5e544/tools/perf/core/path_util.py [modify] https://crrev.com/550f26061a53fd69ada8fafc13170dff06e5e544/tools/perf/core/story_expectation_validator.py [modify] https://crrev.com/550f26061a53fd69ada8fafc13170dff06e5e544/tools/perf/validate_wpr_archives
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7829a6d05471d9d574e94ee59cb187580fe18919 commit 7829a6d05471d9d574e94ee59cb187580fe18919 Author: bsheedy <bsheedy@chromium.org> Date: Wed Aug 16 16:42:36 2017 Remove VR specific benchmark runner Moves the remainder of setup code out of the VR Telemetry benchmark runner and removes the runner. Also removes all the extra files/code that were necessary to make presubmit happy while using it. Bug: 726906 Change-Id: Ib5095a9354fc7d33660d63a0ffa73de562b688ab Reviewed-on: https://chromium-review.googlesource.com/616320 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Brian Sheedy <bsheedy@chromium.org> Cr-Commit-Position: refs/heads/master@{#494822} [modify] https://crrev.com/7829a6d05471d9d574e94ee59cb187580fe18919/tools/perf/contrib/vr_benchmarks/BUILD.gn [delete] https://crrev.com/7a3d16ca2f9a822f455b2bf03253c064654c290c/tools/perf/contrib/vr_benchmarks/run_benchmark [modify] https://crrev.com/7829a6d05471d9d574e94ee59cb187580fe18919/tools/perf/contrib/vr_benchmarks/vr_benchmarks/__init__.py [modify] https://crrev.com/7829a6d05471d9d574e94ee59cb187580fe18919/tools/perf/contrib/vr_benchmarks/vr_benchmarks/vr_memory.py [delete] https://crrev.com/7a3d16ca2f9a822f455b2bf03253c064654c290c/tools/perf/contrib/vr_benchmarks/vr_config.py [modify] https://crrev.com/7829a6d05471d9d574e94ee59cb187580fe18919/tools/perf/contrib/vr_benchmarks/vr_page_sets/shared_android_vr_page_state.py [modify] https://crrev.com/7829a6d05471d9d574e94ee59cb187580fe18919/tools/perf/contrib/vr_benchmarks/vr_page_sets/webvr_sample_pages.py
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ddfe31eb97e10bcb7235a4e8d60fc2b0af7c32b0 commit ddfe31eb97e10bcb7235a4e8d60fc2b0af7c32b0 Author: Brian Sheedy <bsheedy@chromium.org> Date: Wed Aug 16 19:01:24 2017 Revert "Remove VR specific benchmark runner" This reverts commit 7829a6d05471d9d574e94ee59cb187580fe18919. Reason for revert: Depends on a change that breaks ChromeOS unittests https://bugs.chromium.org/p/chromium/issues/detail?id=756053 Original change's description: > Remove VR specific benchmark runner > > Moves the remainder of setup code out of the VR Telemetry benchmark > runner and removes the runner. Also removes all the extra files/code > that were necessary to make presubmit happy while using it. > > Bug: 726906 > Change-Id: Ib5095a9354fc7d33660d63a0ffa73de562b688ab > Reviewed-on: https://chromium-review.googlesource.com/616320 > Reviewed-by: Ned Nguyen <nednguyen@google.com> > Commit-Queue: Brian Sheedy <bsheedy@chromium.org> > Cr-Commit-Position: refs/heads/master@{#494822} TBR=nednguyen@google.com,bsheedy@chromium.org Change-Id: Ia91be3223bc0a7e715a0e93429105d1d72133e1a No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: 726906 Reviewed-on: https://chromium-review.googlesource.com/617620 Reviewed-by: Michael Thiessen <mthiesse@chromium.org> Commit-Queue: Brian Sheedy <bsheedy@chromium.org> Cr-Commit-Position: refs/heads/master@{#494888} [modify] https://crrev.com/ddfe31eb97e10bcb7235a4e8d60fc2b0af7c32b0/tools/perf/contrib/vr_benchmarks/BUILD.gn [add] https://crrev.com/ddfe31eb97e10bcb7235a4e8d60fc2b0af7c32b0/tools/perf/contrib/vr_benchmarks/run_benchmark [modify] https://crrev.com/ddfe31eb97e10bcb7235a4e8d60fc2b0af7c32b0/tools/perf/contrib/vr_benchmarks/vr_benchmarks/__init__.py [modify] https://crrev.com/ddfe31eb97e10bcb7235a4e8d60fc2b0af7c32b0/tools/perf/contrib/vr_benchmarks/vr_benchmarks/vr_memory.py [add] https://crrev.com/ddfe31eb97e10bcb7235a4e8d60fc2b0af7c32b0/tools/perf/contrib/vr_benchmarks/vr_config.py [modify] https://crrev.com/ddfe31eb97e10bcb7235a4e8d60fc2b0af7c32b0/tools/perf/contrib/vr_benchmarks/vr_page_sets/shared_android_vr_page_state.py [modify] https://crrev.com/ddfe31eb97e10bcb7235a4e8d60fc2b0af7c32b0/tools/perf/contrib/vr_benchmarks/vr_page_sets/webvr_sample_pages.py
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/60c714a56920407ad19926d553a1bc8f98ef4335 commit 60c714a56920407ad19926d553a1bc8f98ef4335 Author: Brian Sheedy <bsheedy@chromium.org> Date: Wed Aug 16 19:04:35 2017 Revert "Move some VR Telemetry setup into shared state" This reverts commit 550f26061a53fd69ada8fafc13170dff06e5e544. Reason for revert: Breaks ChromeOS unittests https://bugs.chromium.org/p/chromium/issues/detail?id=756053 Original change's description: > Move some VR Telemetry setup into shared state > > Moves the VrCore installation and setup out of run_benchmark > and into the shared state class used by VR benchmarks. Has some > ugly temporary hacks to make presubmit happy that will be removed > in a follow-up CL that moves the remaining of setup code out of > run_benchmark, allowing us to remove the file completely. > > This CL relies on the following Catapult CL being submitted > beforehand: https://codereview.chromium.org/2995713002/ > > Bug: 726906 > Change-Id: Ib5969b1657aa11b9b640ea4f7420ff746991f0c1 > Reviewed-on: https://chromium-review.googlesource.com/608984 > Commit-Queue: Brian Sheedy <bsheedy@chromium.org> > Reviewed-by: Ned Nguyen <nednguyen@google.com> > Reviewed-by: John Budorick <jbudorick@chromium.org> > Cr-Commit-Position: refs/heads/master@{#494603} TBR=nednguyen@google.com,bsheedy@chromium.org,jbudorick@chromium.org Change-Id: I0279bfa3fe6cf67e5c6fb0456509742054090e0c No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: 726906 Reviewed-on: https://chromium-review.googlesource.com/617660 Reviewed-by: Michael Thiessen <mthiesse@chromium.org> Commit-Queue: Brian Sheedy <bsheedy@chromium.org> Cr-Commit-Position: refs/heads/master@{#494889} [modify] https://crrev.com/60c714a56920407ad19926d553a1bc8f98ef4335/build/android/pylib/local/device/local_device_instrumentation_test_run.py [modify] https://crrev.com/60c714a56920407ad19926d553a1bc8f98ef4335/build/android/pylib/utils/shared_preference_utils.py [modify] https://crrev.com/60c714a56920407ad19926d553a1bc8f98ef4335/tools/perf/BUILD.gn [modify] https://crrev.com/60c714a56920407ad19926d553a1bc8f98ef4335/tools/perf/PRESUBMIT.py [modify] https://crrev.com/60c714a56920407ad19926d553a1bc8f98ef4335/tools/perf/contrib/vr_benchmarks/run_benchmark [modify] https://crrev.com/60c714a56920407ad19926d553a1bc8f98ef4335/tools/perf/contrib/vr_benchmarks/vr_benchmarks/vr_memory.py [delete] https://crrev.com/ddfe31eb97e10bcb7235a4e8d60fc2b0af7c32b0/tools/perf/contrib/vr_benchmarks/vr_page_sets/shared_android_vr_page_state.py [modify] https://crrev.com/60c714a56920407ad19926d553a1bc8f98ef4335/tools/perf/contrib/vr_benchmarks/vr_page_sets/webvr_sample_pages.py [modify] https://crrev.com/60c714a56920407ad19926d553a1bc8f98ef4335/tools/perf/core/path_util.py [modify] https://crrev.com/60c714a56920407ad19926d553a1bc8f98ef4335/tools/perf/core/story_expectation_validator.py [modify] https://crrev.com/60c714a56920407ad19926d553a1bc8f98ef4335/tools/perf/validate_wpr_archives
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e95bb966ef5e097fab7a2897e474908ad25712e7 commit e95bb966ef5e097fab7a2897e474908ad25712e7 Author: bsheedy <bsheedy@chromium.org> Date: Thu Aug 17 16:48:54 2017 Reland "Move some VR Telemetry setup into shared state" This is a reland of 550f26061a53fd69ada8fafc13170dff06e5e544 Original change's description: > Move some VR Telemetry setup into shared state > > Moves the VrCore installation and setup out of run_benchmark > and into the shared state class used by VR benchmarks. Has some > ugly temporary hacks to make presubmit happy that will be removed > in a follow-up CL that moves the remaining of setup code out of > run_benchmark, allowing us to remove the file completely. > > This CL relies on the following Catapult CL being submitted > beforehand: https://codereview.chromium.org/2995713002/ > > Bug: 726906 > Change-Id: Ib5969b1657aa11b9b640ea4f7420ff746991f0c1 > Reviewed-on: https://chromium-review.googlesource.com/608984 > Commit-Queue: Brian Sheedy <bsheedy@chromium.org> > Reviewed-by: Ned Nguyen <nednguyen@google.com> > Reviewed-by: John Budorick <jbudorick@chromium.org> > Cr-Commit-Position: refs/heads/master@{#494603} Bug: 726906 Change-Id: I2ec5475b0e04023fb50ff8cc3d17bd77a8a07f37 Reviewed-on: https://chromium-review.googlesource.com/617820 Reviewed-by: Ned Nguyen <nednguyen@google.com> Reviewed-by: Achuith Bhandarkar <achuith@chromium.org> Reviewed-by: John Budorick <jbudorick@chromium.org> Commit-Queue: Brian Sheedy <bsheedy@chromium.org> Cr-Commit-Position: refs/heads/master@{#495196} [modify] https://crrev.com/e95bb966ef5e097fab7a2897e474908ad25712e7/build/android/pylib/local/device/local_device_instrumentation_test_run.py [modify] https://crrev.com/e95bb966ef5e097fab7a2897e474908ad25712e7/build/android/pylib/utils/shared_preference_utils.py [modify] https://crrev.com/e95bb966ef5e097fab7a2897e474908ad25712e7/tools/perf/BUILD.gn [modify] https://crrev.com/e95bb966ef5e097fab7a2897e474908ad25712e7/tools/perf/PRESUBMIT.py [modify] https://crrev.com/e95bb966ef5e097fab7a2897e474908ad25712e7/tools/perf/bootstrap_deps [modify] https://crrev.com/e95bb966ef5e097fab7a2897e474908ad25712e7/tools/perf/contrib/vr_benchmarks/run_benchmark [modify] https://crrev.com/e95bb966ef5e097fab7a2897e474908ad25712e7/tools/perf/contrib/vr_benchmarks/vr_benchmarks/vr_memory.py [add] https://crrev.com/e95bb966ef5e097fab7a2897e474908ad25712e7/tools/perf/contrib/vr_benchmarks/vr_page_sets/shared_android_vr_page_state.py [modify] https://crrev.com/e95bb966ef5e097fab7a2897e474908ad25712e7/tools/perf/contrib/vr_benchmarks/vr_page_sets/webvr_sample_pages.py [modify] https://crrev.com/e95bb966ef5e097fab7a2897e474908ad25712e7/tools/perf/core/path_util.py [modify] https://crrev.com/e95bb966ef5e097fab7a2897e474908ad25712e7/tools/perf/core/story_expectation_validator.py [modify] https://crrev.com/e95bb966ef5e097fab7a2897e474908ad25712e7/tools/perf/validate_wpr_archives
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/fb8cd09dfdaf9c033ebec00a3d54c27f16b31909 commit fb8cd09dfdaf9c033ebec00a3d54c27f16b31909 Author: bsheedy <bsheedy@chromium.org> Date: Thu Aug 17 18:45:29 2017 Reland "Remove VR specific benchmark runner" This is a reland of 7829a6d05471d9d574e94ee59cb187580fe18919 Original change's description: > Remove VR specific benchmark runner > > Moves the remainder of setup code out of the VR Telemetry benchmark > runner and removes the runner. Also removes all the extra files/code > that were necessary to make presubmit happy while using it. > > Bug: 726906 > Change-Id: Ib5095a9354fc7d33660d63a0ffa73de562b688ab > Reviewed-on: https://chromium-review.googlesource.com/616320 > Reviewed-by: Ned Nguyen <nednguyen@google.com> > Commit-Queue: Brian Sheedy <bsheedy@chromium.org> > Cr-Commit-Position: refs/heads/master@{#494822} Bug: 726906 Change-Id: Ia92715b34111a4436fd3a8352cf75e82e685912a Reviewed-on: https://chromium-review.googlesource.com/618947 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Brian Sheedy <bsheedy@chromium.org> Cr-Commit-Position: refs/heads/master@{#495250} [modify] https://crrev.com/fb8cd09dfdaf9c033ebec00a3d54c27f16b31909/tools/perf/contrib/vr_benchmarks/BUILD.gn [delete] https://crrev.com/f23737c7751ceb3aacf8bee825f987357325c920/tools/perf/contrib/vr_benchmarks/run_benchmark [modify] https://crrev.com/fb8cd09dfdaf9c033ebec00a3d54c27f16b31909/tools/perf/contrib/vr_benchmarks/vr_benchmarks/__init__.py [modify] https://crrev.com/fb8cd09dfdaf9c033ebec00a3d54c27f16b31909/tools/perf/contrib/vr_benchmarks/vr_benchmarks/vr_memory.py [delete] https://crrev.com/f23737c7751ceb3aacf8bee825f987357325c920/tools/perf/contrib/vr_benchmarks/vr_config.py [modify] https://crrev.com/fb8cd09dfdaf9c033ebec00a3d54c27f16b31909/tools/perf/contrib/vr_benchmarks/vr_page_sets/shared_android_vr_page_state.py [modify] https://crrev.com/fb8cd09dfdaf9c033ebec00a3d54c27f16b31909/tools/perf/contrib/vr_benchmarks/vr_page_sets/webvr_sample_pages.py
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/fa2a2f26ea8ad5ede0d9b721531a8203d28eac24 commit fa2a2f26ea8ad5ede0d9b721531a8203d28eac24 Author: bsheedy <bsheedy@chromium.org> Date: Thu Aug 24 23:51:56 2017 Make VR telemetry work on swarming Makes the changes necessary to allow the VR Telemetry tests to run on swarming: 1. Include the files for run_telemetry_benchmark_as_googletest.py 2. Add a target to gn_isolate_map.pyl 3. Make the benchmarks turn on the device screen Bug: 726906 Change-Id: Iee2089db6064a780f4bd4a212812e8ddbf0b62d4 Reviewed-on: https://chromium-review.googlesource.com/627340 Reviewed-by: Charlie Andrews <charliea@chromium.org> Reviewed-by: Tim 'mithro' Ansell <tansell@chromium.org> Commit-Queue: Brian Sheedy <bsheedy@chromium.org> Cr-Commit-Position: refs/heads/master@{#497251} [modify] https://crrev.com/fa2a2f26ea8ad5ede0d9b721531a8203d28eac24/testing/buildbot/gn_isolate_map.pyl [modify] https://crrev.com/fa2a2f26ea8ad5ede0d9b721531a8203d28eac24/testing/buildbot/manage.py [modify] https://crrev.com/fa2a2f26ea8ad5ede0d9b721531a8203d28eac24/tools/perf/contrib/vr_benchmarks/BUILD.gn
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/faa11d3830485f9679bc94119a08955b3ca08115 commit faa11d3830485f9679bc94119a08955b3ca08115 Author: bsheedy <bsheedy@chromium.org> Date: Fri Aug 25 19:17:01 2017 Make VR Telemetry tests turn on screen Makes SharedAndroidVrPageState inherit from AndroidScreenRestorationSharedState, which ensures that the device's screen is on, but is otherwise identical to SharedPageState. Bug: 726906 Change-Id: Ib25b3247e5af2a6ad30adf68099b6237bca302c6 Reviewed-on: https://chromium-review.googlesource.com/635187 Reviewed-by: Charlie Andrews <charliea@chromium.org> Commit-Queue: Brian Sheedy <bsheedy@chromium.org> Cr-Commit-Position: refs/heads/master@{#497484} [modify] https://crrev.com/faa11d3830485f9679bc94119a08955b3ca08115/tools/perf/contrib/vr_benchmarks/vr_page_sets/shared_android_vr_page_state.py
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e79a07b0208e361b25b1140813c42c866a8ee59b commit e79a07b0208e361b25b1140813c42c866a8ee59b Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org> Date: Tue Aug 29 20:55:46 2017 Roll src/third_party/catapult/ eebaedf9b..1bd3fc1ae (3 commits) https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/eebaedf9bcac..1bd3fc1aefb8 $ git log eebaedf9b..1bd3fc1ae --date=short --no-merges --format='%ad %ae %s' 2017-08-29 dtu [pinpoint] Add culprit info. 2017-08-29 leilei Add TBMv2 metrics for WebVR. 2017-08-29 hjd tracing: Don't crash on unknown process name Created with: roll-dep src/third_party/catapult BUG= 726906 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, see: http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel TBR=sullivan@chromium.org Change-Id: Idf76ccc612dd5f00233f6e0755a9dcc1f1a3f0d8 Reviewed-on: https://chromium-review.googlesource.com/641929 Reviewed-by: <catapult-deps-roller@chromium.org> Commit-Queue: <catapult-deps-roller@chromium.org> Cr-Commit-Position: refs/heads/master@{#498222} [modify] https://crrev.com/e79a07b0208e361b25b1140813c42c866a8ee59b/DEPS
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d33a9557ccd125e36a6aa3b269a6f93424256eb0 commit d33a9557ccd125e36a6aa3b269a6f93424256eb0 Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org> Date: Sat Sep 02 02:20:04 2017 Roll src/third_party/catapult/ 017fd5cf4..e29053eb7 (11 commits) https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/017fd5cf4ccb..e29053eb7dfc $ git log 017fd5cf4..e29053eb7 --date=short --no-merges --format='%ad %ae %s' 2017-09-01 dtu [pinpoint] Handle missing chartjson files. 2017-09-01 simonhatch Pinpoint - Update job dialog with latest names + tir_label. 2017-09-01 tiborg [vr] Add metric to measure frame cycle times 2017-09-01 charliea Move the benchmark health report's deps into a requirements.txt file 2017-09-01 kristipark [Telemetry] Removed legacy blob storage metrics 2017-09-01 kraynov Android systrace: Plot used memory from /proc/meminfo. 2017-09-01 benjhayden Clean up histogram-set-json-format.md 2017-09-01 benjhayden Serialize traces when --output-format=html. 2017-09-01 charliea Upload a new version of the BattOr firmware 2017-09-01 charliea Make BattOrWrapper StopTracing check for 'Done.' 2017-09-01 djordje.golubovic.imgtec Update the update_v8 script to support MIPS Created with: roll-dep src/third_party/catapult BUG= 726906 , 717515 , 757508 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, see: http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel TBR=sullivan@chromium.org Change-Id: I22caf15b3a034b5b0779e40dd967f468e401c892 Reviewed-on: https://chromium-review.googlesource.com/647655 Reviewed-by: <catapult-deps-roller@chromium.org> Commit-Queue: <catapult-deps-roller@chromium.org> Cr-Commit-Position: refs/heads/master@{#499376} [modify] https://crrev.com/d33a9557ccd125e36a6aa3b269a6f93424256eb0/DEPS
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4d6ae07f4c3343cef070dbdec7f9c35771099a6c commit 4d6ae07f4c3343cef070dbdec7f9c35771099a6c Author: Lei Lei <leilei@chromium.org> Date: Tue Sep 05 23:46:37 2017 Update vr_memory benchmark to make it as a generic benchmark for webvr. + Rename it to webvr.sample_pages + Add webvr fps and frame time metrics. Bug: 726906 Change-Id: I1ea7acc1410fc22fc06eae553382873a41148512 Reviewed-on: https://chromium-review.googlesource.com/642072 Commit-Queue: Lei Lei <leilei@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#499802} [delete] https://crrev.com/4debc9c11ba69a927af7d9438e7b5b141f3ea6a5/tools/perf/contrib/vr_benchmarks/vr_benchmarks/vr_memory.py [add] https://crrev.com/4d6ae07f4c3343cef070dbdec7f9c35771099a6c/tools/perf/contrib/vr_benchmarks/vr_benchmarks/webvr.py
The following revision refers to this bug: https://chrome-internal.googlesource.com/chrome/tools/build_limited/scripts/slave/+/6b9e1610a420bf07a47479956a0ef0892589a7ad commit 6b9e1610a420bf07a47479956a0ef0892589a7ad Author: Lei Lei <leilei@google.com> Date: Wed Sep 06 00:23:08 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7a69043896b53a5f45b4143f9f5a91ba43f8382b commit 7a69043896b53a5f45b4143f9f5a91ba43f8382b Author: Tibor Goldschwendt <tiborg@chromium.org> Date: Thu Sep 14 18:45:57 2017 [vr] Add frame cycle times telemetry test The benchmark added here uses the metric from: https://codereview.chromium.org/3010693002/ This CL is based on https://chromium-review.googlesource.com/c/chromium/src/+/567604 and was started with leilei@. Credit is due. Bug: 726906 Change-Id: I0336f060e3b82efca4dded41929c18bbf1bf8504 Reviewed-on: https://chromium-review.googlesource.com/604871 Commit-Queue: Tibor Goldschwendt <tiborg@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#502003} [modify] https://crrev.com/7a69043896b53a5f45b4143f9f5a91ba43f8382b/tools/perf/contrib/vr_benchmarks/BUILD.gn [rename] https://crrev.com/7a69043896b53a5f45b4143f9f5a91ba43f8382b/tools/perf/contrib/vr_benchmarks/shared_android_vr_page_state.py [add] https://crrev.com/7a69043896b53a5f45b4143f9f5a91ba43f8382b/tools/perf/contrib/vr_benchmarks/vr_benchmarks.py [delete] https://crrev.com/81e9248898bb4a9acd25f2a8ca5e3590ad5e4ebc/tools/perf/contrib/vr_benchmarks/vr_benchmarks/__init__.py [delete] https://crrev.com/81e9248898bb4a9acd25f2a8ca5e3590ad5e4ebc/tools/perf/contrib/vr_benchmarks/vr_benchmarks/webvr.py [add] https://crrev.com/7a69043896b53a5f45b4143f9f5a91ba43f8382b/tools/perf/contrib/vr_benchmarks/vr_browsing_mode_pages.py [delete] https://crrev.com/81e9248898bb4a9acd25f2a8ca5e3590ad5e4ebc/tools/perf/contrib/vr_benchmarks/vr_page_sets/__init__.py [delete] https://crrev.com/81e9248898bb4a9acd25f2a8ca5e3590ad5e4ebc/tools/perf/contrib/vr_benchmarks/vr_page_sets/webvr_sample_pages.py [add] https://crrev.com/7a69043896b53a5f45b4143f9f5a91ba43f8382b/tools/perf/contrib/vr_benchmarks/vr_sample_page.py [add] https://crrev.com/7a69043896b53a5f45b4143f9f5a91ba43f8382b/tools/perf/contrib/vr_benchmarks/webvr_sample_pages.py
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build/+/60637e807c93b6c36def59e355e37d91227a2e05 commit 60637e807c93b6c36def59e355e37d91227a2e05 Author: bsheedy <bsheedy@chromium.org> Date: Tue Sep 19 00:22:51 2017 Add support for per-test perf-id Adds support for isolated script tests to override the perf-id arg that is passed to the test in cases where the same test is run multiple times from the same bot on different devices, as is the case for VR. Still defaults to the perf-id value specified in the bot config if not specified in the test spec, so should not have any impact on any tests that do not specifically use this functionality. Bug: 726906 Change-Id: I023f7450cf3693bbf0b162841a1f9e5d6bf13de0 Reviewed-on: https://chromium-review.googlesource.com/671477 Commit-Queue: Brian Sheedy <bsheedy@chromium.org> Reviewed-by: John Budorick <jbudorick@chromium.org> [modify] https://crrev.com/60637e807c93b6c36def59e355e37d91227a2e05/scripts/slave/recipe_modules/chromium_tests/steps.py
The following revision refers to this bug: https://chrome-internal.googlesource.com/chrome/tools/build_limited/scripts/slave/+/6c2639d1b919799e5b7760a209ac1ea96948f44d commit 6c2639d1b919799e5b7760a209ac1ea96948f44d Author: bsheedy <bsheedy@google.com> Date: Thu Oct 19 20:18:55 2017
Closing this out since we have a number of Telemetry tests, and this bug is pretty vague. We can open new bugs for any specific new tests we want.
Comment 1 by leilei@chromium.org
, May 30 2017