New issue
Advanced search Search tips

Issue 709620 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Mar 2018
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Feature
Proj-XR

Blocked on:
issue 710534
issue 710624


Show other hotlists

Hotlists containing this issue:
VR-Automated-Tests


Sign in to add a comment

Run KLM VR e2e tests on main waterfall

Project Member Reported by bsheedy@chromium.org, Apr 7 2017

Issue description

The swarmed KLM end-to-end tests have been looking pretty stable https://build.chromium.org/p/chromium.fyi/builders/Android%20VR%20Tests/builds/6474 (ignore the failing "chrome_public_test_vr_apk" tests, as those are the local tests run on N that won't moved yet). So, it should be fine to get them running on the main waterfall where any changes that break them will actually get noticed/reverted quickly.
 
Blockedon: 710534
Blockedon: 710624
Wasn't automatically posted here for whatever reason, but the e2e tests have been added to the main waterfall in https://codereview.chromium.org/2810813002/ on:
- Lollipop Phone Tester (L)
- Marshmallow 64 bit Tester (M)
- Android N5X Swarm Builder (M)

As a result of the way the CQ is set up, they've also been added to the CQ bots:
- android_n5x_swarming_rel
- android_n5x_swarming_dbg

I'll look into getting K coverage in the near future, although the only bot I've seen that's obviously running tests on K only does so on tablets.
Update on this: Infra recently doubled the number of K devices in the swarming pool, although they're still split between the old configuration (4-7 devices per bot) and the new configuration (single device swarming). Once they're all configured the same (bpastene@ estimates a couple weeks), we can go ahead and run the tests on K on the CQ and close this bug.
Project Member

Comment 5 by bugdroid1@chromium.org, Mar 5 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/4ec75bbd6c042730b5c7a36e75ba94e6745ca613

commit 4ec75bbd6c042730b5c7a36e75ba94e6745ca613
Author: bsheedy <bsheedy@chromium.org>
Date: Mon Mar 05 18:11:47 2018

Make KitKat Phone Tester (rel) download VR APKs

Ensures that the bot has the APKs necessary for running VR
instrumentation tests. This is in preparation for adding the tests to
KitKat Phone Tester (rel), and by extension, the CQ.

Bug:  709620 
Change-Id: Idf58a2f157c712a78f1b6e8991545ab000abe141
Reviewed-on: https://chromium-review.googlesource.com/947576
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Benjamin Pastene <bpastene@chromium.org>

[modify] https://crrev.com/4ec75bbd6c042730b5c7a36e75ba94e6745ca613/scripts/slave/recipes/chromium.expected/dynamic_swarmed_gn_instrumentation_test.json
[modify] https://crrev.com/4ec75bbd6c042730b5c7a36e75ba94e6745ca613/scripts/slave/recipes/chromium.expected/dynamic_instrumentation_test.json
[modify] https://crrev.com/4ec75bbd6c042730b5c7a36e75ba94e6745ca613/scripts/slave/recipe_modules/chromium_tests/chromium_android.py
[modify] https://crrev.com/4ec75bbd6c042730b5c7a36e75ba94e6745ca613/scripts/slave/recipes/chromium.expected/dynamic_instrumentation_test_custom_name.json
[modify] https://crrev.com/4ec75bbd6c042730b5c7a36e75ba94e6745ca613/scripts/slave/recipes/chromium.expected/dynamic_instrumentation_nodefault_build.json
[modify] https://crrev.com/4ec75bbd6c042730b5c7a36e75ba94e6745ca613/scripts/slave/recipes/chromium.expected/dynamic_instrumentation_nodefault_test.json
[modify] https://crrev.com/4ec75bbd6c042730b5c7a36e75ba94e6745ca613/scripts/slave/recipes/chromium.expected/dynamic_junit_test.json

Project Member

Comment 6 by bugdroid1@chromium.org, Mar 5 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/553a1d5bd0ec8c474303fc92e93622e5db0a66fe

commit 553a1d5bd0ec8c474303fc92e93622e5db0a66fe
Author: bsheedy <bsheedy@chromium.org>
Date: Mon Mar 05 20:27:21 2018

Add VR tests to CQ on K

Enables chrome_public_test_vr_apk on linux_android_rel_ng. There is
enough capacity for these tests, and there have been enough VR issues
getting past the CQ recently due to reproducing on K/L but not M to
warrant running the tests on K.

Bug:  709620 
Change-Id: I0c5048031c520b2d47b475ee2765d002da0be6c0
Reviewed-on: https://chromium-review.googlesource.com/949444
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Commit-Position: refs/heads/master@{#540915}
[modify] https://crrev.com/553a1d5bd0ec8c474303fc92e93622e5db0a66fe/testing/buildbot/chromium.android.json
[modify] https://crrev.com/553a1d5bd0ec8c474303fc92e93622e5db0a66fe/testing/buildbot/test_suite_exceptions.pyl

Project Member

Comment 7 by bugdroid1@chromium.org, Mar 7 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fd38337b6ed749f4958a85493c7c85aef066a0d8

commit fd38337b6ed749f4958a85493c7c85aef066a0d8
Author: bsheedy <bsheedy@chromium.org>
Date: Wed Mar 07 20:33:19 2018

Remove experiment percentage for VR tests on K

Removes the experiment percentage from the VR tests on the K CQ bot.
This would make failures blocking, except there was a typo so failures
were already blocking.

Bug:  709620 
Change-Id: I8d7727cd92d8e400540f1f9937cf60832ac8e536
Reviewed-on: https://chromium-review.googlesource.com/952334
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#541559}
[modify] https://crrev.com/fd38337b6ed749f4958a85493c7c85aef066a0d8/testing/buildbot/chromium.android.json
[modify] https://crrev.com/fd38337b6ed749f4958a85493c7c85aef066a0d8/testing/buildbot/test_suite_exceptions.pyl

Status: Fixed (was: Started)
Labels: Test-Complete
Components: Blink>WebXR

Sign in to add a comment