Enable telemetry_unittest on android-marshmallow-arm64-rel |
||||
Issue descriptionWe should enable telemetry_unittest on android-marshmallow-arm64-rel. For historical context, on most of the popular platforms, we run two suites: telemetry_unittest & telemetry_perf_unittests. Somehow telemetry_unittest is not enabled on android-marshmallow-arm64-rel but telemetry_perf_unittests is. This bug is becoming urgent since Simon Hatch is working on migrating Catapult CQ to LUCI, and that effort is being blocked by telemetry_unittest not properly sharded on Catapult LUCI android builder. Since we plan to migrate Telemetry back to chromium/src (issue 886939), we agreed that it's easier if we just remove telemetry_unittests on Android in Catapult CQ for now. However, before doing that, we need to ensure telemetry_unittests is at least run on one Android platform on Chromium CQ.
,
Sep 20
This bot has become capacity-constrained. I don't really expect this to be possible.
,
Sep 20
Thanks for the replies, John. Can we use android-kitkat-arm-rel instead?
,
Sep 25
Ping John again
,
Sep 25
Can you clarify how this blocks migrating the catapult CQ to LUCI?
,
Sep 25
... and also roughly how long telemetry_unittests take on Android these days.
,
Sep 25
#5: John: that's because LUCI use swarming bot, which only allows 1 builder mapping to 1 device. If we just convert Android catapult CQ to LUCI, we ends up running telemetry_unittests only on 1 shard, making the test timed out after 15 minutes (Simon can fill in more details here). Simon@ also can tell how long does it take to run telemetry_unittest on 1 phone.
,
Sep 25
The chromium/src CQ would be the same configuration?
,
Sep 25
chromium/src CQ support swarming, whereas catapult doesn't use swarming yet. Test recipe run on the same place as test :-/
,
Sep 25
Was that not going to be part of migrating the catapult CQ to LUCI?
,
Sep 25
Also, I presume that telemetry_unittests can support sharding?
,
Sep 25
#11: yes, it can. #10: I think Simon is not planning to do sharding for catapult CQ
,
Sep 25
re: #c7 I'm not sure since the luci runs time out after an hour, if I had to estimate, probably about an hour for just the telemetry unit tests (6 devices, 10 mins on main catapult cq, 6 x 10 = 60 mins).
,
Sep 25
android-kitkat-arm-rel should be possible, but this is a nontrivial addition. I'd want better timing info (via a low % experiment on the chromium CQ) before moving forward.
,
Oct 30
nednguyen or jbudorick: any update here? Can one of you own this? Forwarding ping from efoo on 825432 :-)
,
Oct 30
I'm still not convinced of the premise, that this somehow needs to block migrating the catapult CQ to LUCI. Perhaps it needs to block merging telemetry into chromium, but that's a different matter...
,
Nov 7
Catapult Android Tryserver is consistently failing on luci: https://ci.chromium.org/p/catapult/builders/luci.catapult.try/Catapult%20Android%20Tryserver Should the failing tests be disabled so that 825432 can continue?
,
Nov 14
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/fd4a2c4d912c789c3431b9062a896dbd8f69cef7 commit fd4a2c4d912c789c3431b9062a896dbd8f69cef7 Author: benshayden <benjhayden@chromium.org> Date: Wed Nov 14 19:36:29 2018 Disable telemetry tests that fail on Luci Android. This is blocking migrating the android trybot to luci. Bug: chromium:886940 Change-Id: If26ce909effc98b4bbc0ba56c7c553d9207dcbde Reviewed-on: https://chromium-review.googlesource.com/c/1335891 Commit-Queue: Ben Hayden <benjhayden@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> [modify] https://crrev.com/fd4a2c4d912c789c3431b9062a896dbd8f69cef7/telemetry/telemetry/internal/backends/chrome/android_browser_finder_unittest.py [modify] https://crrev.com/fd4a2c4d912c789c3431b9062a896dbd8f69cef7/telemetry/telemetry/internal/backends/chrome/desktop_browser_finder_unittest.py [modify] https://crrev.com/fd4a2c4d912c789c3431b9062a896dbd8f69cef7/telemetry/telemetry/internal/browser/browser_unittest.py
,
Nov 14
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/0e55d4452afdab2949faf5141aac8224626bcefc commit 0e55d4452afdab2949faf5141aac8224626bcefc Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Date: Wed Nov 14 22:03:43 2018 Roll src/third_party/catapult 35d4c6c88f92..4925b069e1cf (6 commits) https://chromium.googlesource.com/catapult.git/+log/35d4c6c88f92..4925b069e1cf git log 35d4c6c88f92..4925b069e1cf --date=short --no-merges --format='%ad %ae %s' 2018-11-14 benjhayden@chromium.org Add polymer-svg-template to third_party for v2spa 2018-11-14 benjhayden@chromium.org Temporarily permit alerts to have lists of ownership components. 2018-11-14 benjhayden@chromium.org Add triage-existing for v2spa 2018-11-14 benjhayden@chromium.org Disable telemetry tests that fail on Luci Android. 2018-11-14 benjhayden@chromium.org Add timeseries-descriptor for v2spa 2018-11-14 benjhayden@chromium.org Fix timestamp x-axis in V1 charts. Created with: gclient setdep -r src/third_party/catapult@4925b069e1cf The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG=chromium:902796,chromium:886940 TBR=sullivan@chromium.org Change-Id: I8a8237f8cdfe354329d1e4e99cb5894f17300e2f Reviewed-on: https://chromium-review.googlesource.com/c/1336215 Reviewed-by: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#608138} [modify] https://crrev.com/0e55d4452afdab2949faf5141aac8224626bcefc/DEPS
,
Jan 16
(6 days ago)
,
Jan 16
(6 days ago)
|
||||
►
Sign in to add a comment |
||||
Comment 1 by simonhatch@chromium.org
, Sep 19