New issue
Advanced search Search tips

Issue 886940 link

Starred by 1 user

Issue metadata

Status: Untriaged
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocking:
issue 825432



Sign in to add a comment

Enable telemetry_unittest on android-marshmallow-arm64-rel

Project Member Reported by nednguyen@chromium.org, Sep 19

Issue description

We should enable telemetry_unittest on android-marshmallow-arm64-rel. 

For historical context, on most of the popular platforms, we run two suites: telemetry_unittest & telemetry_perf_unittests. Somehow telemetry_unittest is not enabled on android-marshmallow-arm64-rel but telemetry_perf_unittests is. 

This bug is becoming urgent since Simon Hatch is working on migrating Catapult CQ to LUCI, and that effort is being blocked by telemetry_unittest not properly sharded on Catapult LUCI android builder. 

Since we plan to migrate Telemetry back to chromium/src (issue 886939), we agreed that it's easier if we just remove telemetry_unittests on Android in Catapult CQ for now. However, before doing that, we need to ensure telemetry_unittests is at least run on one Android platform on Chromium CQ.
 
Blocking: 825432
Cc: bpastene@chromium.org
This bot has become capacity-constrained. I don't really expect this to be possible.
Thanks for the replies, John. Can we use android-kitkat-arm-rel instead?
Ping John again
Can you clarify how this blocks migrating the catapult CQ to LUCI?
... and also roughly how long telemetry_unittests take on Android these days.
#5: John: that's because LUCI use swarming bot, which only allows 1 builder mapping to 1 device. If we just convert Android catapult CQ to LUCI, we ends up running telemetry_unittests only on 1 shard, making the test timed out after 15 minutes (Simon can fill in more details here).

Simon@ also can tell how long does it take to run telemetry_unittest on 1 phone.
The chromium/src CQ would be the same configuration?
chromium/src CQ support swarming, whereas catapult doesn't use swarming yet. Test recipe run on the same place as test :-/
Was that not going to be part of migrating the catapult CQ to LUCI?
Also, I presume that telemetry_unittests can support sharding?
#11: yes, it can. 

#10: I think Simon is not planning to do sharding for catapult CQ
re: #c7

I'm not sure since the luci runs time out after an hour, if I had to estimate, probably about an hour for just the telemetry unit tests (6 devices, 10 mins on main catapult cq, 6 x 10 = 60 mins).
android-kitkat-arm-rel should be possible, but this is a nontrivial addition. I'd want better timing info (via a low % experiment on the chromium CQ) before moving forward.
nednguyen or jbudorick: any update here? Can one of you own this?

Forwarding ping from efoo on 825432 :-)
I'm still not convinced of the premise, that this somehow needs to block migrating the catapult CQ to LUCI. Perhaps it needs to block merging telemetry into chromium, but that's a different matter...
Catapult Android Tryserver is consistently failing on luci:
https://ci.chromium.org/p/catapult/builders/luci.catapult.try/Catapult%20Android%20Tryserver

Should the failing tests be disabled so that 825432 can continue?
Project Member

Comment 19 by bugdroid1@chromium.org, Nov 14

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0e55d4452afdab2949faf5141aac8224626bcefc

commit 0e55d4452afdab2949faf5141aac8224626bcefc
Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Date: Wed Nov 14 22:03:43 2018

Roll src/third_party/catapult 35d4c6c88f92..4925b069e1cf (6 commits)

https://chromium.googlesource.com/catapult.git/+log/35d4c6c88f92..4925b069e1cf


git log 35d4c6c88f92..4925b069e1cf --date=short --no-merges --format='%ad %ae %s'
2018-11-14 benjhayden@chromium.org Add polymer-svg-template to third_party for v2spa
2018-11-14 benjhayden@chromium.org Temporarily permit alerts to have lists of ownership components.
2018-11-14 benjhayden@chromium.org Add triage-existing for v2spa
2018-11-14 benjhayden@chromium.org Disable telemetry tests that fail on Luci Android.
2018-11-14 benjhayden@chromium.org Add timeseries-descriptor for v2spa
2018-11-14 benjhayden@chromium.org Fix timestamp x-axis in V1 charts.


Created with:
  gclient setdep -r src/third_party/catapult@4925b069e1cf

The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG=chromium:902796,chromium:886940
TBR=sullivan@chromium.org

Change-Id: I8a8237f8cdfe354329d1e4e99cb5894f17300e2f
Reviewed-on: https://chromium-review.googlesource.com/c/1336215
Reviewed-by: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#608138}
[modify] https://crrev.com/0e55d4452afdab2949faf5141aac8224626bcefc/DEPS

Comment 20 by benhenry@google.com, Jan 16 (6 days ago)

Components: Test>Telemetry

Comment 21 by benhenry@google.com, Jan 16 (6 days ago)

Components: -Tests>Telemetry

Sign in to add a comment