New issue
Advanced search Search tips

Issue 903849 link

Starred by 3 users

Issue metadata

Status: Untriaged
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Bug-Regression



Sign in to add a comment

Flaky telemetry_perf_unittests under android-marshmallow-arm64-rel?

Project Member Reported by jdeblasio@chromium.org, Nov 9

Issue description

I'm seeing inconsistent results in https://crrev.com/c/1324394 in telemetry_perf_unittests on android-marshmallow-arm64-rel with intermittent timeouts and unrelated failures. While the change itself IS Android related, it doesn't look to be at all on the critical path for the errors I saw.

In particular, see these runs, all of which are for the same CL (albeit insignificantly different changesets):
 - https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/125674
 - https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/125836
 - https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/126519

I'm uncertain what, if any, of those outputs are actually of value, and perhaps my CL IS related, but I can't figure out how.
 
This suite is causing problems again today. I'm dropping it to experimental.
Project Member

Comment 2 by bugdroid1@chromium.org, Nov 12

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6b6acaf8c24a453511f9a96a45f1546433758352

commit 6b6acaf8c24a453511f9a96a45f1546433758352
Author: John Budorick <jbudorick@chromium.org>
Date: Mon Nov 12 23:56:05 2018

Drop telemetry_perf_unittests to 100% experiment on android-marshmallow-arm64-rel.

No-Try: true
Tbr: nednguyen@google.com,sergeyberezin@chromium.org
Bug: 903849
Change-Id: Ib6ca07f44fcb27a9f2b6cb665fbc856539d91698
Reviewed-on: https://chromium-review.googlesource.com/c/1331875
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Sergey Berezin <sergeyberezin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#607376}
[modify] https://crrev.com/6b6acaf8c24a453511f9a96a45f1546433758352/testing/buildbot/chromium.android.json
[modify] https://crrev.com/6b6acaf8c24a453511f9a96a45f1546433758352/testing/buildbot/test_suite_exceptions.pyl

Cc: nedngu...@google.com crouleau@chromium.org
Cc: perezju@chromium.org
it looks like this may now be limited to two benchmarks:
 - benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_mobile/load:games:spychase
 - benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_mobile/browse:news:cnn:2018

I'm going to disable them and then watch the suite on CI.
Cc: cbruni@chromium.org u...@chromium.org sullivan@chromium.org eyaich@chromium.org
+other folks FYI

Project Member

Comment 6 by bugdroid1@chromium.org, Nov 14

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d09ee4df161a762e3b2b736e38d36821b5739e28

commit d09ee4df161a762e3b2b736e38d36821b5739e28
Author: John Budorick <jbudorick@chromium.org>
Date: Wed Nov 14 03:16:51 2018

Disable two system health smoke tests.

Tbr: crouleau@chromium.org,nednguyen@google.com,perezju@chromium.org
Bug: 903849
Change-Id: I2821fc28f5c63a036d563a6350db4659c4fb5440
Reviewed-on: https://chromium-review.googlesource.com/c/1334831
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
Cr-Commit-Position: refs/heads/master@{#607884}
[modify] https://crrev.com/d09ee4df161a762e3b2b736e38d36821b5739e28/tools/perf/benchmarks/system_health_smoke_test.py

😿 Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/15a7a7f7e40000

All of the runs failed. The most common error (19/20 runs) was:
DevtoolsTargetCrashException: Devtools target crashed
TPU seems healthier this morning. Reenabling the suite on android-marshmallow-arm64-rel
Project Member

Comment 11 by bugdroid1@chromium.org, Nov 14

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/99c1e76e6abddbef91b6f64ea5c6997df227ad6f

commit 99c1e76e6abddbef91b6f64ea5c6997df227ad6f
Author: John Budorick <jbudorick@chromium.org>
Date: Wed Nov 14 16:59:40 2018

Reenable telemetry_perf_unittests on android-marshmallow-arm64-rel.

Tbr: crouleau@chromium.org,nednguyen@google.com,perezju@chromium.org
Bug: 903849
Change-Id: Ib0857c3bbd630ea7e47928dcd43b9f9fcc12597e
Reviewed-on: https://chromium-review.googlesource.com/c/1335935
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: John Budorick <jbudorick@chromium.org>
Cr-Commit-Position: refs/heads/master@{#608015}
[modify] https://crrev.com/99c1e76e6abddbef91b6f64ea5c6997df227ad6f/testing/buildbot/chromium.android.json
[modify] https://crrev.com/99c1e76e6abddbef91b6f64ea5c6997df227ad6f/testing/buildbot/test_suite_exceptions.pyl

😿 Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/1401acade40000

All of the runs failed. The most common error (15/20 runs) was:
DevtoolsTargetCrashException: Devtools target crashed
Labels: Pri-2
Issue has a component, but no priority. Updating to have default priority (Pri-2)

Comment 14 by benhenry@google.com, Jan 16 (6 days ago)

Components: Test>Telemetry

Comment 15 by benhenry@google.com, Jan 16 (6 days ago)

Components: -Tests>Telemetry

Sign in to add a comment