New issue
Advanced search Search tips

Issue 757762 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Aug 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 2
Type: Bug



Sign in to add a comment

telemetry_unittests and telemetry_gpu_unittests on chromium.linux/Linux Tests is flaky - "Oauth 2.0 User Account" credentials invalid

Project Member Reported by hbos@chromium.org, Aug 22 2017

Issue description

It looks like the actual tests run passed, e.g:

telemetry_unittests telemetry_unittests
Run on OS: 'Ubuntu-14.04'
Total tests: 854
* Passed: 793 (793 expected, 0 unexpected)
* Skipped: 61 (61 expected, 0 unexpected)
* Failed: 0 (0 expected, 0 unexpected)
* Flaky: 0 (0 expected, 0 unexpected)

But it's red and logs shows failure:

CloudStorageError: Your "Oauth 2.0 User Account" credentials are invalid. For more help, see "gsutil help creds", or re-run the gsutil config command (see "gsutil help config").
Failure: Unable to find the server at accounts.google.com.

https://uberchromegw.corp.google.com/i/chromium.linux/builders/Linux%20Tests/builds/61163
https://uberchromegw.corp.google.com/i/chromium.linux/builders/Linux%20Tests/builds/61154

Similar on Linux Tests (dbg)(1)(32)

https://uberchromegw.corp.google.com/i/chromium.linux/builders/Linux%20Tests%20%28dbg%29%281%29%2832%29/builds/43583

 
This is still failing occasionally:
https://build.chromium.org/p/chromium.linux/builders/Linux%20Tests/builds/61176


I presume that these failures are on different bots:
61176: gce-trusty-c677cdc5-us-central1-c-1b04
61163: gce-trusty-c677cdc5-us-central1-c-hlzw
61154: gce-trusty-c677cdc5-us-central1-c-zwb1


Components: Infra>Platform>Swarming
Labels: OS-Linux
Tagging as 'Swarming', since this appears to be a failure in the swarming bot config. Not sure if troopers should own, or if it's a more general infra thing.
Owner: vadimsh@chromium.org
Status: WontFix (was: Untriaged)
Can't reproduce :(

The machines where it happened are gone already, so I can't investigate on them directly. On other machines (which are supposedly identical to the one that failed) everything WAI.

The machines where it failed initially ran telemetry_unittests successfully after that.

Finally, the actual errors message is "Failure: Unable to find the server at accounts.google.com". 

"Your "Oauth 2.0 User Account" credentials are invalid" is a red herring because gsutil code stupidly catches ALL exceptions and logs this message regardless of what exactly happened (https://cs.chromium.org/chromium/src/third_party/catapult/third_party/gsutil/gslib/gcs_json_api.py?type=cs&q=For+more+help,+see+%22gsutil+help+creds%22&sq=package:chromium&l=281).

Thus I believe there were a brief DNS-related issues that caused these failures. These issues are gone now.

Sign in to add a comment