New issue
Advanced search Search tips

Issue 733749 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jun 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

src.git PRESUBMIT gsutil auth woes

Project Member Reported by benhenry@chromium.org, Jun 15 2017

Issue description

I'm slow

> gclient sync
...
make change
...
> git commit
> git cl upload
...
** Presubmit ERRORS **
Validating story expectation data failed.

***************

Traceback (most recent call last):
  <module> at /usr/local/google/home/benhenry/checkouts/chromium/src/tools/perf/validate_story_expectation_data:12
    sys.exit(story_expectation_validator.main(sys.argv[1:]))
  main at /usr/local/google/home/benhenry/checkouts/chromium/src/tools/perf/core/story_expectation_validator.py:42
    validate_story_names(benchmarks)
  validate_story_names at /usr/local/google/home/benhenry/checkouts/chromium/src/tools/perf/core/story_expectation_validator.py:34
    story_set = b.CreateStorySet(options)
  CreateStorySet at /usr/local/google/home/benhenry/checkouts/chromium/src/tools/perf/benchmarks/system_health.py:116
    take_memory_measurement=True)
  __init__ at /usr/local/google/home/benhenry/checkouts/chromium/src/tools/perf/page_sets/system_health/system_health_stories.py:32
    self.AddStory(story_class(self, take_memory_measurement))
  __init__ at /usr/local/google/home/benhenry/checkouts/chromium/src/tools/perf/page_sets/system_health/system_health_story.py:90
    platform_specific=self.PLATFORM_SPECIFIC)
  __init__ at /usr/local/google/home/benhenry/checkouts/chromium/src/third_party/catapult/telemetry/telemetry/page/__init__.py:46
    cloud_storage.GetIfChanged(credentials_path, credentials_bucket)
  GetIfChanged at /usr/local/google/home/benhenry/checkouts/chromium/src/third_party/catapult/common/py_utils/py_utils/cloud_storage.py:422
    _GetLocked(bucket, expected_hash, file_path)
  _GetLocked at /usr/local/google/home/benhenry/checkouts/chromium/src/third_party/catapult/common/py_utils/py_utils/cloud_storage.py:348
    _RunCommand(['cp', url, partial_download_path.name])
  _RunCommand at /usr/local/google/home/benhenry/checkouts/chromium/src/third_party/catapult/common/py_utils/py_utils/cloud_storage.py:153
    raise GetErrorObjectForCloudStorageStderr(stderr)
CloudStorageError: INFO 0615 11:33:41.783521 oauth2_client.py] 
Attempted to retrieve an access token from an invalid refresh token. Two common
cases in which you will see this error are:
1. Your refresh token was revoked.
2. Your refresh token was typed incorrectly.

Your "Oauth 2.0 User Account" credentials are invalid. For more help, see "gsutil help creds", or re-run the gsutil config command (see "gsutil help config").
Failure: invalid_grant.

Locals:
  args       : ['/usr/local/google/home/benhenry/checkouts/chromium/src/third_party/catapult/third_party/gsutil/gsutil', 'cp', 'gs://chromium-telemetry/195ad9c57ad9e558ffa8c7d94319ecb3c60327a3', '/usr/local/google/home/benhenry/checkouts/chromium/src/tools/perf/page_sets/data/tmp24M4q5']
  gsutil     : <subprocess.Popen object at 0x7f317f0c5a50>
  gsutil_env : None
  stderr     : 'INFO 0615 11:33:41.783521 oauth2_client.py] \nAttempted to retrieve an access token from an invalid refresh token. Two common\ncases in which you will see this error are:\n1. Your refresh token was revoked.\n2. Your refresh token was typed incorrectly.\n\nYour "Oauth 2.0 User Account" credentials are invalid. For more help, see "gsutil help creds", or re-run the gsutil config command (see "gsutil help config").\nFailure: invalid_grant.\n'
  stdout     : ''
***************


So I ran:
> gsutil config

and it had me:
> gcloud components update

which had me remove old instances of google_appengine and gsutil that I had locally.

> gstuil config

made me:
> gcloud auth login

Which I did, and now I get the output seen above. Do I need to store creds somewhere? I've run both prodaccess and glogin.

I was hoping for something quick instead of having to read the gsutil help creds page.
 
This might be of interest.

$ gcloud auth login
Your browser has been opened to visit:

    https://accounts.google.com/o/oauth2/auth?redirect_uri=http%3A%2F%2Flocalhost%3A8085%2F&prompt=select_account&response_type=code&client_id=32555940559.apps.googleusercontent.com&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fappengine.admin+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcompute+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Faccounts.reauth&access_type=offline


[13001:13035:0615/114508.569003:ERROR:browser_gpu_channel_host_factory.cc(103)] Failed to launch GPU process.
Created new window in existing browser session.
WARNING: `gcloud auth login` no longer writes application default credentials.
If you need to use ADC, see:
  gcloud auth application-default --help

You are now logged in as [benhenry@google.com].
Your current project is [None].  You can change this setting by running:
  $ gcloud config set project PROJECT_ID


I'm now assuming the creds haven't been written locally. How can I tell?

Comment 2 by aga...@chromium.org, Jun 15 2017

Cc: d...@chromium.org aga...@chromium.org hinoka@chromium.org
Owner: ----
Status: Available (was: Assigned)
Summary: src.git PRESUBMIT gsutil auth woes (was: git cl upload woes)
Oh this is craziness with PRESUBMIT and gsutil. I have no idea what's going on here, but it's "just" a PRESUBMIT failure, not an actual git-cl-upload bug.

CCing some people who know stuff about things.

Comment 3 by hinoka@chromium.org, Jun 15 2017

gsutil didn't gain the ability to read gcloud tokens until maybe more recently.  It could be possible that catapult is using an older version of gsutil.  If they are, then "gsutil config" must be used to drop a boto instead of a token.
catapult? I'm using a chromium checkout. Also, how do I drop a boto instead of a token?

Comment 5 by hinoka@chromium.org, Jun 15 2017

Catapult is a DEPS of src, the error you linked failed on a perf related step that calls into catapult.

To get a boto, run "gsutil.py config" (Or "download_from_google_storage --config") and ignore the warnings about gcloud.
cr-buildbucket or chrome-infra-logs?

Comment 7 by hinoka@chromium.org, Jun 15 2017

Just type "0" for the project-id
Owner: hinoka@chromium.org
Status: Assigned (was: Available)

Comment 9 by d...@chromium.org, Jun 19 2017

Components: -Infra>Git>Admin
This is not Git-related.
Status: Fixed (was: Assigned)
There was a workaround.

Sign in to add a comment