Thanks Michael for reporting this. I'm I good person to start with this as I added that DEP.

Can you give me more details on the failure? I just synced now and it's working okay.

Also, can you tell if other directories in third_party/android_build_tools are getting synced correctly--in particular aapt2/ and bundletool/?

Could you also paste the .gclient of the checkout that's having problems?

Portion of the logs:

________ running 'cipd ensure -log-level error -root /home/jayrao/google_cast/chromium -ensure-file /tmp/tmpW2L6Qq.ensure' in '.'
[P16628 15:06:33.436 client.go:1371 E] cipd: failed to fetch chromium/third_party/android_build_tools/art:87169fbc701d244c311e6aa8843591a7f1710bc0 - failed to download the package file after multiple attempts
[P16628 15:06:33.460 client.go:1579 E] Failed to install chromium/third_party/android_build_tools/art:87169fbc701d244c311e6aa8843591a7f1710bc0 - failed to download the package file after multiple attempts
[P16628 15:07:49.586 client.go:1371 E] cipd: failed to fetch chromium/third_party/android_sdk/public/emulator:731059df93885359487fd9d6085bc7804a23f9c8 - failed to download the package file after multiple attempts
[P16628 15:07:49.608 client.go:1579 E] Failed to install chromium/third_party/android_sdk/public/emulator:731059df93885359487fd9d6085bc7804a23f9c8 - failed to download the package file after multiple attempts
[P16628 15:09:00.943 client.go:1371 E] cipd: failed to fetch chromium/third_party/android_sdk/public/extras:2f33032ef348e2ff37d90d53563f4cb030a879bf - failed to download the package file after multiple attempts
[P16628 15:09:00.966 client.go:1579 E] Failed to install chromium/third_party/android_sdk/public/extras:2f33032ef348e2ff37d90d53563f4cb030a879bf - failed to download the package file after multiple attempts
[P16628 15:10:09.363 client.go:1371 E] cipd: failed to fetch chromium/third_party/android_sdk/public/tools:1a659d51804abb9461cd19aeffc2102e47a15a25 - failed to download the package file after multiple attempts
[P16628 15:10:09.386 client.go:1579 E] Failed to install chromium/third_party/android_sdk/public/tools:1a659d51804abb9461cd19aeffc2102e47a15a25 - failed to download the package file after multiple attempts
[P16628 15:11:21.464 client.go:1371 E] cipd: failed to fetch chromium/third_party/robolectric:dd4d16960b0242cf42c31da8545fb2c003bae478 - failed to download the package file after multiple attempts
[P16628 15:11:21.486 client.go:1579 E] Failed to install chromium/third_party/robolectric:dd4d16960b0242cf42c31da8545fb2c003bae478 - failed to download the package file after multiple attempts
Error: failed to update packages, see the log.

And b/119058642 contains the full log & would be a good place to communicate with the partner directly. Note that the user is not a google/chromium eng.

OK, it looks like the user is not able to fetch anything of the third_party code. So this doesn't look specific to android_build_tools/art, that just happens to be alphabetized first :)

jayrao@, could you include your .gclient as well as what happens when doing git cl creds-check? Or what happens when you run build/install-build-deps.sh?

(I'm not totally sure how credentials are supposed to work for external partners. But clearly you need some sort of cpid access to download the third_party stuff).

 sh build/install-build-deps.sh --no-chromeos-fonts
build/install-build-deps.sh: 89: build/install-build-deps.sh: [[: not found
Running as non-root user.
You might have to enter your password one or more times for 'sudo'.

build/install-build-deps.sh: 311: build/install-build-deps.sh: backwards_compatible_list+=       libgbm-dev-lts-trusty
      libgl1-mesa-dev-lts-trusty
      libgl1-mesa-glx-lts-trusty:i386
      libgles2-mesa-dev-lts-trusty
      mesa-common-dev-lts-trusty: not found
build/install-build-deps.sh: 371: build/install-build-deps.sh: arm_list+= g++-4.8-multilib-arm-linux-gnueabihf
                gcc-4.8-multilib-arm-linux-gnueabihf: not found
Skipping debugging symbols.
Including 32-bit libraries.
Including ARM cross toolchain.
Including NaCl, NaCl toolchain, NaCl ports dependencies.
Ign http://us.archive.ubuntu.com trusty InRelease
Hit http://security.ubuntu.com trusty-security InRelease
Hit http://us.archive.ubuntu.com trusty-updates InRelease
Hit http://us.archive.ubuntu.com trusty-backports InRelease
Hit http://security.ubuntu.com trusty-security/main Sources
Hit http://us.archive.ubuntu.com trusty Release.gpg
Hit http://us.archive.ubuntu.com trusty-updates/main Sources
Hit http://security.ubuntu.com trusty-security/restricted Sources
Hit http://us.archive.ubuntu.com trusty-updates/restricted Sources
Hit http://security.ubuntu.com trusty-security/universe Sources
Hit http://us.archive.ubuntu.com trusty-updates/universe Sources
Hit http://us.archive.ubuntu.com trusty-updates/multiverse Sources
Hit http://security.ubuntu.com trusty-security/multiverse Sources
Hit http://us.archive.ubuntu.com trusty-updates/main amd64 Packages
Hit http://us.archive.ubuntu.com trusty-updates/restricted amd64 Packages
Hit http://security.ubuntu.com trusty-security/main amd64 Packages
Hit http://us.archive.ubuntu.com trusty-updates/universe amd64 Packages
Hit http://security.ubuntu.com trusty-security/restricted amd64 Packages
Hit http://us.archive.ubuntu.com trusty-updates/multiverse amd64 Packages
Hit http://us.archive.ubuntu.com trusty-updates/main i386 Packages
Hit http://security.ubuntu.com trusty-security/universe amd64 Packages
Hit http://us.archive.ubuntu.com trusty-updates/restricted i386 Packages
Hit http://security.ubuntu.com trusty-security/multiverse amd64 Packages
Hit http://us.archive.ubuntu.com trusty-updates/universe i386 Packages
Hit http://us.archive.ubuntu.com trusty-updates/multiverse i386 Packages
Hit http://security.ubuntu.com trusty-security/main i386 Packages
Hit http://us.archive.ubuntu.com trusty-updates/main Translation-en
Hit http://us.archive.ubuntu.com trusty-updates/multiverse Translation-en
Hit http://security.ubuntu.com trusty-security/restricted i386 Packages
Hit http://us.archive.ubuntu.com trusty-updates/restricted Translation-en
Hit http://security.ubuntu.com trusty-security/universe i386 Packages
Hit http://us.archive.ubuntu.com trusty-updates/universe Translation-en
Hit http://us.archive.ubuntu.com trusty-backports/main Sources
Hit http://security.ubuntu.com trusty-security/multiverse i386 Packages
Hit http://us.archive.ubuntu.com trusty-backports/restricted Sources
Hit http://security.ubuntu.com trusty-security/main Translation-en
Hit http://us.archive.ubuntu.com trusty-backports/universe Sources
Hit http://us.archive.ubuntu.com trusty-backports/multiverse Sources
Hit http://security.ubuntu.com trusty-security/multiverse Translation-en
Hit http://us.archive.ubuntu.com trusty-backports/main amd64 Packages
Hit http://us.archive.ubuntu.com trusty-backports/restricted amd64 Packages
Hit http://security.ubuntu.com trusty-security/restricted Translation-en
Hit http://us.archive.ubuntu.com trusty-backports/universe amd64 Packages
Hit http://us.archive.ubuntu.com trusty-backports/multiverse amd64 Packages
Hit http://security.ubuntu.com trusty-security/universe Translation-en
Hit http://us.archive.ubuntu.com trusty-backports/main i386 Packages
Hit http://us.archive.ubuntu.com trusty-backports/restricted i386 Packages
Hit http://us.archive.ubuntu.com trusty-backports/universe i386 Packages
Hit http://us.archive.ubuntu.com trusty-backports/multiverse i386 Packages
Hit http://us.archive.ubuntu.com trusty-backports/main Translation-en
Hit http://us.archive.ubuntu.com trusty-backports/multiverse Translation-en
Hit http://us.archive.ubuntu.com trusty-backports/restricted Translation-en
Hit http://us.archive.ubuntu.com trusty-backports/universe Translation-en
Hit http://us.archive.ubuntu.com trusty Release
Hit http://us.archive.ubuntu.com trusty/main Sources
Hit http://us.archive.ubuntu.com trusty/restricted Sources
Hit http://us.archive.ubuntu.com trusty/universe Sources
Hit http://us.archive.ubuntu.com trusty/multiverse Sources
Hit http://us.archive.ubuntu.com trusty/main amd64 Packages
Hit http://us.archive.ubuntu.com trusty/restricted amd64 Packages
Hit http://us.archive.ubuntu.com trusty/universe amd64 Packages
Hit http://us.archive.ubuntu.com trusty/multiverse amd64 Packages
Hit http://us.archive.ubuntu.com trusty/main i386 Packages
Hit http://us.archive.ubuntu.com trusty/restricted i386 Packages
Hit http://us.archive.ubuntu.com trusty/universe i386 Packages
Hit http://us.archive.ubuntu.com trusty/multiverse i386 Packages
Hit http://us.archive.ubuntu.com trusty/main Translation-en
Hit http://us.archive.ubuntu.com trusty/multiverse Translation-en
Hit http://us.archive.ubuntu.com trusty/restricted Translation-en
Hit http://us.archive.ubuntu.com trusty/universe Translation-en
Reading package lists... Done
W: Duplicate sources.list entry http://us.archive.ubuntu.com/ubuntu/ trusty-backports/main amd64 Packages (/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_trusty-backports_main_binary-amd64_Packages)
W: Duplicate sources.list entry http://us.archive.ubuntu.com/ubuntu/ trusty-backports/restricted amd64 Packages (/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_trusty-backports_restricted_binary-amd64_Packages)
W: Duplicate sources.list entry http://us.archive.ubuntu.com/ubuntu/ trusty-backports/universe amd64 Packages (/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_trusty-backports_universe_binary-amd64_Packages)
W: Duplicate sources.list entry http://us.archive.ubuntu.com/ubuntu/ trusty-backports/multiverse amd64 Packages (/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_trusty-backports_multiverse_binary-amd64_Packages)
W: Duplicate sources.list entry http://us.archive.ubuntu.com/ubuntu/ trusty-backports/main i386 Packages (/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_trusty-backports_main_binary-i386_Packages)
W: Duplicate sources.list entry http://us.archive.ubuntu.com/ubuntu/ trusty-backports/restricted i386 Packages (/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_trusty-backports_restricted_binary-i386_Packages)
W: Duplicate sources.list entry http://us.archive.ubuntu.com/ubuntu/ trusty-backports/universe i386 Packages (/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_trusty-backports_universe_binary-i386_Packages)
W: Duplicate sources.list entry http://us.archive.ubuntu.com/ubuntu/ trusty-backports/multiverse i386 Packages (/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_trusty-backports_multiverse_binary-i386_Packages)
W: You may want to run apt-get update to correct these problems
Finding missing packages...
Packages required:  libasound2:i386 libcap2:i386 libelf-dev:i386 libfontconfig1:i386 libglib2.0-0:i386 libgpm2:i386 libgtk-3-0:i386 libncurses5:i386 libnss3:i386 libpango1.0-0:i386 libpci3:i386 libssl-dev:i386 libssl1.0.0:i386 libtinfo-dev:i386 libudev1:i386 libuuid1:i386 libx11-xcb1:i386 libxcomposite1:i386 libxcursor1:i386 libxdamage1:i386 libxi6:i386 libxrandr2:i386 libxss1:i386 libxtst6:i386 linux-libc-dev:i386 ant apache2.2-bin autoconf binutils binutils-aarch64-linux-gnu binutils-arm-linux-gnueabihf bison bzip2 cdbs cmake curl dbus-x11 devscripts dpkg-dev elfutils fakeroot flex g++ g++-4.8-multilib g++-arm-linux-gnueabihf g++-mingw-w64-i686 gawk git-core git-svn gperf intltool lib32gcc1 lib32ncurses5-dev lib32stdc++6 lib32z1-dev libapache2-mod-php5 libappindicator3-1 libappindicator3-dev libasound2 libasound2-dev libatk1.0-0 libatspi2.0-0 libatspi2.0-dev libav-tools libbluetooth-dev libbrlapi-dev libbrlapi0.6 libbz2-1.0 libbz2-dev libc6 libc6-dev-armhf-cross libc6-i386 libcairo2 libcairo2-dev libcap-dev libcap2 libcups2 libcups2-dev libcurl4-gnutls-dev libdrm-dev libelf-dev libexpat1 libffi-dev libffi6 libfontconfig1 libfreetype6 libgbm-dev libglib2.0-0 libglib2.0-dev libglu1-mesa-dev libgnome-keyring-dev libgnome-keyring0 libgtk-3-0 libgtk-3-dev libjpeg-dev libkrb5-dev libnspr4 libnspr4-dev libnss3 libnss3-dev libpam0g libpam0g-dev libpango1.0-0 libpci-dev libpci3 libpcre3 libpixman-1-0 libpng12-0 libpulse-dev libpulse0 libsctp-dev libspeechd-dev libspeechd2 libsqlite3-0 libsqlite3-dev libssl-dev libstdc++6 libtinfo-dev libtool libudev-dev libudev1 libuuid1 libwayland-egl1-mesa libwww-perl libx11-6 libx11-xcb1 libxau6 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxdmcp6 libxext6 libxfixes3 libxi6 libxinerama1 libxkbcommon-dev libxrandr2 libxrender1 libxslt1-dev libxss-dev libxt-dev libxtst-dev libxtst6 linux-libc-dev-armhf-cross locales openbox p7zip patch perl php5-cgi pkg-config python python-cherrypy3 python-crypto python-dev python-numpy python-opencv python-openssl python-psutil python-yaml realpath rpm ruby subversion texinfo uuid-dev wdiff x11-utils xcompmgr xsltproc xutils-dev xvfb xz-utils zip zlib1g

No missing packages, and the packages are up to date.

Skipping installation of Chrome OS fonts.
Installing locales.
Generating locales...
  da_DK.UTF-8... up-to-date
Generation complete.
Generating locales...
  fr_FR.UTF-8... done
Generation complete.
Generating locales...
  he_IL.UTF-8... done
Generation complete.
Generating locales...
  zh_TW.UTF-8... done
Generation complete

Matt, please note that jayroa was able to sync after applying this change locally to the deps file:

https://paste.googleplex.com/6010583807688704

So it looks like he was able to sync some dependencies, but specifically these ones failed.

What about git cl creds-check or your .gclient?

I think jayroa is missing access to our cpid repository somehow. What happens if you run the following?

cipd resolve chromium/third_party/android_build_tools/art -version 87169fbc701d244c311e6aa8843591a7f1710bc0

Also (as recently seen on a thread), after undoing your DEPS patch, try the following and see if it helps.

~/clankium$ rm -rf .cipd
~/clankium/src$ gclient sync

Hi Matt,

git cl creds-check
Traceback (most recent call last):
  File "/home/jayrao/bin/depot_tools/metrics.py", line 262, in print_notice_and_exit
    yield
  File "/home/jayrao/bin/depot_tools/git_cl.py", line 5919, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/home/jayrao/bin/depot_tools/git_cl.py", line 5901, in main
    return dispatcher.execute(OptionParser(), argv)
  File "/home/jayrao/bin/depot_tools/subcommand.py", line 252, in execute
    return command(parser, args[1:])
  File "/home/jayrao/bin/depot_tools/git_cl.py", line 3960, in CMDcreds_check
    authn = gerrit_util.Authenticator.get()
  File "/home/jayrao/bin/depot_tools/gerrit_util.py", line 99, in get
    if GceAuthenticator.is_gce():
  File "/home/jayrao/bin/depot_tools/gerrit_util.py", line 285, in is_gce
    cls._cache_is_gce = cls._test_is_gce()
  File "/home/jayrao/bin/depot_tools/gerrit_util.py", line 292, in _test_is_gce
    resp, _ = cls._get(cls._INFO_URL)
  File "/home/jayrao/bin/depot_tools/gerrit_util.py", line 305, in _get
    resp, contents = c.request(url, 'GET', **kwargs)
  File "/home/jayrao/bin/depot_tools/third_party/httplib2/__init__.py", line 1694, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/home/jayrao/bin/depot_tools/third_party/httplib2/__init__.py", line 1434, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/home/jayrao/bin/depot_tools/third_party/httplib2/__init__.py", line 1354, in _conn_request
    conn.connect()
  File "/home/jayrao/bin/depot_tools/third_party/httplib2/__init__.py", line 961, in connect
    self.sock.setproxy(proxy_type, proxy_host, proxy_port, proxy_rdns, proxy_user, proxy_pass, proxy_headers)
TypeError: setproxy() takes at most 7 arguments (8 given)


cipd resolve chromium/third_party/android_build_tools/art -version 87169fbc701d244c311e6aa8843591a7f1710bc0
Packages:
  chromium/third_party/android_build_tools/art:87169fbc701d244c311e6aa8843591a7f1710bc0

I did try this:

jayrao@SC-KVMCLT11:~ rm -rf .cipd
jayrao@SC-KVMCLT11:~/google_cast/chromium$ gclient sync --with_branch_heads --di
sable-syntax-validation

________ running 'cipd ensure -log-level error -root /home/jayrao/google_cast/chromium -ensure-file /tmp/tmpJDNDFt.ensure' in '.'
[P7954 11:00:12.302 client.go:1357 E] cipd: failed to fetch chromium/third_party/android_build_tools/art:87169fbc701d244c311e6aa8843591a7f1710bc0 - failed to download the package file after multiple attempts
[P7954 11:00:12.323 client.go:1570 E] Failed to install chromium/third_party/android_build_tools/art:87169fbc701d244c311e6aa8843591a7f1710bc0 - failed to download the package file after multiple attempts

.gclient:

"""Adds cast-specific configutation and hooks to `gclient sync`."""

solutions = [{
    'managed': False,
    'name': 'src',
    'url': 'https://chromium.googlesource.com/chromium/src.git',
    'deps_file': '.DEPS.git'
}]
target_os = ['android', 'unix', 'fuchsia']


# A note to developers:
# If you are adding or modifying a hook that depends on a file hosted in a
# eureka-internal project, you need to instruct the Cast CQ to re-run `gclient
# sync` if this file changes in a CL. This is necessary to test changes to these
# files properly on the CQ. For an example CL demonstrating how to do this, see
# eureka-internal/74402.
hooks = [
    {
        'pattern': '.',
        'action': ['src/chromecast/internal/build/fetch_google3_binaries.py'],
    },
    {
        'pattern': '.',
        'action': [
            'src/chromecast/internal/build/fetch_internal_only_builddeps.py'
        ],
    },
    {
        'pattern': '.',
        'action': [
            'src/chromecast/internal/build/fetch_libassistant_binary.sh'],
    },
    {
        'pattern': '.',
        'action': ['src/chromecast/internal/build/apply_v8_patches.sh'],
    }
]


Thanks
Sanjay

Just to add the gclient ultimately fails:

[P7954 11:05:59.139 client.go:1357 E] cipd: failed to fetch chromium/third_party/android_sdk/public/emulator:731059df93885359487fd9d6085bc7804a23f9c8 - failed to download the package file after multiple attempts
[P7954 11:05:59.159 client.go:1570 E] Failed to install chromium/third_party/android_sdk/public/emulator:731059df93885359487fd9d6085bc7804a23f9c8 - failed to download the package file after multiple attempts
[P7954 11:06:59.540 client.go:1357 E] cipd: failed to fetch chromium/third_party/android_sdk/public/extras:2f33032ef348e2ff37d90d53563f4cb030a879bf - failed to download the package file after multiple attempts
[P7954 11:06:59.559 client.go:1570 E] Failed to install chromium/third_party/android_sdk/public/extras:2f33032ef348e2ff37d90d53563f4cb030a879bf - failed to download the package file after multiple attempts
[P7954 11:08:21.228 client.go:1357 E] cipd: failed to fetch chromium/third_party/android_sdk/public/tools:1a659d51804abb9461cd19aeffc2102e47a15a25 - failed to download the package file after multiple attempts
[P7954 11:08:21.250 client.go:1570 E] Failed to install chromium/third_party/android_sdk/public/tools:1a659d51804abb9461cd19aeffc2102e47a15a25 - failed to download the package file after multiple attempts
[P7954 11:10:52.598 client.go:1357 E] cipd: failed to fetch chromium/third_party/robolectric:dd4d16960b0242cf42c31da8545fb2c003bae478 - failed to download the package file after multiple attempts
[P7954 11:10:52.621 client.go:1570 E] Failed to install chromium/third_party/robolectric:dd4d16960b0242cf42c31da8545fb2c003bae478 - failed to download the package file after multiple attempts
Error: failed to update packages, see the log.
Error: Command 'cipd ensure -log-level error -root /home/jayrao/google_cast/chromium -ensure-file /tmp/tmpJDNDFt.ensure' returned non-zero exit status 1

Thanks
Sanjay

If you go to the following link, what do you see in the "ACLs" column? You will probably need to follow the sign-in link that appears on the page.

https://chrome-infra-packages.appspot.com/p/chromium/third_party/android_build_tools/art/+/

Hi Matt,

ACLs
Your roles in the prefix are:

Reader
To view all ACLs you must be an owner.

Thanks
Sanjay

OK, those permissions look okay. Are you correctly logged in for the terminal that you're running gclient sync on?

As a googler I run download_from_google_storage --config (this command is in in depot_tools). 

Alternatively look in the [Credentials] section of ~/.boto and see if you have a gs_oauth2_refresh_token line.

cat .boto | grep gs_oauth2_refresh_token
gs_oauth2_refresh_token = 1/p-UoyCEthLWCy-qc4ZQVq4OG-hjo1e6uVxmiUZ2LRmU

I am logged in correctly otherwise My sync should fail due to lack of permissions for the rest of the code as well

Ok I'm not sure what's going on. +jbudorick for the general infra problem.

Hi jbudorick,
Can you check what else could be wrong here?

Thanks
Sanjay

+vadimsh: any idea why cipd would fail to download publicly accessible packages?

This looks like something related to network configuration. CIPD client can't reach Google Storage URL. Do you run it behind a proxy? Try setting HTTP_PROXY/HTTPS_PROXY environment variables.

If you go here https://chrome-infra-packages.appspot.com/p/chromium/third_party/android_sdk/public/emulator/+/731059df93885359487fd9d6085bc7804a23f9c8 and click Download icon, does it work?

Re #14: revoke this token ASAP (if haven't already) and do not post refresh tokens anywhere. Anyone who can see it (and this bug is publicly visible) can access/modify all your Google Storage files. See https://developers.google.com/identity/protocols/OAuth2WebServer#tokenrevoke