New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 824915 link

Starred by 7 users

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Mar 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

Repo sync failed due bad URL request error 400

Project Member Reported by haoweiw@google.com, Mar 22 2018

Issue description

I cannot fetch internal code repository due to below errors. 

Looks like this is a URL request error to "android.googlesource.com"

error: Cannot fetch platform/external/puffin
error: Cannot fetch platform/external/libunwind_llvm
error: Cannot fetch toolchain/binutils
error: Cannot fetch platform/external/bsdiff
error: Cannot fetch platform/external/libmojo
fatal: unable to access 'https://android.googlesource.com/platform/external/libcxx/': The requested URL returned error: 400
fatal: unable to access 'https://android.googlesource.com/platform/external/minijail/': The requested URL returned error: 400
fatal: unable to access 'https://android.googlesource.com/toolchain/gcc/': The requested URL returned error: 400
error: Cannot fetch platform/external/libcxx
error: Cannot fetch platform/external/minijail
error: Cannot fetch toolchain/gcc

error: Exited sync due to fetch errors

 
what is the command you are using to fetch?

Comment 2 by haoweiw@google.com, Mar 22 2018

repo sync
Can you paste the complete command and the directory under which you ran it?

I assume the internal code repository you mean is chromeos-admin?
chromeos-admin (master|✔) $ repo sync .
remote: Finding sources: 100% (11/11)           
remote: Total 11 (delta 5), reused 11 (delta 5)        
From https://chrome-internal.googlesource.com/chromeos/manifest-internal
   015bb9e..89bc467  master               -> origin/master
 * [new branch]      stabilize-10323.62.B -> origin/stabilize-10323.62.B
project .repo/manifests/
Updating 015bb9e..89bc467
Fast-forward
 internal_full.xml | 4 ++++
 1 file changed, 4 insertions(+)

Fetching project repo
Fetching project chromeos/chromeos-admin
remote: Counting objects: 29, done        
remote: Finding sources: 100% (29/29)           
remote: Total 29 (delta 15), reused 29 (delta 15)        
From https://chrome-internal.googlesource.com/chromeos/chromeos-admin
   45c1759e..d02542df  master     -> cros-internal/master
Fetching projects: 100% (2/2), done.  

I can't reproduce the failure you have?

Comment 4 by haoweiw@google.com, Mar 22 2018

I mean chromiumos.
haoweiw@haoweiw:~/chromiumos$ repo sync

haoweiw@haoweiw:~/chromiumos$ repo sync > /tmp/repo.txt
Fetching projects:   2% (11/504)  Auto packing deferred; not on AC
Fetching projects:  22% (111/504)  fatal: unable to access 'https://android.googlesource.com/platform/external/libcxxabi/': The requested URL returned error: 400
Fetching projects:  24% (121/504)  fatal: unable to access 'https://android.googlesource.com/platform/external/libunwind_llvm/': The requested URL returned error: 400
Fetching projects:  25% (126/504)  fatal: unable to access 'https://android.googlesource.com/platform/external/bsdiff/': The requested URL returned error: 400
Fetching projects:  33% (167/504)  fatal: unable to access 'https://android.googlesource.com/platform/external/puffin/': The requested URL returned error: 400
Fetching projects:  34% (172/504)  fatal: unable to access 'https://android.googlesource.com/platform/external/libmojo/': The requested URL returned error: 400
Fetching projects:  35% (177/504)  fatal: unable to access 'https://android.googlesource.com/toolchain/binutils/': The requested URL returned error: 400
Fetching projects:  36% (182/504)  Auto packing deferred; not on AC
Fetching projects:  37% (187/504)  remote: Total 0 (delta 0), reused 0 (delta 0)        
Fetching projects:  41% (207/504)  Auto packing deferred; not on AC
Fetching projects:  48% (242/504)  Auto packing deferred; not on AC
remote: Counting objects: 4207, done        ounting objects: 1           
remote: Finding sources: 100% (10/10)           
remote: Total 10 (delta 5), reused 10 (delta 5)        
From https://chromium.googlesource.com/a/chromiumos/overlays/board-overlays
   3f5157a0d6..b44c7079f3  factory-scarlet-10211.B -> cros/factory-scarlet-10211.B
Auto packing deferred; not on AC
Fetching projects:  56% (283/504)  Auto packing deferred; not on AC
Fetching projects:  68% (343/504)  Auto packing deferred; not on AC
Fetching projects:  74% (373/504)  fatal: unable to access 'https://android.googlesource.com/platform/external/minijail/': The requested URL returned error: 400
Fetching projects:  76% (384/504)  fatal: unable to access 'https://android.googlesource.com/platform/external/libcxx/': The requested URL returned error: 400
Fetching projects:  82% (414/504)  Auto packing deferred; not on AC
Fetching projects:  84% (424/504)  Auto packing deferred; not on AC
Fetching projects:  95% (479/504)  fatal: unable to access 'https://android.googlesource.com/toolchain/gcc/': The requested URL returned error: 400
Fetching projects:  96% (484/504)  Auto packing deferred; not on AC
Auto packing deferred; not on AC
Fetching projects:  98% (494/504)  fatal: unable to access 'https://android.googlesource.com/toolchain/binutils/': The requested URL returned error: 400
fatal: unable to access 'https://android.googlesource.com/platform/external/libcxxabi/': The requested URL returned error: 400
fatal: unable to access 'https://android.googlesource.com/platform/external/libunwind_llvm/': The requested URL returned error: 400
fatal: unable to access 'https://android.googlesource.com/platform/external/bsdiff/': The requested URL returned error: 400
fatal: unable to access 'https://android.googlesource.com/platform/external/puffin/': The requested URL returned error: 400
fatal: unable to access 'https://android.googlesource.com/platform/external/libmojo/': The requested URL returned error: 400
fatal: unable to access 'https://android.googlesource.com/platform/external/libcxx/': The requested URL returned error: 400
fatal: unable to access 'https://android.googlesource.com/platform/external/minijail/': The requested URL returned error: 400
error: Cannot fetch toolchain/binutils
fatal: unable to access 'https://android.googlesource.com/toolchain/gcc/': The requested URL returned error: 400
error: Cannot fetch platform/external/libunwind_llvm
error: Cannot fetch platform/external/bsdiff
error: Cannot fetch platform/external/libcxxabi
error: Cannot fetch platform/external/libmojo
error: Cannot fetch platform/external/puffin
error: Cannot fetch platform/external/minijail
error: Cannot fetch platform/external/libcxx
error: Cannot fetch toolchain/gcc

error: Exited sync due to fetch errors


yep, I also have the similar failure. 
Owner: akes...@chromium.org
Status: Assigned (was: Untriaged)
I tried to git clone the failed repo in a tmp folder:

$ git clone https://android.googlesource.com/platform/external/libunwind_llvm
Cloning into 'libunwind_llvm'...
fatal: unable to access 'https://android.googlesource.com/platform/external/libunwind_llvm/': The requested URL returned error: 400

Maybe we don't have the permission to access https://android.googlesource.com?
... aaaand I'm seeing it, too.

The repos complained of are exactly the repos that specify
`remote="aosp"` in the manifest.  So, basically, there's something
wrong with android.googlesource.com...

We're not seeing this on the internal builders, so I don't have a full
explanation.

Curiously, I had the inverse of this problem: repo sync on Android failed to fetch repos hosted on chromium.googlesource.com. Comments on b:74247685 helped me unblock the sync failures: these two issues might be related.
I've been seeing this too, on both android and chromium (but not chrome-internal), and only on my workstation. I tried from home (on the first two; no access to chrome-internal from home) and had no problem.

$ git ls-remote https://chrome-internal.googlesource.com/chromeos/overlays/overlay-scarlet-private | head
ee8b34eb77caaa0e989d1384a1a7f46bcffae61d	HEAD
ca100a41f6a884a7a6ee27fb98f3bf91b9413ec9	refs/changes/00/589500/1
7299ee82248c082a10d5b37442a21ef2860b3b57	refs/changes/00/589500/2
606c39d75efe152fffb34e5eb1fa33fda23b8eea	refs/changes/00/589500/meta
0833b88b9f6240a37ff898a047cdef05d996a496	refs/changes/00/589600/1
4a0cb298e988ce6d985e07f3ae3d22b1bcc1da24	refs/changes/00/589600/2
cbff1981aa82700385a46d178d2bb6f51c563311	refs/changes/00/589600/meta
e6c4ab267c055c73759e5ef01958ef6a61919018	refs/changes/00/590100/1
0bdda4923006afb5f945a6c8a825fc147dfd514b	refs/changes/00/590100/2
823c2c6cc952e241a3a8b6f2c521f29524ec9d60	refs/changes/00/590100/meta
$ git ls-remote https://chromium.googlesource.com/chromiumos/chromite
fatal: unable to access 'https://chromium.googlesource.com/chromiumos/chromite/': The requested URL returned error: 400
$ git ls-remote https://chromium.googlesource.com/chromiumos/platform2
fatal: unable to access 'https://chromium.googlesource.com/chromiumos/platform2/': The requested URL returned error: 400
$ git ls-remote https://android.googlesource.com/device/generic/mips
fatal: unable to access 'https://android.googlesource.com/device/generic/mips/': The requested URL returned error: 400
$ git ls-remote https://android.googlesource.com/device/generic/x86_64
fatal: unable to access 'https://android.googlesource.com/device/generic/x86_64/': The requested URL returned error: 400

FWIW, chrome-internal is resolving to a different IP than android and chromium. And chromium and android resolved to different IPs at home (where things worked). But I tried swapping IPs in my /etc/hosts, and the problems still followed my machine.
Labels: -Pri-1 Pri-0
Dare I say this is a Pri-0? I can't get any work done.
Owner: jrbarnette@chromium.org
I'm digging in now.  Can't say as I actually understand.

First step (for everyone) is to see if this is related to b/74247685.

OK.  After digging through the buganizer bug, I got it to work.
Here's what I did:

 1. Visited this URL:
    http://android.googlesource.com/new-password
 2. Signed in with corp credentials.
 3. Followed the instructions to update ~/.gitcookies.

I'm getting sporadic reports of similar problems for chromium (not android)
access; I'm not seeing those problems, so I can't say if they're related
or if the instructions above (mutatis mutandis) will help.  Someone should
try it.
Labels: -Pri-0 -Hardware-Lab Pri-1
Downgrading, since I think we have a workaround/fix.

We can re-escalate if someone comes forward who's tried the
steps in comment #12 but still experiences the problem.

Comment 14 by haoweiw@google.com, Mar 23 2018

The fix works for me. :) 
Yes, works for me on chromium so far (need to try with android too). I did have to still massage my .gitcookies, because now I had several entries for the same hosts. It seems like git might have still been picking up the old ones.

Thanks Richard!

I'll have to read through the other bug sometime, to see if we expect a lot of people to hit this. Seems like a PSA would be in order if so?
> I'll have to read through the other bug sometime [ ... ]

My reading was that the problem was the 6th field in the entry:
    android.googlesource.com,FALSE,/,TRUE,2147483647,o,...

If that field were "gi" instead of "o", it would fail.  I can't
even pretend to understand what that means.

Comment 17 by haoweiw@google.com, Mar 23 2018

I also have to remove one of the entry to make it work. 

android.googlesource.com        FALSE   /       TRUE    0       gi      mogbm58jhJR1KWqM12jqGWW8MVLfcRjmDBRZda9qpYo=haoweiw%40google.com

> If that field were "gi" instead of "o", it would fail.  I can't
even pretend to understand what that means.

Looks like the cookie name + format changed.  The new "o" cookies start with "git-cernekee.google.com=".

I tried changing the "gi" cookie name to "o" while leaving the old value intact, but that didn't work.  Instead of HTTP error 400, `repo sync` told me to visit the new-password page.

Regenerating the password did work.  Unsure of whether I also need to update ~/.netrc.
> I tried changing the "gi" cookie name to "o" while leaving the old
> value intact, but that didn't work.  Instead of HTTP error 400,
> `repo sync` told me to visit the new-password page.

Yes, that was my experience, too, which is why the instructions for
the fix start with the new-password page.  :-)

> Regenerating the password did work.  Unsure of whether I also need
> to update ~/.netrc.

I didn't update my .netrc, and all of my workflows are fine.  YMMV.

The instructions for users to fix this are receding further and
further back as folks add to this bug, so to repeat, here's what has
to be done:
 1. Visit this URL:
    http://android.googlesource.com/new-password
 2. Sign in with corp credentials.
 3. Follow the instructions to update ~/.gitcookies.

You may also need to do this for chromeos-internal, mutatis mutandis.

Status: Fixed (was: Assigned)
I've sent a PSA to chromium-os-dev@chromium.org.  At this point,
everything that might be done centrally has been done; it's up to
individual developers to follow the instructions above.

Comment 21 by sjg@chromium.org, Mar 29 2018

This worked for me (fixed asop), along with #17, thanks.

Comment 22 by sjg@chromium.org, Mar 29 2018

 Issue 819443  has been merged into this issue.
Cc: allenwebb@chromium.org jclinton@chromium.org vapier@chromium.org phoenixshen@chromium.org
 Issue 848923  has been merged into this issue.

Sign in to add a comment