New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 105 users
Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Apr 2010
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 2
Type: Bug

Restricted
  • Only users with Commit permission may comment.



Sign in to add a comment
Google chrome repository takes 5 minutes to send the response headers
Reported by alk...@gmail.com, Mar 19 2010 Back to list
Chrome Version       : 5.0.307.11 beta

What steps will reproduce the problem?
1. Install google chrome
2. run: apt-get update

What is the expected result?
It should finish in 5 secs.

What happens instead?
It needs 5 minutes.

Please provide any additional information below. Attach a screenshot if
possible.

While installing Chrome under Ubuntu, it puts its repository in my sources:
$ cat /etc/apt/sources.list.d/google-chrome.list 
deb http://dl.google.com/linux/deb/ stable main

Then, on every `apt-get update`, it stucks for about 5 minutes on that
page, telling me "99% [Waiting for headers]".
Then I get this:
  Ign http://dl.google.com/linux/deb/ stable/main Translation-en_US 

But, the following 2 hits happen in less than a second:
  Get:2 http://dl.google.com stable Release [2,540B]
  Get:3 http://dl.google.com stable/main Packages [916

So I guess the problem is that http://dl.google.com needs 5 minutes to send
a "Page not found" or something like that.

I've verified that this also happens for a lot of other users before
reporting it.
 
Comment 1 by mmoss@chromium.org, Mar 19 2010
Labels: -Area-Undefined Area-Internals OS-Linux
Status: ExternalDependency
There is a known issue with HTTP pipelining. For a workaround, please try:

sudo apt-get -o Acquire::http::Pipeline-Depth=0 update

Comment 2 by mmoss@chromium.org, Mar 19 2010
 Issue 37346  has been merged into this issue.
Comment 3 by evan@chromium.org, Mar 24 2010
 Issue 39119  has been merged into this issue.
Comment 4 by evan@chromium.org, Mar 24 2010
Labels: Mstone-5 ReleaseBlock-Stable
Labels: -ReleaseBlock-stable ReleaseBlock-Beta
If this is an issue with HTTP/1.1 pipelining, it's the server's fault. From the man
page for apt.conf:

One setting is provided to control the pipeline depth in cases where the remote
server is not RFC conforming or buggy (such as Squid 2.0.2). 
Acquire::http::Pipeline-Depth can be a value from 0 to 5 indicating how many
outstanding requests APT should send. A value of zero MUST be specified if the remote
host does not properly linger on TCP connections - otherwise data corruption will
occur. Hosts which require this are in violation of RFC 2068.

Since setting this flag to 0 makes it work, it would seem that the web server is
mis-configured in some way and/or is violating specifications. Looking at a TCP dump
file (and comparing it to other dump files from different sources), it appears that
dl.google.com is not returning the "Connection: close" header. Since it doesn't seem
to want to support keep-alive properly, it should send that header. From the RFC

   HTTP/1.1 applications that do not support persistent connections MUST
   include the "close" connection option in every message. (14.10)

So, possible solutions are:
1) Enable keep-alive, so that apt's pipelining works properly
2) Send the connection: close header to inform apt that it's not going to respond to
more than one file per request

If you want TCP dump files, I will provide them upon request.
Comment 7 by jonol...@gmail.com, Mar 26 2010
I too am experiencing this problem on both my ubuntu machines and my friend also has 
the same problem on his computer. As has been pointed out, this is strictly a problem with 
the google repositories, as soon as these are commented out or removed, apt-get updates 
speed up from about 2-5 minutes to about 5 seconds!

This is incredibly annoying as each time I want to update my computer, I have to wait a 
few minutes - whereas before it would only take seconds. This will affect ANYBODY who 
installs Google Chrome on their Debian or Ubuntu box.

I suggest that Google look into this issue ASAP!

Thanks
This problem forced me to switch to the Launchpad Chromium PPA.  "sudo apt-get -o
Acquire::http::Pipeline-Depth=0 update" doesn't help when updating via Synaptic or
Aptitude.  For that, you need to add the option to apt.conf and that slows down all
other downloads.
Comment 9 by Deleted ...@, Mar 27 2010
Everyone who use this repository has the problem
Comment 10 by yig...@gmail.com, Mar 27 2010
sorry for not being helpful with this comment, but this is a really irritating bug. 
Comment 11 by evan@chromium.org, Mar 30 2010
This is an issue server-side.  Googlers can look at
  http://b/issue?id=2486517
As of last Friday they hoped to get to it this week.
Labels: Mstone-5-b3
 Issue 39871  has been merged into this issue.
Comment 14 by evan@chromium.org, Apr 6 2010
Assigning myself as owner just so it's clear we're tracking this bug.  (I don't 
understand it completely but it looks like the fix is high priority and coming soon.)
Labels: -ReleaseBlock-Beta
Removing release block from all these assuming that things that were marked 
5-b3 were meant for targeting and not blocking.
Comment 16 by omat...@gmail.com, Apr 8 2010
For the average user, the visible effect of this bug is it breaks security updates, 
not only of chrome, but all updates to all other packages on the system.

It should therefore be a high priority, otherwise we could see news headlines like 
"Google Chrome makes Linux insecure", which can't be good.
finally figured out this repo was the culprit. removed & fixed. hope i can enable it
again in the near future.
a possible workaround for this annoying problem was posted here:
http://groups.google.com/group/google-linux-repositories-help-basics/msg/3acbd5a65823301c

You can enable this setting globally by putting it in a file somewhere
in /etc/apt/apt.conf.d/. E.g. create a file called /etc/apt/apt.conf.d/
90localsettings with the following contents:

Acquire::http::Pipeline-Depth "0"; 
Workarounds and hacks?!

Seriously, is it not just AMAZING that Google, aka "The Internet", uses more than 2 
minutes to fix this glaring, superannoying bug?!
Comment 20 by evan@chromium.org, Apr 8 2010
It's almost as if Google is actually just a collection of human beings, fallible and 
stressed by many competing concerns just like all other humans.  ;)
Comment 21 by evan@chromium.org, Apr 8 2010
(As of today the fix is implemented and undergoing testing; probably another week 
before it's visible everywhere.)
good to know. thanks for the hard work guys!
I don't usually leave pointless comments on random bugs I see posted on social bookmarking sites, but 
as an ex-googler when I saw this and watched the Wireshark capture of it happening, a little bit of me 
died. How the hell did your team make it into production?
I don't usually leave pointless comments on random bugs I see posted on social bookmarking sites, but 
as an ex-googler when I saw this and watched the Wireshark capture of it happening, a little bit of me 
died. How the hell did your team make it into production?
On Ubuntu 10.04 with the latest version of Chrome I experience this problem as well.
I'm surprised that this server misconfiguration was just caught in March. I'm running 
Ubuntu Karmic and I've had this problem for several months, presumably as long as I've 
been running the Chrome beta. I'm so glad that a fix is finally being released, as 
it's had a dripping faucet effect on me for a while now. Thank you alkisg for posting 
this. I should have done it myself, but I just assumed that one of the other drones 
had already notified the Borg Queen. :) 
Comment 27 by Deleted ...@, Apr 12 2010
have this problem too, and my friends too... FIX IT!! 
Please add a note with known issues and how to fix them on
http://www.google.com/linuxrepositories/apt.html
Comment 29 by omat...@gmail.com, Apr 18 2010
This is still an issue with the latest version as of today.
Comment 30 by mshu...@gmail.com, Apr 18 2010
@omattos, of course it is.. that is because this issue has *nothing* to do with the 
software..  this is an issue with the web service running the apt repository.  Evan 
has already said they are working on a release for the web service.  Be patient.

To future posters:  please read *all* the comments and educate yourself - if you do 
not understand the problem, nor the simple workaround, nor the fact that this has 
nothing to do with the browser software, then please, do not post yet another "me too 
and I'm running the latest version of Chrome.."

Thanks, have a nice day.
Comment 31 by stols...@gmail.com, Apr 19 2010
Amazing comment. Cool.

And Evan is a really fast guy. When he takes ownership of a bug, you can bet that 
it'll be fixed in a matter of short time. Just like with the PDF bug which affects 
thousands of people:

http://code.google.com/p/chromium/issues/detail?id=19587

(Notice that the bug is closed for comments. That is actually A FIRST: I've never 
seen that on ANY project on Google Code. Evan, The Man - a progressive)

And as with this bug, where everyone knows what is wrong, and still nothing happens, 
a help page offers different hacks and workarounds, waiting for the super-speed of 
Evan:

http://www.google.com/support/forum/p/Chrome/thread?
fid=543f0c80bace6c4200048352a4bbc477&hl=en

"Be patient"?! On what grounds?!
Comment 32 by evan@chromium.org, Apr 22 2010
Status: Fixed
The fix has now propagated to most servers.  Sorry for the delay.
This was a longstanding and annoying problem. Thank you for fixing it. 
Comment 34 by alk...@gmail.com, Apr 23 2010
Thank you.
everything started working OK since yesterday.
Comment 36 by evan@chromium.org, Apr 23 2010
For what it's worth, the fix was in a Google server unrelated to the Chrome project -- 
apparently all of them didn't work with HTTP pipelining, it's just that no browsers 
implement pipelining since so many servers break (like ours!).  The fix was 
implemented quickly, but it took a while for it to be tested and rolled out to all of 
our clusters.
Comment 37 by euph...@gmail.com, Apr 23 2010
This is good stuff, thank you very much!
very nice to hear that it was addressed & tested properly. thank you very much for
your attention to this matter!
Comment 39 by Deleted ...@, May 5 2010
This problem went away, for me, on a Kubuntu9.10 64 bit machine but not on a 
Kubuntu9.10 32 bit machine or several Kubuntu 8.04 (32 bit) machines. upgraded one 32 
bit to Kubuntu 10.4 and did a fresh install of 10.4 on a 64 bit and a 32 bit machine - 
all are now showing this problem again (or still showing it as the case may be).
I think the problem is fixed for me but I rarely am using Chrome because of other
problems I've been having and I just don't have the time right now to be reporting
bugs. But I think the speed issue is fixed.
I should also note I am using:
Ubuntu 10.04 64-bit
Google Chrome 5.0.375.29 beta
Comment 42 by Deleted ...@, May 18 2010
Unfortunately I'm still getting the same issue on Ubuntu 8.04

Failed to fetch http://dl.google.com/linux/deb/dists/stable/Release  Unable to find 
expected entry  non-free/binary-lpia/Packages in Meta-index file (malformed Release 
file?)
Some index files failed to download, they have been ignored, or old ones used instead.
Justin, that's not what this issue is about.

This issue is about poor response times from Google's servers.

Your issue seems to be that something can't be found.

Please open a separate issue.
Comment 44 by bishi...@gmail.com, Dec 14 2010
This is starting to happen again. Huge delays when running apt-get update due to dl.google.com being slow to answer.
I've been updating via the GUI several times today. I haven't noticed this issue yet. Could it have been a temporary issue? 
Comment 46 Deleted
I confirm. seems that it was fixed about half a year ago.
Comment 48 by bishi...@gmail.com, Dec 14 2010
Well, it was hapening this morning (at least from Madrid, Spain), but now seems to work. I attach a tcpdump capture grabbed during the issue this morning.

You can see 12 seconds delay for example here:

1) My request
09:49:40.916781 IP (tos 0x0, ttl 64, id 31410, offset 0, flags [DF], proto TCP (6), length 269)
    <MY_IP>.50478 > mad01s03-in-f93.1e100.net.www: Flags [P.], cksum 0x3221 (correct), seq 2194:2411, ack 7424, win 251, options [nop,nop,TS val 206261 ecr 1582927276], length 217

2) Empty ACK
09:49:40.998947 IP (tos 0x0, ttl 56, id 2726, offset 0, flags [none], proto TCP (6), length 52)
    mad01s03-in-f93.1e100.net.www > <MY_IP>.50478: Flags [.], cksum 0xc932 (correct), seq 7424, ack 2411, win 301, options [nop,nop,TS val 1582927360 ecr 206261], length 0

3) 12 seconds later!!!, the response:
09:49:52.257211 IP (tos 0x0, ttl 56, id 59809, offset 0, flags [none], proto TCP (6), length 1408)
    mad01s03-in-f93.1e100.net.www > <MY_IP>.50478: Flags [.], cksum 0x3b6b (correct), seq 7424:8780, ack 2411, win 301, options [nop,nop,TS val 1582938618 ecr 206261], length 1356

Ping time at the very same time was very low: ~42 ms
ping dl.google.com
PING dl.l.google.com (72.14.235.93) 56(84) bytes of data.
64 bytes from mad01s03-in-f93.1e100.net (72.14.235.93): icmp_req=1 ttl=56 time=42.2 ms
64 bytes from mad01s03-in-f93.1e100.net (72.14.235.93): icmp_req=2 ttl=56 time=42.6 ms
64 bytes from mad01s03-in-f93.1e100.net (72.14.235.93): icmp_req=3 ttl=56 time=42.5 ms
64 bytes from mad01s03-in-f93.1e100.net (72.14.235.93): icmp_req=4 ttl=56 time=42.4 ms

Leave the ticket closed. If happens again I will try to capture full data.
log.txt
18.9 KB View Download
The original issue has been identified and fixed. If anyone has a new issue
with similar symptoms, please open a new bug report.

Thanks!
Leo

206261

proto
Still recieving "W: Failed to fetch http://dl.google.com/linux/chrome/deb/dists/stable/Release  Unable to find expected entry  main/source/Sources in Meta-index file (malformed Release file?)"
And this is before, AND after all suggested workarounds.

So, please reopen, as the problem still exists.
Comment 51 by evan@chromium.org, Dec 20 2010
Labels: Restrict-AddIssueComment-Commit
To repeat comment 49, please open new bugs if you are encountering new problems.
Project Member Comment 52 by bugdroid1@chromium.org, Mar 10 2013
Labels: -Area-Internals Cr-Internals
Sign in to add a comment