New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 107 users
Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Jan 2012
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Feature

Blocked on:
issue 13289
issue 30357

Blocking:
issue 110790

Restricted
  • Only users with Commit permission may comment.



Sign in to add a comment
Optional HTTP pipelining mode
Project Member Reported by wtc@chromium.org, Mar 19 2009 Back to list
We should implement an optional HTTP pipelining mode.
Pipelining is specified in the HTTP 1.1 RFC:
http://www.ietf.org/rfc/rfc2616.txt
 
Comment 1 by shik...@schmizz.net, Mar 22 2009
Do you think this can be tackled as a GSOC project?
Comment 2 by jon@chromium.org, Mar 27 2009
Labels: -Type-Bug Mstone-X
Status: Available
Comment 3 by Deleted ...@, Apr 2 2009
I have created three components for my own server that would be useful for the 
HTTP/1.1 pipelining effort.

* Fast input recognition system: http://www.myersdaily.org/joseph/unix/listswitch.html
http://soar.wichita.edu:8080/dspace/handle/10057/848

* Fast HTTP header processing:
http://www.myersdaily.org/joseph/unix/server/
http://www.myersdaily.org/joseph/unix/server/readheader/index.html

* Fast HTTP request parser:
Based on the experiences gained from writing the easy.pm pragma and Web::* Perl 
modules, used world wide with no bugs found since July, 2000.
Canonicalizes all operating systems by recognizing line ends with LF and ignoring CR. 
"The best decision Unix made was to use a single newline character." (Perl Cookbook)
* Minimizes data copy operations by using independent structures for meta-data and IO 
streams.

I would be interested in working on HTTP/1.1 support provided that I have graduated 
with my Ph.D. in applied math by then. If I still have research left I won't be able 
to do that. I plan to keep you posted.

Status: Assigned
Blockedon: 13289
Status: Started
Rough layout of the work ahead:

* Need to complete work on  bug 13289 .
* Switch HttpNetworkTransaction code to write to HttpStreams rather than sockets.  
Start with just wrapping the sockets with a simple HttpStream (perhaps call it 
HttpBasicStream).
* Create HttpStream factories to create HttpStreams.  Create the basic http streams 
via the factory.  The RequestStream() function will likely take a ClientSocketPool as 
an argument.
* Move the HttpNetworkTransaction<->ClientSocketPool interaction out of 
HttpNetworkTransaction and into the HttpStream factories.
* Implement HttpPipelinedStream and HttpPipelinedStreamFactory.  Likely I'll need to 
introduce a HttpPipelinedStreamManager to coordinate the various HttpPipelinedStreams 
sharing the same socket connection.
Comment 6 by bugdro...@gmail.com, Jun 11 2009
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=18199 

------------------------------------------------------------------------
r18199 | willchan@chromium.org | 2009-06-11 14:21:36 -0700 (Thu, 11 Jun 2009) | 8 lines
Changed paths:
   A http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_basic_stream.h
   M http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_network_transaction.cc?r1=18199&r2=18198
   M http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_network_transaction.h?r1=18199&r2=18198
   A http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_stream.h

Introduce HttpStream and HttpBasicStream.
This is the beginning of the http pipelining work.  Introduce HttpStream, an interface for reading and writing to http streams.  Provide a basic implementation with HttpBasicStream.  Switch HttpNetworkTransaction to reading/writing via HttpStream rather than directly to the socket.
Note that the interface will have to change later on.  Read/Write() is the wrong interface, since a pipelining HttpStream implementation will have to detect the end of an http response, rather than having the client (HttpNetworkTransaction) do that.  This is just the first step.
For information of the general roadmap for http pipelining, please refer to the bug.
BUG= http://crbug.com/8991 
TEST=none

Review URL: http://codereview.chromium.org/119346
------------------------------------------------------------------------

Steve's helping me with this and will take over a large amount of the work while I'm 
on my 6 week vacation.
Comment 9 by oritm@chromium.org, Dec 17 2009
Labels: -Area-BrowserBackend Area-Internals
Replacing labels:
   Area-BrowserBackend by Area-Internals

Comment 10 by arv...@gmail.com, Feb 11 2010
Is this still being actively worked on? Does Chromium support HTTP/1.1 pipelining? If 
so, is it via an option or default?
Comment 11 by arv...@gmail.com, Feb 11 2010
Forgot to mention one thing: I guess buggy proxies make this feature lower priority, but 
what about HTTPS connections? Those would be easier to handle.
It's being worked on, but has a long way to go.  We haven't really discussed yet about 
making it an option or default.
Blockedon: 30357
Comment 14 by ekera...@gmail.com, May 29 2010
Opera has pipelining enabled by default (you can disable it through "opera:config").
Firefox has pipelining disabled, but you can enable it through "about:config" etc.

When I compare Opera/Firefox with pipelining and Chrome, I find that they are much 
faster than Chrome at loading web pages, *especially* when there are *a lot of* 
objects/images embedded.


Comment 15 by Deleted ...@, Sep 12 2010
Any update on this? It's been several months now.
Most of the basic infrastructural refactoring we need to support pipelining is done now.  No direct work on pipelining has happened yet.  Definitely not in Chrome 7 and probably not in Chrome 8.
Comment 17 by Deleted ...@, Sep 17 2010
sigh... I was wondering why chrome was slower loading pages
If you have example URLs that you know are slower due to lack of client-side pipelining support (please verify the server supports pipelining), I would love to hear them.
When directly work on http pipelining start?

Chrome 10?
Comment 20 by Deleted ...@, Nov 5 2010
Deviantart wiht large pieces per page in gallery is substantially faster when pipelined. At least for me. It's 120 connections just for miniatures, and those connections goes via NAT and a wi-fi gate.
Labels: -NewHTTP Internals-Network
@rafal.molotkiewicz: What browser&version are you using to verify that Deviantart is substantially faster with pipelining enabled?  And please provide the URL that you are testing.
I've been doing testing too with the nightly builds of Mozilla Milestone vs. Chromium Nightlies using http://www.numion.com/Stopwatch/index.html.  With pipelining enabled in Milestone it beats Chromium 9.5 times out of 10 albeit not but much in most cases.  But in a few it's by over a second.  Granted it's really splitting hairs at this point LOL but imagine Chromium WITH pipelining.  It would utterly trounce everything in every category then hehehe.
Please provide the target URL that you're measuring.  I assume you aren't testing how fast www.numion.com loads, but rather use that website to test how fast the browser loads some target URL.
Yes, I used the sample links included with that site such as Google, Yahoo, Altavista, Excite, Go.com, Microsoft, ZDNet, IMDB, Tucows.  Also tested with bestbuy.com and a few others.  Here are my results using Minefield Nightly vs Chromium 9.0.580.0 I'll list site then Minefield time then Chromium time:
Google.com:  0.496, 1.192
Yahoo.com:   1.555, 1,567
Altavista:   0.411, 0.916
BestBuy.com: 1.815, 2.373
Cnn.com:     1.996, 2.530

You can see a pattern.  In most cases is negligible, but in others you can really "feel" the difference. 
Also note that I'm on a Dual Wan setup both 11Mbps/1Mbps apiece.  Not sure if that makes any difference or if pipelining takes advantage of any of that.
Thanks for the websites!  FWIW, I know for a fact that google.com doesn't truly support pipelining.  It doesn't error out on it, but it doesn't fully take advantage of it.  How are you sure that pipelining is the reason Minefield Nightly is loading these websites faster?  Can you use wireshark to get packet dumps and send them my way?  Then I can see if pipelining is really the reason in your case.
Hmm, what you said made me think.  So I decided to disable Pipelining in Minefield and clear cache and retest.  It didn't seem to affect anything LOL.  So I guess the Mozilla guys just must be on to something with their upcoming release.  I was hoping to pin it on the pipelining but it would appear that Minefield is just faster at loading.


Comment 28 by jay...@gmail.com, Nov 11 2010
There have been Firefox checkins in this area, improving non-pipelining and pipelining, and more to come with patches on many of the blockers of bugzilla tracking  bug 603503 .
is there a good, clean way of testing whether or not a particular web server supports or responds well to pipelining?
Comment 30 by Deleted ...@, Dec 31 2010
OMG it's been like 3 years and chrome does not support pipelining... still !!
And I heard the reason is that buggy proxies is the priority... oh come on so few ppl use proxies and so much would benefit from faster loading pages. And to all the pros: how about you go and test some 'big' pages yourself instead of asking on'an'on? I see the difference without a stopper so should you !
Comment 31 by arv...@gmail.com, Apr 21 2011
What is the current status on this? How about turning it on for https where there are no proxy issues to deal with?
Comment 32 by arvind@google.com, Apr 26 2011
Btw, FF is also talking about putting an HTTP header (or negotiation in TLS handshake) to allow a server to tell the browser that it supports pipelining.

https://bugzilla.mozilla.org/show_bug.cgi?id=652308


I was testing load times in Firebug on Firefox4/Mac on this website: http://www.browserscope.org/network/tests/connections-per-hostname and the results were suprising: with pipelining: 20.17s, without: 10.57s! How come?
Comment 34 by mnott...@gmail.com, Apr 27 2011
FF4 release as well as their current tree doesn't have the pipelining code; you have to download a separate build.

You're *probably* seeing the connection pre-warming that FF4 does, at a guess.
Comment 35 by tonyg@chromium.org, May 15 2011
Cc: tonyg@chromium.org simonjam@chromium.org
Comment 36 by Deleted ...@, Sep 12 2011
Anyone can shed a light on whether pipelining will cause any problem with responses containing cookies? Suppose the browser sends two requests at the same time (without waiting for the response), the first response comes with a cookie, the second request (thus already sent by browser) obviously would not contain that cookie. What should the server do? Close the connection after the first request?
Comment 37 by mnott...@gmail.com, Sep 12 2011
It'd do the same thing as if it had made the two requests on two connections.
Project Member Comment 38 by bugdroid1@chromium.org, Oct 19 2011
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=106364

------------------------------------------------------------------------
r106364 | simonjam@chromium.org | Wed Oct 19 13:14:29 PDT 2011

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/chrome_browser_main.cc?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/about_flags.cc?r1=106364&r2=106363&pathrev=106364
 A http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_pipelined_connection_impl_unittest.cc?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_stream_parser.cc?r1=106364&r2=106363&pathrev=106364
 A http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_pipelined_connection_impl.h?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_stream_factory_impl_request.h?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/base/net_error_list.h?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_stream_factory.cc?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/spdy/spdy_http_stream.cc?r1=106364&r2=106363&pathrev=106364
 A http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_pipelined_stream.h?r1=106364&r2=106363&pathrev=106364
 A http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_pipelined_host.cc?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/app/generated_resources.grd?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_network_transaction.cc?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_stream_factory_impl_request.cc?r1=106364&r2=106363&pathrev=106364
 A http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_pipelined_connection_impl.cc?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_basic_stream.h?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/base/net_log_event_type_list.h?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/net.gyp?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_basic_stream.cc?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_response_body_drainer_unittest.cc?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/common/chrome_switches.cc?r1=106364&r2=106363&pathrev=106364
 A http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_pipelined_connection.h?r1=106364&r2=106363&pathrev=106364
 A http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_pipelined_host_pool.cc?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_network_transaction_unittest.cc?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_stream.h?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_stream_factory_impl.cc?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_stream_factory_impl_job.cc?r1=106364&r2=106363&pathrev=106364
 A http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_pipelined_host_unittest.cc?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_stream_factory_impl_job.h?r1=106364&r2=106363&pathrev=106364
 A http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_pipelined_host.h?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_stream_factory_impl.h?r1=106364&r2=106363&pathrev=106364
 A http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_pipelined_host_pool.h?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/spdy/spdy_http_stream.h?r1=106364&r2=106363&pathrev=106364
 A http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_pipelined_stream.cc?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_stream_factory.h?r1=106364&r2=106363&pathrev=106364
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/common/chrome_switches.h?r1=106364&r2=106363&pathrev=106364

Basic HTTP pipelining support.

This must be enabled in about:flags. It is naive and assumes all servers correctly implement pipelining. Proxies are not supported.

Immediate future work:
- Integration tests.
- Additional NetLog logging.
- Refactor HttpPipelinedConnectionImpl.

Long term:
- Detect broken transparent proxies.
- Detect and/or mitigate broken servers.
- Buffer HttpPipelinedStream.
- Optimize number of pipelines and their depth.
- Support proxies.

BUG= 8991 
TEST=net_unittests


Review URL: http://codereview.chromium.org/7289006
------------------------------------------------------------------------
Cc: -simonjam@chromium.org vandebo@chromium.org
Labels: -Mstone-X -Internals-Network Mstone-17 Internals-Network-HTTP
Owner: simonjam@chromium.org
Comment 40 by k...@google.com, Dec 19 2011
Labels: -Mstone-17 Mstone-18 MovedFrom-17
Moving bugs marked as Started but not blockers from M17 to M18.  Please move back if you think this is a blocker, and add the ReleaseBlock-Stable label.  If you're able.
Labels: -Mstone-18 -MovedFrom-17
What's the criteria for this being "done?" Technically, we already have an "optional HTTP pipelining mode." You just have to flip the flag.

Instead, should this bug become a metabug for the feature? There's still a bunch of work left before it's on by default for everyone.
My suggestion would be to mark this as fixed and file a new meta-bug, maybe with some general todo's in the first post, CCing everyone in this bug and linking to the new one in the update where you mark it fixed.

That'll give us a cleaner bug to work from, and have a task list in a nice, prominent place.  If you prefer to continue to use this one, however, I'm fine with that, too.
Blocking: 110790
Status: Fixed
Sounds good to me.

For those that want to follow along, see this meta bug:

http://code.google.com/p/chromium/issues/detail?id=110790
Project Member Comment 45 by bugdroid1@chromium.org, Oct 13 2012
Blockedon: -chromium:13289 -chromium:30357 chromium:13289 chromium:30357
Blocking: -chromium:110790 chromium:110790
Labels: Restrict-AddIssueComment-Commit
This issue has been closed for some time. No one will pay attention to new comments.
If you are seeing this bug or have new data, please click New Issue to start a new bug.
Project Member Comment 46 by bugdroid1@chromium.org, Mar 10 2013
Labels: -Internals-Network-HTTP -Area-Internals Cr-Internals-Network-HTTP Cr-Internals
Sign in to add a comment