New issue
Advanced search Search tips

Issue 806281 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Feb 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 2
Type: Bug



Sign in to add a comment

setRequestInterception causes HTTP/2 pseudo-headers to be removed from requests

Reported by guean...@amazon.com, Jan 26 2018

Issue description

UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36

Steps to reproduce the problem:
1. Create a request interceptor via the Network::setRequestInterception API
2. Allow all requests to continue unmodified

The easiest way (that I know of) to attach a request interceptor is to pass the --deterministic-fetch flag

What is the expected behavior?
Pages will load as expected and requests will include all original headers, unmodified.

What went wrong?
Servers which expect requests to be HTTP/2 will return errors because at some point the HTTP/2 pseudo-headers have all been removed

Did this work before? N/A 

Chrome version:   Channel: dev
OS Version: Ubuntu 16.04.3 LTS
Flash Version: 

These headers are getting removed even before the request makes it to the interceptor. If I log out the request headers, it does not include any HTTP/2 pseudo headers. If I do not attach an interceptor, requests go through as expected. I've attached two har files which show similar requests, one passing through an interceptor, the other not. You can see that the headers look completely different and in the interceptor case the request is entirely rejected by the server.
 
successful_login.har
162 KB Download
failed_login.har
114 KB Download

Comment 1 by guean...@amazon.com, Jan 26 2018

Chrome version: 65.0.3322.0
Cc: caseq@chromium.org dgozman@chromium.org
FYI.  I wonder if this is something to do with URLRequestJob::SetRequestHeadersCallback and URLRequestJob::SetResponseHeadersCallback.  We don't proxy those but maybe we should.
Cc: -caseq@chromium.org
Owner: caseq@chromium.org
Status: Assigned (was: Unconfirmed)
I think you are right - that problem is on caseq's radar.

Comment 4 by guean...@amazon.com, Jan 26 2018

I'm also more than happy to submit a fix for this if someone is at least willing to provide some context and point me in the right direction.

Comment 5 by guean...@amazon.com, Jan 31 2018

Ping on this. This issue is causing a lot of problems for us and as I said, if someone can even nudge me in the right direction I'm more than happy to contribute a fix.
Here [1] is an interceptor job class, which does most of the work. The problem is probably there. Sorry, didn't look close enough to be sure.

[1] https://cs.chromium.org/chromium/src/content/browser/devtools/devtools_url_interceptor_request_job.h
Alright, so taking a look into this, I have a suspicion of what is going on.

I've tried logging out the headers of the original request to see when they might be getting removed. At no point in the process does the original URLRequest ever have its headers attached. I suspect this means they get attached later in the process and the fact that the interceptor stops that request from moving forward means they never get attached.

Data from the original URLRequest is then attached to the SubRequest which is sent and its own response data is funneled back into the original request. This SubRequest never has HTTP/2 headers at any point. Because of this, I suspect one of two things is happening:

1. There is some data from the initial URLRequest which isn't getting copied to the SubRequest which would cause it to be sent as HTTP/2
2. The SubRequest is somehow sent differently from the original request causing it to always be sent as HTTP/1.1

I don't have a smoking gun on either case and I'm now trying to understand how the net code decides which HTTP version to use on the request.

Comment 8 by caseq@chromium.org, Feb 5 2018

gueandre, we appreciate your willing to contribute a fix, but please note we're currently working on re-implementing support for the interception on a different level (it's going to be a proxy for URLLoader/URLLoaderFactory interfaces, so it will be involved on a higher level of network service rather than under net stack). This will most likely take care of numerous bugs like this -- so not sure if you want to invest into effort of fixing this now.
@alexclarke: You were right - Exposing the RequestHeadersCallback & ResponseHeadersCallback in the URLRequest[1] and then setting them in the DevtoolsUrlInterceptorRequestJob::SubRequest[2] fixes the issue. 

@caseq: I have this fix working locally. Would this be something you guys would be interested in accepting as a fix? 

[1]: https://cs.chromium.org/chromium/src/net/url_request/url_request.h?type=cs&l=687
[2]: https://cs.chromium.org/chromium/src/content/browser/devtools/devtools_url_interceptor_request_job.cc?l=112
Is there any timeline for the reimplementation caseq? I've been troubled by a similar bug and am also looking into patching it locally until the rework is released.
Project Member

Comment 11 by bugdroid1@chromium.org, Feb 21 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d15f52c9d708f3a0f127a8bc770cfc22f12f63df

commit d15f52c9d708f3a0f127a8bc770cfc22f12f63df
Author: Yash Vempati <vempatiy@amazon.com>
Date: Wed Feb 21 01:05:07 2018

Plumb raw headers when intercepting a request

This CL ensures that |RequestHeadersCallback| and |ResponseHeadersCallback|
are proxied from the |URLRequest| to the |DevToolsURLInterceptorJob::SubRequest|
when intercepting a request.

This allows the SubRequest to receive SPDY/QUIC internal headers, as well as plain HTTP raw headers.

BUG= 806281 
R=caseq@chromium.org

Change-Id: Id152c63426468ab7637d2885dad7b91f124b54aa
Reviewed-on: https://chromium-review.googlesource.com/916901
Commit-Queue: Andrey Kosyakov <caseq@chromium.org>
Reviewed-by: Andrey Kosyakov <caseq@chromium.org>
Cr-Commit-Position: refs/heads/master@{#537971}
[modify] https://crrev.com/d15f52c9d708f3a0f127a8bc770cfc22f12f63df/AUTHORS
[modify] https://crrev.com/d15f52c9d708f3a0f127a8bc770cfc22f12f63df/content/browser/devtools/devtools_url_interceptor_request_job.cc
[modify] https://crrev.com/d15f52c9d708f3a0f127a8bc770cfc22f12f63df/content/browser/devtools/devtools_url_interceptor_request_job.h
[modify] https://crrev.com/d15f52c9d708f3a0f127a8bc770cfc22f12f63df/third_party/WebKit/LayoutTests/http/tests/inspector-protocol/network/redirect-interception-mocked-expected.txt
[add] https://crrev.com/d15f52c9d708f3a0f127a8bc770cfc22f12f63df/third_party/WebKit/LayoutTests/http/tests/inspector-protocol/network/request-interception-raw-headers-expected.txt
[add] https://crrev.com/d15f52c9d708f3a0f127a8bc770cfc22f12f63df/third_party/WebKit/LayoutTests/http/tests/inspector-protocol/network/request-interception-raw-headers.js

Comment 12 by caseq@chromium.org, Feb 21 2018

Components: -Platform>DevTools Platform>DevTools>Network
Status: Fixed (was: Assigned)
Fixed by vempatiy@amazon.com, see commit referenced above.

Sign in to add a comment