New issue
Advanced search Search tips
Starred by 264 users

Issue metadata

Status: Available
Owner: ----
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Feature

Blocking:
issue 104058


Show other hotlists

Hotlists containing this issue:
Hotlist-1


Sign in to add a comment
link

Issue 487422: WebRequest API: allow extensions to read response body

Reported by jetrico...@gmail.com, May 12 2015

Issue description

UserAgent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36

Steps to reproduce the problem:
A related issue entry, bug 104058 aims to edit response bodies, this entry aims to only "read". You can set this as a blocking to the related issue mentioned.

What is the expected behavior?
To have the ability to receive and read response bodies via WebRequest API, probably a "onResponseBodyReceived" listener.

What went wrong?
As a small step forward, "Reading" response bodies was approved and determined possible in the related bug so it needs its own issue entry for easier progress tracking.

WebStore page: 

Did this work before? No 

Chrome version: 42.0.2311.135  Channel: stable
OS Version: 6.1 (Windows 7, Windows Server 2008 R2)
Flash Version: Shockwave Flash 17.0 r0
 

Comment 1 by kavvaru@chromium.org, May 13 2015

Labels: TE-NeedsFurtherTriage

Comment 2 by rob@robwu.nl, May 31 2015

Blocking: chromium:104058

Comment 3 by rob@robwu.nl, May 31 2015

Labels: -Type-Bug -OS-Windows -TE-NeedsFurtherTriage Type-Feature OS-All
Owner: rob@robwu.nl
Status: Assigned
Not actively working on it, but it's on my wishlist.

The extension team has already expressed approval towards this feature, if a patch were to be submitted: https://groups.google.com/a/chromium.org/d/msg/apps-dev/v176iCmRgSs/iM-72Evf8JgJ.

Comment 4 by himanshu...@gmail.com, Jun 29 2015

I am writing an extension for chrome which involve monitoring on requests. This feature would really help for my extension. Please share updates about its availability.

Comment 5 by allankim...@gmail.com, Jul 20 2015

I had started work on a extention that would use chrome.webRequest to monitor some URLS body, this would be a very nice feature.

Now I have to redo the whole thing in chrome.debugger I guess. Should have read the documentation first :-)

Comment 6 by and...@blazemeter.com, Jul 20 2015

Same for me, desperately waiting for this to be implemented.

Comment 7 by jagdish....@gmail.com, Jul 25 2015

Is there any update on this feature? Anyone in the extension development team can, please, tell the timeline? Is this already work-in-progress? Would it be available in the near future?

Thanks a lot in advance for the answer.

Comment 8 by rob@robwu.nl, Jul 25 2015

There is no progress yet, don't expect this feature to become available within a few months.

When I start working on this feature, the status (see left side) will change from "Assigned" to "Started".

Comment 9 by sunochi....@gmail.com, Oct 13 2015

As I use an extension that could gain more features if this feature was implemented, I'd like this to go through.

Comment 10 by jagdish....@gmail.com, Oct 14 2015

The same is true for me too. I desperately want it to be implemented.

Comment 11 by jetrico...@gmail.com, Nov 23 2015

is there no easy way just to include the completed data stream into chrome.webRequest.onCompleted event? seems the whole response is ready at that point, just include it in the callback?

https://developer.chrome.com/extensions/webRequest#event-onCompleted

Comment 12 by jagdish....@gmail.com, Nov 23 2015

I think that it would be very useful to have the response body.

Comment 13 by mmi...@gmail.com, Dec 8 2015

I think that read-only would really be useful, but also a way to cancel/abort the request at that time. So not fully modify the response, but canceling/aborting would be great. That would allow us to implement some security and integrity checks (like checking for viruses and malicious code) in an extension, which probably should not be part of the core Chrome features, but it would be a great way to improve security of users on the web.

Comment 14 by jagdish....@gmail.com, Jan 21 2016

This feature is urgently needed for my extension MyTrackingChoices https://chrome.google.com/webstore/detail/mytrackingchoices/fmonkjimgifgcgeocdhhgbfoncmjclka?hl=fr
This is to categorize the content of the Web pages before they are rendered by the browser. 
It would be great if we can have this feature.

Comment 15 by jagdish....@gmail.com, Apr 14 2016

Any news on read only access, please?

Comment 16 by windma...@gmail.com, May 9 2016

'read only' is absolutely needed. Please implement!

Comment 17 by salman.s...@gmail.com, Jul 21 2016

Read and write both are important, in the right context they both have immense value, building an extension like Charles proxy right inside chrome gives huge debugging power to developer to modify response body and simulate different test cases.

Comment 18 by jerzyglo...@gmail.com, Apr 28 2017

I strongly support this functionality, which should not be really hard to implement:
- add RESPONSE_BODY to struct ExtraInfoSpec (web_request_api_helpers.h)
- add property response_body_ (web_request_event_details.h)
- add method WebRequestEventDetails::SetResponseBody() (web_request_event_details.cc)
- add response_body_ to method WebRequestEventDetails::GetFilteredDict() (web_request_event_details.cc)
- add event_details->SetResponseBody(request) to method ExtensionWebRequestEventRouter::OnCompleted() (web_request_api.cc)

Anybody willing to help?

Comment 19 by jetrico...@gmail.com, Apr 28 2017

for anyone wanting an alternative which is doable at the moment, you can use chrome.debugger in a background/event page to attach to the specific tab you want to listen to (or attach to all tabs if that's possible, haven't tested all tabs personally), then use the network API of the debugging protocol.

the only problem with this is that there will be the usual yellow bar at the top of the tab's viewport, unless user turns it off at chrome://flags

Comment 20 by alla...@gmail.com, Nov 17 2017

It's perhaps worth noting that Firefox WebExtensions have this ability as of Firefox 57.0 via webRequest.filterResponseData()

Documentation: https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/webRequest/filterResponseData

Equivalent bug in Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=1255894

Comment 21 by michael....@gmail.com, Dec 18 2017

@rob@robwu.nl 

I see this feature is approved but just not started. What would it take to get it going? Do you need support? As noted the FF team recently cranked out this feature and also solved issue 104058. Issue 104058 is blocking the porting of my extension over from FF. Willing to even try to contract help to make these two issues go away, just need guidance.

Thanks,
Michael

Comment 22 by woxxom@gmail.com, Dec 18 2017

@michael, looking at the header of this report I see rob@ wasn't here for over a month and there are no other developers in the CC list so it's unlikely the developers will notice. Consider posting on https://groups.google.com/a/chromium.org/forum/#!forum/chromium-extensions - some of the developers are there.

Comment 23 by rob@robwu.nl, Dec 20 2017

#21
> What would it take to get it going?

Time.

> Do you need support?

Patches are welcome.

I will ask internally whether an implementation of Firefox's webRequest.filterResponseData API would be accepted in Chrome.

Comment 24 by michael....@gmail.com, Dec 20 2017

Thanks @rob@robwu.nl. 

If they would be willing to accept, I can have our team look into submitting a patch. Like I said before, Issue 104058 is the real concern but webRequest.filterResponseData addresses both. 

Michael

Comment 25 by nat...@techsure.ca, Feb 21 2018

Would love to see this move forward as well.  Any updates?

Comment 26 by rob@robwu.nl, Feb 21 2018

Owner: ----
Status: Available (was: Assigned)
Not from me. I didn't get a reply either when I asked internally.

Comment 27 by himanshu...@gmail.com, Feb 21 2018

Is this issue fixed. What does Available means?

Comment 28 by mexmat.s...@gmail.com, Feb 21 2018

@27

"Available" means that there is nobody actively working on this issue - it's available for anyone to start working on it.

Rob essentially just removed himself as the issue owner - the person working on this - because he presumably doesn't have time to do it.

Comment 29 by rob@robwu.nl, Feb 21 2018

For some use cases (those where you want to modify requests in specific websites/tabs/frames), the DevTools protocol can be used to intercept and rewrite requests:
https://chromedevtools.github.io/devtools-protocol/tot/Network#method-setRequestInterception

The DevTools protocol is available via the chrome.debugger extension API,
via the --remote-debugging-port flag, e.g. via libraries such as Puppeteer -https://github.com/GoogleChrome/puppeteer , example: https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pagesetrequestinterceptionvalue

Comment 30 by m...@seanw.org, Feb 21 2018

I'm really interested in seeing this move forward as well. I have a Chrome extension that checks if websites follow best practices (see https://chrome.google.com/webstore/detail/checkbot-seo-web-speed-se/dagohlmlhagincbfilmkadjgmdnkjinl) and my options right now if I want to access responses are:

1) Get the extension to request the debugging permission and ask users to edit their Chrome flags to turn off the scary looking warning. This is going to scare users away.

or 

2) Force users to have the dev tools window open. This is going to create a clunky user experience and I want the interface for the extension to live in a tab.

Comment 31 by sscar...@gmail.com, Feb 22 2018

...and now're back to square zero, I don't think this will ever get implemented. Don't get your hopes up.

Comment 32 by heber.to...@gmail.com, Feb 22 2018

Hi,

Unfortunately, requestBody is also not working well: https://bugs.chromium.org/p/chromium/issues/detail?id=813285

If I ever get the other fixed, I don't mind giving this one a go as well.

In any-case if it's ever done it should remain consistent with FF (as mentioned above):

https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/webRequest/StreamFilter

https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/webRequest/filterResponseData

It will also solve https://bugs.chromium.org/p/chromium/issues/detail?id=104058

Now I just need to get a strong machine to be able to compile chromium /:

Comment 33 by bra...@cosmocode.de, Apr 20 2018

I really hope to see that implemented soon as my chrome extension would be much more robust and does not need to rely on hacks.

For everyone that needs a workaround for intercepting AJAX requests there is another possibility. You can monkey patch the XMLHttpRequest object in the tab you want to intercept requests in. For an explanation of how to implement this see: https://www.moesif.com/blog/technical/apirequest/How-We-Captured-AJAX-Requests-with-a-Chrome-Extension/

This solution does not have the caveat to show this scary warning when using debugging permissions.

Comment 34 by sscar...@gmail.com, Apr 20 2018

@#33

Won't happen until someone decides to Assign themselves to the bug and start working on it, so far there seems no one who's interested, so if I were you, I wouldn't hold my breath on it and start looking for alternatives.

Comment 35 by jessenka...@gmail.com, May 21 2018

The ability to replace content opens up lots of possibilities. I am actively working on a FF extension but have held off making a Chrome version because this API does not exist. So frustrating!

Comment 36 by thomasga...@gmail.com, Jun 23 2018

+1 for reading the response body size - trying to monitor traffic loads with an extension and you can't rely on chrome.webRequest.onCompleted and content-length header as some sites don't bother with it.

Comment 37 Deleted

Comment 38 by wk3...@gmail.com, Sep 6

https://blog.csdn.net/wk3368/article/details/82466303

got into this problem,and figure out the reason is according to gzip and response length(>600K or 700K?)

Comment 39 Deleted

Comment 40 by brian.mc...@gmail.com, Sep 17

CloudFlare's public blog pointed to this issue on September 17, 2018 as a blocker for their IPFS Chrome extension.

https://blog.cloudflare.com/e2e-integrity/

Comment 41 by artem.fe...@blazemeter.com, Oct 11

Hi all
I'm working on patch for this feature. The comment https://bugs.chromium.org/p/chromium/issues/detail?id=487422#c18 is looks like a part of solution, but I have a problem and maybe somebody can help me. 

1) Where I can find response_body buffer? I saw method net::URLRequest::Read(IOBuffer* buf, int max_bytes) but when I tried read from request in onCompleted event this method return 0 - because all body data was read in the other place. 
2) In which event I should read the response body? In onCompleted or onResponseStarted?

Sign in to add a comment