Network.getResponseBody fails sporadically
Reported by
i...@netreturn.co.za,
Jan 25 2018
|
||||
Issue descriptionUserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36 Steps to reproduce the problem: 1. git clone https://github.com/ilanc/devtools-bugs 2. cd devtools-bugs 3. npm install 4. node ./getResponseBody.js 5. look for output like this: getResponseBody exception 4612.1449 https://files.coinmarketcap.com/generated/search/quick_search.json No data found for resource with given identifier What is the expected behavior? Either `Network.getResponseBody` should just work or if it is a cache related issue then I would like to be able to specify which requests should be preserved e.g. something like `Network.preserveResponseBody(requestId)` (which could be called from `Network.requestWillBeSent`) What went wrong? * `getResponseBody` fails sporadically - you can't predict which request will fail ahead of time. * output = No data found for resource with given identifier Did this work before? No Chrome version: 63.0.3239.132 Channel: stable OS Version: 10.0 Flash Version: Shockwave Flash 28.0 r0 There are some old threads about this: https://groups.google.com/forum/#!topic/google-chrome-developer-tools/8KWv_gNBfYY But it looks like they were never resolved
,
Jan 25 2018
Seems more reliable but still sometimes fails, new output (nb running the test 10 times in a loop): $ node getResponseBody.js getResponseBody exception 14788.4841 https://www.google.com/csi?v=3&s=gpt&action=global&srt=36&tbsrt=744&tran=16&p=s&npn=1&npnv=h2&count_of_slots=3&count_of_requested_slots=3&count_of_rendered_slots=3&count_of_requests=1&count_of_refreshes_called=1&count_of_passback=0&vrg=175&pl_id=7920774384702404&e=v175,n48901027,publisher_ads,sra,fif,page_load,page_unload,21061446,21060361,21060554,21061071,21061149,21061300&rt=page_unload_time.10257 No resource with given identifier found
,
Jan 25 2018
Hander pola?
,
Jan 25 2018
Kanker pola,
,
Jan 25 2018
,
Jan 26 2018
I've been playing with this and want to add some more thoughts - alas I haven't established a reliable repeat case yet. 1. I'm actually using the code on other websites. I can't share this code because I'm working with website which require a login. 2. There are a number of requests (14 in total) across a number of sites (6 in total) where I'm attempting to use getResponseBody. Of these requests some are are more flakey than others - i.e. getResponseBody fails about 50% of the time on the flakiest of the requests. 3. I log out the headers on success and fail for all getResponseBody requests. The headers don't seem to change, i.e. I haven't isolated a header setting which causes getResponseBody to fail 100% of the time. 4. The most flakey requests have these headers: Pragma: 'no-cache', 'Cache-Control': 'no-cache', 5. But some of the reliable ones also have have no-cache So headers may be a red-herring - maybe it's down to the volume of requests occurring at the same time? like I originally thought but that's why I used coinmarketcap.com in the first place and it seems more reliable. Finally I did also run fiddler to capture http traffic during my tests. The flakey requests are being captured by fiddler - i.e. the server is definitely sending a body, it's just that getResponseBody is failing to retrieve it.
,
Jan 31 2018
Have you had any luck confirming this? There is one other possible problem - I've just noticed that I am getting duplicate `Network.responseReceived` & `Network.loadingFinished` events for a single requestId. I've put in code to ensure that I only call getResponseBody once per requestId. I'll update here if this cures the problem.
,
Feb 6 2018
Hi @eostroukhov have you had a look at this?
,
Mar 21 2018
I found that invalid characters in the original file (e.g. HTML, JS, etc) stop getResponseBody returning at all. It's not sporadic but is another case where getResponseBody fails to return either the content or an error. You can find out more in issue 824174
,
Mar 22 2018
Thanks for the feedback. What would constitute an invalid character? The responses I'm trying to capture are ajax request with json payloads.
,
Mar 22 2018
An invalid character would be a sequence of bytes that don't conform to UTF8 encoding. This page has some examples: http://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-test.txt
,
Oct 12
,
Dec 29
ilan@netreturn.co.za I ran your node script but it didn't say anything about getResponseBody. Are you still having any issues? Can you provide better steps to reproduce? Side note - I would recommend using puppeteer for browser automation stuff like this since it wraps the devtools protocol and our team maintains it: https://pptr.dev
,
Jan 5
Hi jarhar@chromium.org. It looks like the problem went away. I've searched through 1 year of logs (from the prod code where I use this functionality) and not found any getResponseBody fails since 2018-01-29. Perhaps the `Network.loadingFinished` fix above worked? It's a long time ago now so I can't be sure. Thanks I'm aware of puppeteer but alas it doesn't fit my requirements.
,
Jan 5
|
||||
►
Sign in to add a comment |
||||
Comment 1 by i...@netreturn.co.za
, Jan 25 2018