How does `Network.getCookies` in the remote debugger interface actually work?
Reported by
lem...@gmail.com,
Nov 28 2016
|
||||||
Issue descriptionUserAgent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.100 Safari/537.36 Vivaldi/1.5.658.44 Steps to reproduce the problem: 1. Using remote debug interaface, navigate to site A which sets a cookie. 2. Then, in the same tab, navigate to site B 3. Execute `Network.getCookies` remote debugger command. What is the expected behavior? All cookies, including cookies for site A and site B should be returned (the documentation for `Network.getCookies` states: "Returns all browser cookies. Depending on the backend support, will return detailed cookie information in the <code>cookies</code> field." Instead, only cookies for the currently active site are returned, in this case, site B. What went wrong? Either the documentation for what `Network.getCookies` does is wrong, and it only returns cookies scoped to the currently active webpage, or the function call is not returning the correct value. --- Attached is a lot of the web-socket messages send while executing my test-script here: https://github.com/fake-name/ChromeController/blob/master/cookie_wtf.py Briefly: > Spawned process: <subprocess.Popen object at 0x7f53e4581d68> > <ChromeController.transport.ChromeSocketManager object at 0x7f53e3ff44a8> > Synchronous_command: > command: 'Page.enable' > params: '{}' > Sending: '{"id": 0, "method": "Page.enable"}' > Received: '{"id":0,"result":{}}' > Response: '{'id': 0, 'result': {}}' > Synchronous_command: > command: 'DOM.enable' > params: '{}' > Sending: '{"id": 1, "method": "DOM.enable"}' > Received: '{"id":1,"result":{}}' > Response: '{'id': 1, 'result': {}}' > Synchronous_command: > command: 'Network.enable' > params: '{}' > Sending: '{"id": 2, "method": "Network.enable"}' > Received: '{"id":2,"result":{}}' > Response: '{'id': 2, 'result': {}}' > Synchronous_command: > command: 'Emulation.setVisibleSize' > params: '{'height': 1000, 'width': 1500}' > Sending: '{"id": 3, "method": "Emulation.setVisibleSize", "params": {"height": 1000, "width": 1500}}' > Received: '{"id":3,"result":{}}' > Response: '{'id': 3, 'result': {}}' > Synchronous_command: > command: 'Page.navigate' > params: '{'url': 'http://www.whatarecookies.com/cookietest.asp'}' > Sending: '{"id": 4, "method": "Page.navigate", "params": {"url": "http://www.whatarecookies.com/cookietest.asp"}}' > Received: '{"method":"Page.frameResized","params":{}}' > <snip a bunch of network messages> > Received: '{"id":4,"result":{"frameId":"13569.1"}}' > Response: '{'id': 4, 'result': {'frameId': '13569.1'}}' > <snip a bunch of network messages> > Received: '{"method":"Page.frameStoppedLoading","params":{"frameId":"13569.1"}}' > Synchronous_command: > command: 'Network.getCookies' > params: '{}' > Sending: '{"id": 5, "method": "Network.getCookies"}' > Received: '{"id":5,"result":{"cookies":[{"domain":".doubleclick.net","expires":1480309018414.03,"httpOnly":false,"name":"test_cookie","path":"/","secure":false,"session":false,"size":29,"value":"CheckForPermission"},{"domain":".whatarecookies.com","expires":1480308717000.0,"httpOnly":false,"name":"_gat","path":"/","secure":false,"session":false,"size":5,"value":"1"},{"domain":".whatarecookies.com","expires":1543380117000.0,"httpOnly":false,"name":"_ga","path":"/","secure":false,"session":false,"size":29,"value":"GA1.2.497879937.1480308118"},{"domain":"www.whatarecookies.com","expires":0.0,"httpOnly":false,"name":"ASPSESSIONIDASBDRQQD","path":"/","secure":false,"session":true,"size":44,"value":"LBNKGPDBHCOMEHDMCDAOBBHD"},{"domain":"www.whatarecookies.com","expires":1480309317000.0,"httpOnly":false,"name":"dta","path":"/","secure":false,"session":false,"size":36,"value":"vcount%3D0%2Cprev%3D1480308117549"}]}}' > Response: '{'id': 5, 'result': {'cookies': [{'path': '/', 'value': 'CheckForPermission', 'name': 'test_cookie', 'domain': '.doubleclick.net', 'httpOnly': False, 'expires': 1480309018414.03, 'session': False, 'secure': False, 'size': 29}, {'path': '/', 'value': '1', 'name': '_gat', 'domain': '.whatarecookies.com', 'httpOnly': False, 'expires': 1480308717000.0, 'session': False, 'secure': False, 'size': 5}, {'path': '/', 'value': 'GA1.2.497879937.1480308118', 'name': '_ga', 'domain': '.whatarecookies.com', 'httpOnly': False, 'expires': 1543380117000.0, 'session': False, 'secure': False, 'size': 29}, {'path': '/', 'value': 'LBNKGPDBHCOMEHDMCDAOBBHD', 'name': 'ASPSESSIONIDASBDRQQD', 'domain': 'www.whatarecookies.com', 'httpOnly': False, 'expires': 0.0, 'session': True, 'secure': False, 'size': 44}, {'path': '/', 'value': 'vcount%3D0%2Cprev%3D1480308117549', 'name': 'dta', 'domain': 'www.whatarecookies.com', 'httpOnly': False, 'expires': 1480309317000.0, 'session': False, 'secure': False, 'size': 36}]}}' > Synchronous_command: > command: 'Page.navigate' > params: '{'url': 'http://google.com'}' > Sending: '{"id": 6, "method": "Page.navigate", "params": {"url": "http://google.com"}}' > Received: '{"id":6,"result":{"frameId":"13586.1"}}' > Response: '{'id': 6, 'result': {'frameId': '13586.1'}}' > <snip a bunch of network messages> > Received: '{"method":"Page.frameStoppedLoading","params":{"frameId":"13586.1"}}' > Synchronous_command: > command: 'Network.getCookies' > params: '{}' > Sending: '{"id": 7, "method": "Network.getCookies"}' > Received: '{"id":7,"result":{"cookies":[{"domain":".google.com","expires":1496119318679.717,"httpOnly":true,"name":"NID","path":"/","secure":false,"session":false,"size":149,"value":"91=08XuV0_PuzpDlm0ZkMDpig8p0M342GEmu1K9yHVC7xpD-n7e3_6XS4ruT70T717rjB22ByloeRrZb4ROvISN_Ig2opVs3tOOj3uAgkdwb9QesVK0IIiIhW2trR9NCQgxTH6QXq9a8GGzZgg"}]}}' > Response: '{'id': 7, 'result': {'cookies': [{'path': '/', 'value': '91=08XuV0_PuzpDlm0ZkMDpig8p0M342GEmu1K9yHVC7xpD-n7e3_6XS4ruT70T717rjB22ByloeRrZb4ROvISN_Ig2opVs3tOOj3uAgkdwb9QesVK0IIiIhW2trR9NCQgxTH6QXq9a8GGzZgg', 'name': 'NID', 'domain': '.google.com', 'httpOnly': True, 'expires': 1496119318679.717, 'session': False, 'secure': False, 'size': 149}]}}' > > Cookies after first 'Page.navigate' command: > <Cookie test_cookie=CheckForPermission for .doubleclick.net/> > <Cookie _gat=1 for .whatarecookies.com/> > <Cookie _ga=GA1.2.497879937.1480308118 for .whatarecookies.com/> > <Cookie ASPSESSIONIDASBDRQQD=LBNKGPDBHCOMEHDMCDAOBBHD for www.whatarecookies.com/> > <Cookie dta=vcount%3D0%2Cprev%3D1480308117549 for www.whatarecookies.com/> > > Cookies after second 'Page.navigate' command: > <Cookie NID=91=08XuV0_PuzpDlm0ZkMDpig8p0M342GEmu1K9yHVC7xpD-n7e3_6XS4ruT70T717rjB22ByloeRrZb4ROvISN_Ig2opVs3tOOj3uAgkdwb9QesVK0IIiIhW2trR9NCQgxTH6QXq9a8GGzZgg for .google.com/> > > <snip a bunch of network messages> > > Sending sigint to chromium You can see after the first page.navigate to "http://www.whatarecookies.com/cookietest.asp" (e.g. "Site A"), Network.getCookies returns 5 different cookies. However, after navigating to "http://google.com" (e.g. "Site B"), Network.getCookies only returns a single cookie for google.com Did this work before? No Does this work in other browsers? Yes Chrome version: headless_shell 537.36 Channel: dev OS Version: 4.4.0-38-generic #57-Ubuntu, Ubuntu 16.04.1 LTS Flash Version: None I am in the process of implementing a python wrapper for the chrome remote debugger API, it's available here: https://github.com/fake-name/ChromeController (BSD licensed!) I'm currently working on allowing interaction between the python native cookie store (`http.cookielib.Cookie()`) and chromium's cookie store, but the documentation is rather sparse, and following the chromium source is challenging. In this case, I'm interested in being able to put the browser into a completely known state, including overriding all headers and saved cookies, and later read that state out. That means I really do need to be able to read out *all* saved cookies, not just the cookies for the currently active site.
,
Nov 28 2016
Arrgh, how did "Does this work in other browsers? Yes" end up in there? ------ Since it's not too obvious, I'm working with the headless version of chromium. I'm not sure the actual version, since I can't figure out how to extract the version information, aside from the user-agent string (which is headless_shell 537.36). I've tried all the command line switches I can think of that would return a version (-v --version --help -h etc...), and they all seem to do nothing. I'm also not sure what "channel" this would be, since I compiled it from source myself (about 2 weeks ago). ------ Also, why can I not copy-edit my comments?
,
Nov 28 2016
/cc skyostil for headless (I guess there's no bug label for that yet?)
,
Nov 28 2016
(We've only got a Proj-Headless label at the moment but no component) The Network.getCookies command returns the same cookies you can see in DevTools. Does that match what you're getting? I think the problem is that the way the command is implemented means that it's only returning the cookies for the current URL: https://cs.chromium.org/chromium/src/content/browser/devtools/protocol/network_handler.cc?rcl=0&l=213 Perhaps we should add a "global" argument that would retrieve all cookies instead?
,
Nov 28 2016
,
Dec 6 2016
,
Dec 7 2016
Just out of curiosity, what does "started" mean? I've been flailing about trying to get a build env setup so I could poke around, but if someone else is implementing this, I'll focus on other minor changes that I could use.
,
Dec 7 2016
I'm working on a patch that adds a way to get retrieve all cookies instead of just the cookies for the current URL.
,
Dec 8 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b0512a0bd23b83a5a00c79fd1169b820f3673e04 commit b0512a0bd23b83a5a00c79fd1169b820f3673e04 Author: skyostil <skyostil@chromium.org> Date: Thu Dec 08 15:14:18 2016 devtools: Make it possible to retrieve all cookies This patch adds a new "global" mode to the Network.getCookies command which allows retrieving all cookies in the browser cookie jar instead of just the cookies for the current URL. BUG= 668932 , 166637 Review-Url: https://codereview.chromium.org/2554293002 Cr-Commit-Position: refs/heads/master@{#437246} [modify] https://crrev.com/b0512a0bd23b83a5a00c79fd1169b820f3673e04/content/browser/devtools/protocol/devtools_protocol_browsertest.cc [modify] https://crrev.com/b0512a0bd23b83a5a00c79fd1169b820f3673e04/content/browser/devtools/protocol/network_handler.cc [modify] https://crrev.com/b0512a0bd23b83a5a00c79fd1169b820f3673e04/content/browser/devtools/protocol/network_handler.h [modify] https://crrev.com/b0512a0bd23b83a5a00c79fd1169b820f3673e04/third_party/WebKit/Source/core/inspector/browser_protocol.json [modify] https://crrev.com/b0512a0bd23b83a5a00c79fd1169b820f3673e04/third_party/WebKit/Source/devtools/front_end/sdk/CookieParser.js
,
Dec 8 2016
Marking this one fixed. If you find other problems with the cookie commands, please open a new bug.
,
Dec 9 2016
Awesomesauce! I see that the protocol.json file is still marked version 1.2. When does this version actually get incremented? Additionally, it appears there are actually *two* "getCookies" calls: "Network.getCookies" and "Page.getCookies". "Page.getCookies" is marked `"redirect": "Network"`, which I assume means "use the network version of this call", but it still has the old documentation string.
,
Dec 9 2016
Ok, I can confirm the fix is fully functional. Thanks so much!
,
Dec 13 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/9349e8fc1900cab593c7b3c17bc8d840fcae6f8d commit 9349e8fc1900cab593c7b3c17bc8d840fcae6f8d Author: skyostil <skyostil@chromium.org> Date: Tue Dec 13 17:12:37 2016 devtools: Split Network.getCookies into Network.get{All,}Cookies This patch splits the Network.getCookies command into Network.getCookies, which returns the cookies for the current URL and Network.getAllCookies, which returns all the cookies in the cookie jar. BUG= 668932 , 166637 Review-Url: https://codereview.chromium.org/2568503002 Cr-Commit-Position: refs/heads/master@{#438198} [modify] https://crrev.com/9349e8fc1900cab593c7b3c17bc8d840fcae6f8d/content/browser/devtools/protocol/devtools_protocol_browsertest.cc [modify] https://crrev.com/9349e8fc1900cab593c7b3c17bc8d840fcae6f8d/content/browser/devtools/protocol/network_handler.cc [modify] https://crrev.com/9349e8fc1900cab593c7b3c17bc8d840fcae6f8d/content/browser/devtools/protocol/network_handler.h [modify] https://crrev.com/9349e8fc1900cab593c7b3c17bc8d840fcae6f8d/content/browser/devtools/protocol_config.json [modify] https://crrev.com/9349e8fc1900cab593c7b3c17bc8d840fcae6f8d/third_party/WebKit/Source/core/inspector/browser_protocol.json [modify] https://crrev.com/9349e8fc1900cab593c7b3c17bc8d840fcae6f8d/third_party/WebKit/Source/core/inspector/inspector_protocol_config.json [modify] https://crrev.com/9349e8fc1900cab593c7b3c17bc8d840fcae6f8d/third_party/WebKit/Source/devtools/front_end/sdk/CookieParser.js
,
Dec 13 2016
Note: we decided to change the syntax a bit with the above patch. Instead of the 'global' flag there's now a separate Network.getAllCookies() command. |
||||||
►
Sign in to add a comment |
||||||
Comment 1 Deleted