New issue
Advanced search Search tips

Issue 668932 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Dec 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 2
Type: Bug



Sign in to add a comment

How does `Network.getCookies` in the remote debugger interface actually work?

Reported by lem...@gmail.com, Nov 28 2016

Issue description

UserAgent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.100 Safari/537.36 Vivaldi/1.5.658.44

Steps to reproduce the problem:
1. Using remote debug interaface, navigate to site A which sets a cookie. 
2. Then, in the same tab, navigate to site B
3. Execute `Network.getCookies` remote debugger command.

What is the expected behavior?
All cookies, including cookies for site A and site B should be returned (the documentation for `Network.getCookies` states: "Returns all browser cookies. Depending on the backend support, will return detailed cookie information in the <code>cookies</code> field."

Instead, only cookies for the currently active site are returned, in this case, site B.

What went wrong?
Either the documentation for what `Network.getCookies` does is wrong, and it only returns cookies scoped to the currently active webpage, or the function call is not returning the correct value.

---

Attached is a lot of the web-socket messages send while executing my test-script here: https://github.com/fake-name/ChromeController/blob/master/cookie_wtf.py

Briefly:

>	Spawned process: <subprocess.Popen object at 0x7f53e4581d68>
>	<ChromeController.transport.ChromeSocketManager object at 0x7f53e3ff44a8>
>	Synchronous_command:
>		command: 'Page.enable'
>		params:  '{}'
>			Sending: '{"id": 0, "method": "Page.enable"}'
>			Received: '{"id":0,"result":{}}'
>		Response: '{'id': 0, 'result': {}}'
>	Synchronous_command:
>		command: 'DOM.enable'
>		params:  '{}'
>			Sending: '{"id": 1, "method": "DOM.enable"}'
>			Received: '{"id":1,"result":{}}'
>		Response: '{'id': 1, 'result': {}}'
>	Synchronous_command:
>		command: 'Network.enable'
>		params:  '{}'
>			Sending: '{"id": 2, "method": "Network.enable"}'
>			Received: '{"id":2,"result":{}}'
>		Response: '{'id': 2, 'result': {}}'
>	Synchronous_command:
>		command: 'Emulation.setVisibleSize'
>		params:  '{'height': 1000, 'width': 1500}'
>			Sending: '{"id": 3, "method": "Emulation.setVisibleSize", "params": {"height": 1000, "width": 1500}}'
>			Received: '{"id":3,"result":{}}'
>		Response: '{'id': 3, 'result': {}}'
>	Synchronous_command:
>		command: 'Page.navigate'
>		params:  '{'url': 'http://www.whatarecookies.com/cookietest.asp'}'
>			Sending: '{"id": 4, "method": "Page.navigate", "params": {"url": "http://www.whatarecookies.com/cookietest.asp"}}'
>			Received: '{"method":"Page.frameResized","params":{}}'
>			<snip a bunch of network messages>
>			Received: '{"id":4,"result":{"frameId":"13569.1"}}'
>		Response: '{'id': 4, 'result': {'frameId': '13569.1'}}'
>			<snip a bunch of network messages>
>			Received: '{"method":"Page.frameStoppedLoading","params":{"frameId":"13569.1"}}'
>	Synchronous_command:
>		command: 'Network.getCookies'
>		params:  '{}'
>			Sending: '{"id": 5, "method": "Network.getCookies"}'
>			Received: '{"id":5,"result":{"cookies":[{"domain":".doubleclick.net","expires":1480309018414.03,"httpOnly":false,"name":"test_cookie","path":"/","secure":false,"session":false,"size":29,"value":"CheckForPermission"},{"domain":".whatarecookies.com","expires":1480308717000.0,"httpOnly":false,"name":"_gat","path":"/","secure":false,"session":false,"size":5,"value":"1"},{"domain":".whatarecookies.com","expires":1543380117000.0,"httpOnly":false,"name":"_ga","path":"/","secure":false,"session":false,"size":29,"value":"GA1.2.497879937.1480308118"},{"domain":"www.whatarecookies.com","expires":0.0,"httpOnly":false,"name":"ASPSESSIONIDASBDRQQD","path":"/","secure":false,"session":true,"size":44,"value":"LBNKGPDBHCOMEHDMCDAOBBHD"},{"domain":"www.whatarecookies.com","expires":1480309317000.0,"httpOnly":false,"name":"dta","path":"/","secure":false,"session":false,"size":36,"value":"vcount%3D0%2Cprev%3D1480308117549"}]}}'
>		Response: '{'id': 5, 'result': {'cookies': [{'path': '/', 'value': 'CheckForPermission', 'name': 'test_cookie', 'domain': '.doubleclick.net', 'httpOnly': False, 'expires': 1480309018414.03, 'session': False, 'secure': False, 'size': 29}, {'path': '/', 'value': '1', 'name': '_gat', 'domain': '.whatarecookies.com', 'httpOnly': False, 'expires': 1480308717000.0, 'session': False, 'secure': False, 'size': 5}, {'path': '/', 'value': 'GA1.2.497879937.1480308118', 'name': '_ga', 'domain': '.whatarecookies.com', 'httpOnly': False, 'expires': 1543380117000.0, 'session': False, 'secure': False, 'size': 29}, {'path': '/', 'value': 'LBNKGPDBHCOMEHDMCDAOBBHD', 'name': 'ASPSESSIONIDASBDRQQD', 'domain': 'www.whatarecookies.com', 'httpOnly': False, 'expires': 0.0, 'session': True, 'secure': False, 'size': 44}, {'path': '/', 'value': 'vcount%3D0%2Cprev%3D1480308117549', 'name': 'dta', 'domain': 'www.whatarecookies.com', 'httpOnly': False, 'expires': 1480309317000.0, 'session': False, 'secure': False, 'size': 36}]}}'
>	Synchronous_command:
>		command: 'Page.navigate'
>		params:  '{'url': 'http://google.com'}'
>			Sending: '{"id": 6, "method": "Page.navigate", "params": {"url": "http://google.com"}}'
>			Received: '{"id":6,"result":{"frameId":"13586.1"}}'
>		Response: '{'id': 6, 'result': {'frameId': '13586.1'}}'
>			<snip a bunch of network messages>
>			Received: '{"method":"Page.frameStoppedLoading","params":{"frameId":"13586.1"}}'
>	Synchronous_command:
>		command: 'Network.getCookies'
>		params:  '{}'
>			Sending: '{"id": 7, "method": "Network.getCookies"}'
>			Received: '{"id":7,"result":{"cookies":[{"domain":".google.com","expires":1496119318679.717,"httpOnly":true,"name":"NID","path":"/","secure":false,"session":false,"size":149,"value":"91=08XuV0_PuzpDlm0ZkMDpig8p0M342GEmu1K9yHVC7xpD-n7e3_6XS4ruT70T717rjB22ByloeRrZb4ROvISN_Ig2opVs3tOOj3uAgkdwb9QesVK0IIiIhW2trR9NCQgxTH6QXq9a8GGzZgg"}]}}'
>		Response: '{'id': 7, 'result': {'cookies': [{'path': '/', 'value': '91=08XuV0_PuzpDlm0ZkMDpig8p0M342GEmu1K9yHVC7xpD-n7e3_6XS4ruT70T717rjB22ByloeRrZb4ROvISN_Ig2opVs3tOOj3uAgkdwb9QesVK0IIiIhW2trR9NCQgxTH6QXq9a8GGzZgg', 'name': 'NID', 'domain': '.google.com', 'httpOnly': True, 'expires': 1496119318679.717, 'session': False, 'secure': False, 'size': 149}]}}'
>
>	Cookies after first 'Page.navigate' command:
>		 <Cookie test_cookie=CheckForPermission for .doubleclick.net/>
>		 <Cookie _gat=1 for .whatarecookies.com/>
>		 <Cookie _ga=GA1.2.497879937.1480308118 for .whatarecookies.com/>
>		 <Cookie ASPSESSIONIDASBDRQQD=LBNKGPDBHCOMEHDMCDAOBBHD for www.whatarecookies.com/>
>		 <Cookie dta=vcount%3D0%2Cprev%3D1480308117549 for www.whatarecookies.com/>
>
>	Cookies after second 'Page.navigate' command:
>		 <Cookie NID=91=08XuV0_PuzpDlm0ZkMDpig8p0M342GEmu1K9yHVC7xpD-n7e3_6XS4ruT70T717rjB22ByloeRrZb4ROvISN_Ig2opVs3tOOj3uAgkdwb9QesVK0IIiIhW2trR9NCQgxTH6QXq9a8GGzZgg for .google.com/>
>
>	<snip a bunch of network messages>
>
>	Sending sigint to chromium

You can see after the first page.navigate to "http://www.whatarecookies.com/cookietest.asp" (e.g. "Site A"), Network.getCookies returns 5 different cookies. However, after navigating to "http://google.com" (e.g. "Site B"), Network.getCookies only returns a single cookie for google.com

Did this work before? No 

Does this work in other browsers? Yes

Chrome version: headless_shell 537.36  Channel: dev
OS Version: 4.4.0-38-generic #57-Ubuntu, Ubuntu 16.04.1 LTS
Flash Version: None

I am in the process of implementing a python wrapper for the chrome remote debugger API, it's available here: https://github.com/fake-name/ChromeController (BSD licensed!)

I'm currently working on allowing interaction between the python native cookie store (`http.cookielib.Cookie()`) and chromium's cookie store, but the documentation is rather sparse, and following the chromium source is challenging.

In this case, I'm interested in being able to put the browser into a completely known state, including overriding all headers and saved cookies, and later read that state out. That means I really do need to be able to read out *all* saved cookies, not just the cookies for the currently active site.
 
test-log.txt
127 KB View Download

Comment 1 Deleted

Comment 2 Deleted

Comment 3 by lem...@gmail.com, Nov 28 2016

Arrgh, how did "Does this work in other browsers? Yes" end up in there?

------

Since it's not too obvious, I'm working with the headless version of chromium. I'm not sure the actual version, since I can't figure out how to extract the version information, aside from the user-agent string (which is headless_shell 537.36). I've tried all the command line switches I can think of that would return a version (-v --version --help -h etc...), and they all seem to do nothing.

I'm also not sure what "channel" this would be, since I compiled it from source myself (about 2 weeks ago).

------

Also, why can I not copy-edit my comments?

Comment 4 by rbyers@chromium.org, Nov 28 2016

Cc: skyos...@chromium.org
Components: Platform>DevTools
Labels: -Hotlist-Interop
/cc skyostil for headless (I guess there's no bug label for that yet?)
Labels: Proj-Headless
(We've only got a Proj-Headless label at the moment but no component)

The Network.getCookies command returns the same cookies you can see in DevTools. Does that match what you're getting?

I think the problem is that the way the command is implemented means that it's only returning the cookies for the current URL: https://cs.chromium.org/chromium/src/content/browser/devtools/protocol/network_handler.cc?rcl=0&l=213

Perhaps we should add a "global" argument that would retrieve all cookies instead?

Status: Available (was: Unconfirmed)
Owner: skyos...@chromium.org
Status: Started (was: Available)

Comment 8 by lem...@gmail.com, Dec 7 2016

Just out of curiosity, what does "started" mean?

I've been flailing about trying to get a build env setup so I could poke around, but if someone else is implementing this, I'll focus on other minor changes that I could use.
I'm working on a patch that adds a way to get retrieve all cookies instead of just the cookies for the current URL.
Status: Fixed (was: Started)
Marking this one fixed. If you find other problems with the cookie commands, please open a new bug.

Comment 12 by lem...@gmail.com, Dec 9 2016

Awesomesauce!

I see that the protocol.json file is still marked version 1.2. When does this version actually get incremented?

Additionally, it appears there are actually *two* "getCookies" calls: "Network.getCookies" and "Page.getCookies". "Page.getCookies" is marked `"redirect": "Network"`, which I assume means "use the network version of this call", but it still has the old documentation string. 

Comment 13 by lem...@gmail.com, Dec 9 2016

Ok, I can confirm the fix is fully functional. Thanks so much!
Project Member

Comment 14 by bugdroid1@chromium.org, Dec 13 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9349e8fc1900cab593c7b3c17bc8d840fcae6f8d

commit 9349e8fc1900cab593c7b3c17bc8d840fcae6f8d
Author: skyostil <skyostil@chromium.org>
Date: Tue Dec 13 17:12:37 2016

devtools: Split Network.getCookies into Network.get{All,}Cookies

This patch splits the Network.getCookies command into
Network.getCookies, which returns the cookies for the current URL and
Network.getAllCookies, which returns all the cookies in the cookie jar.

BUG= 668932 , 166637 

Review-Url: https://codereview.chromium.org/2568503002
Cr-Commit-Position: refs/heads/master@{#438198}

[modify] https://crrev.com/9349e8fc1900cab593c7b3c17bc8d840fcae6f8d/content/browser/devtools/protocol/devtools_protocol_browsertest.cc
[modify] https://crrev.com/9349e8fc1900cab593c7b3c17bc8d840fcae6f8d/content/browser/devtools/protocol/network_handler.cc
[modify] https://crrev.com/9349e8fc1900cab593c7b3c17bc8d840fcae6f8d/content/browser/devtools/protocol/network_handler.h
[modify] https://crrev.com/9349e8fc1900cab593c7b3c17bc8d840fcae6f8d/content/browser/devtools/protocol_config.json
[modify] https://crrev.com/9349e8fc1900cab593c7b3c17bc8d840fcae6f8d/third_party/WebKit/Source/core/inspector/browser_protocol.json
[modify] https://crrev.com/9349e8fc1900cab593c7b3c17bc8d840fcae6f8d/third_party/WebKit/Source/core/inspector/inspector_protocol_config.json
[modify] https://crrev.com/9349e8fc1900cab593c7b3c17bc8d840fcae6f8d/third_party/WebKit/Source/devtools/front_end/sdk/CookieParser.js

Note: we decided to change the syntax a bit with the above patch. Instead of the 'global' flag there's now a separate Network.getAllCookies() command.

Sign in to add a comment