New issue
Advanced search Search tips

Issue 1997 link

Starred by 34 users

Issue metadata

Status: Fixed
Owner:
Closed: Nov 30

Blocking:
issue 2536



Sign in to add a comment

New Session is not spec compliant

Project Member Reported by kereliuk@chromium.org, Sep 8 2017

Issue description

We need to validate capabilities as is indicated in the spec:
https://w3c.github.io/webdriver/webdriver-spec.html#new-session


 
Project Member

Comment 1 by johnchen@chromium.org, Aug 3

Labels: StandardsCompliance
Blocking: 2536
Project Member

Comment 3 by johnchen@chromium.org, Aug 22

Labels: Pri-1
Owner: ----
Project Member

Comment 4 by johnchen@chromium.org, Sep 24

Owner: johnchen@chromium.org
Status: Started (was: Available)
Project Member

Comment 5 by bugdroid1@chromium.org, Sep 25

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e9d39029faf55d93a7e62c30ea9a244e48e0b132

commit e9d39029faf55d93a7e62c30ea9a244e48e0b132
Author: John Chen <johnchen@chromium.org>
Date: Tue Sep 25 17:36:23 2018

[ChromeDriver] Expand Capabilities for W3C compliance

This CL is one step in making InitSession command W3C compliant.
It expands Capabilities object to include all capabilities that are
specified in the W3C spec. Changes include:
* Re-ordered fields in Capabilities into W3C section and ChromeDriver
  specific section. Entries in W3C section are listed in the same
  order as they appear in W3C spec.
* Added browserName, browserVersion, and platformName capabilities.
  They will be needed for proper firstMatches handling.
* Added fields needed for timeouts capability. Parsing and using of
  these fields will be added in an upcoming CL.
* Replaced legacy name unexpected_alert_behaviour with W3C-compliant
  name unhandled_prompt_behavior.
* Modified Capabilities::Parse to accept all capability values,
  including those not yet supported by ChromeDriver. Added another
  method Capabilities::IsSupported to check for unsupported options.
  This is needed for upcoming W3C-compliant capability validation.

Bug:  chromedriver:1997 
Change-Id: I5f53341cf47f77a1853f40028b1efe6f0b69c611
Reviewed-on: https://chromium-review.googlesource.com/1241213
Commit-Queue: John Chen <johnchen@chromium.org>
Reviewed-by: Caleb Rouleau <crouleau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#593989}
[modify] https://crrev.com/e9d39029faf55d93a7e62c30ea9a244e48e0b132/chrome/test/chromedriver/capabilities.cc
[modify] https://crrev.com/e9d39029faf55d93a7e62c30ea9a244e48e0b132/chrome/test/chromedriver/capabilities.h
[modify] https://crrev.com/e9d39029faf55d93a7e62c30ea9a244e48e0b132/chrome/test/chromedriver/session.h
[modify] https://crrev.com/e9d39029faf55d93a7e62c30ea9a244e48e0b132/chrome/test/chromedriver/session_commands.cc
[modify] https://crrev.com/e9d39029faf55d93a7e62c30ea9a244e48e0b132/chrome/test/chromedriver/window_commands.cc

Project Member

Comment 8 by bugdroid1@chromium.org, Nov 9

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/886942637f14189c6bc2a1333c920dfbb93680cf

commit 886942637f14189c6bc2a1333c920dfbb93680cf
Author: John Chen <johnchen@chromium.org>
Date: Fri Nov 09 04:31:14 2018

[ChromeDriver] W3C compliant unknown capability handling

Per W3C spec, unrecognized capabilities should trigger invalid argument
error, instead of being silently ignored.

Bug:  chromedriver:1997 
Change-Id: I84b2a888ad97d709737368b9394c8efdd475bf3a
Reviewed-on: https://chromium-review.googlesource.com/c/1325562
Reviewed-by: Caleb Rouleau <crouleau@chromium.org>
Commit-Queue: John Chen <johnchen@chromium.org>
Cr-Commit-Position: refs/heads/master@{#606733}
[modify] https://crrev.com/886942637f14189c6bc2a1333c920dfbb93680cf/chrome/test/chromedriver/capabilities.cc
[modify] https://crrev.com/886942637f14189c6bc2a1333c920dfbb93680cf/chrome/test/chromedriver/capabilities.h
[modify] https://crrev.com/886942637f14189c6bc2a1333c920dfbb93680cf/chrome/test/chromedriver/capabilities_unittest.cc
[modify] https://crrev.com/886942637f14189c6bc2a1333c920dfbb93680cf/chrome/test/chromedriver/client/chromedriver.py
[modify] https://crrev.com/886942637f14189c6bc2a1333c920dfbb93680cf/chrome/test/chromedriver/log_replay/client_replay.py
[modify] https://crrev.com/886942637f14189c6bc2a1333c920dfbb93680cf/chrome/test/chromedriver/log_replay/client_replay_unittest.py
[modify] https://crrev.com/886942637f14189c6bc2a1333c920dfbb93680cf/chrome/test/chromedriver/session_commands.cc

Project Member

Comment 9 by bugdroid1@chromium.org, Nov 17

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f02041b4c0dd29f0b0490da666417d9f9e4a939d

commit f02041b4c0dd29f0b0490da666417d9f9e4a939d
Author: John Chen <johnchen@chromium.org>
Date: Sat Nov 17 01:47:42 2018

[ChromeDriver] Handle null capabilities

According to W3C spec, new session capabilities with null values should
be ignored, but they cause errors in ChromeDriver. This CL fixes it.

See https://www.w3.org/TR/webdriver/#dfn-validate-capabilities, step 3,
sub-steps 3 and 4.

Bug:  chromedriver:1997 
Change-Id: Ibee5e383c46139df1e5efc9f8a46872820c81046
Reviewed-on: https://chromium-review.googlesource.com/c/1340834
Commit-Queue: John Chen <johnchen@chromium.org>
Reviewed-by: Caleb Rouleau <crouleau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#609069}
[modify] https://crrev.com/f02041b4c0dd29f0b0490da666417d9f9e4a939d/chrome/test/chromedriver/capabilities.cc
[modify] https://crrev.com/f02041b4c0dd29f0b0490da666417d9f9e4a939d/chrome/test/chromedriver/session_commands.cc
[modify] https://crrev.com/f02041b4c0dd29f0b0490da666417d9f9e4a939d/chrome/test/chromedriver/session_commands_unittest.cc

Project Member

Comment 10 by bugdroid1@chromium.org, Nov 21

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a2c6ab4f1ba75cef6190a92b5ef6e7b79bc952f1

commit a2c6ab4f1ba75cef6190a92b5ef6e7b79bc952f1
Author: John Chen <johnchen@chromium.org>
Date: Wed Nov 21 21:39:45 2018

[ChromeDriver] W3C compliant New Session response

Updating capabilities returned from New Session command for W3C spec
compliance. Change verified by Web Platform Tests (13 additional
new_session tests passed).

Bug:  chromedriver:1997 
Change-Id: I7683acb33cbf2739a867787ba2a3fedd28be32c5
Reviewed-on: https://chromium-review.googlesource.com/c/1347065
Commit-Queue: John Chen <johnchen@chromium.org>
Reviewed-by: Caleb Rouleau <crouleau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#610223}
[modify] https://crrev.com/a2c6ab4f1ba75cef6190a92b5ef6e7b79bc952f1/chrome/test/chromedriver/session_commands.cc

Project Member

Comment 11 by bugdroid1@chromium.org, Nov 29

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3d0a07f6a474eb9a1735518a3d2928138543d59f

commit 3d0a07f6a474eb9a1735518a3d2928138543d59f
Author: Tatiana Buldina <buldina@chromium.org>
Date: Thu Nov 29 18:00:30 2018

[ChromeDriver] Add processing of extension capabilities contained ":"

Bug:  chromedriver:1997 
Change-Id: Ic231f01e68cc141877d501b5aee6926bc48ac1ee
Reviewed-on: https://chromium-review.googlesource.com/c/1355172
Reviewed-by: John Chen <johnchen@chromium.org>
Commit-Queue: Tatiana Buldina <buldina@chromium.org>
Cr-Commit-Position: refs/heads/master@{#612266}
[modify] https://crrev.com/3d0a07f6a474eb9a1735518a3d2928138543d59f/chrome/test/chromedriver/capabilities.cc

Project Member

Comment 12 by bugdroid1@chromium.org, Nov 29

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/30e5bc4548e97b64b2116f9cdc506061879ec67b

commit 30e5bc4548e97b64b2116f9cdc506061879ec67b
Author: John Chen <johnchen@chromium.org>
Date: Thu Nov 29 21:38:09 2018

[ChromeDriver] W3c-compliant proxyType parsing

The proxyType string should be handled case-sensitively in W3C mode.
Resolves 2 WPT test failures in
webdriver/tests/new_session/invalid_capabilities.py.

Bug:  chromedriver:1997 
Change-Id: If931b4d1d982aae8646e9b5783cbf7ee52fa4b60
Reviewed-on: https://chromium-review.googlesource.com/c/1355437
Reviewed-by: Caleb Rouleau <crouleau@chromium.org>
Commit-Queue: John Chen <johnchen@chromium.org>
Cr-Commit-Position: refs/heads/master@{#612369}
[modify] https://crrev.com/30e5bc4548e97b64b2116f9cdc506061879ec67b/chrome/test/chromedriver/capabilities.cc
[modify] https://crrev.com/30e5bc4548e97b64b2116f9cdc506061879ec67b/chrome/test/chromedriver/capabilities_unittest.cc

Project Member

Comment 13 by johnchen@chromium.org, Nov 30

Labels: ChromeDriver-2.45 ToBeReleased
Status: Fixed (was: Started)
With the changes recorded in comments 5 through 12, the New Session command is now very close to W3C spec, and I'm resolving this issue as Fixed. Currently (as of WPT commit 25f3222fcb and Chromium r612369), there are still 21 WPT subtest failures inside webdriver/tests/new_session directory. These are listed below and will be tracked independently from this issue.

6 failures because strictFileInteractability is not yet implemented,  issue 2654 :
* create_alwaysMatch.py and create_firstMatch.py: test_valid[strictFileInteractability-True] and test_valid[strictFileInteractability-False]
* response.py: test_capability_type[strictFileInteractability-bool] and test_capability_default_value[strictFileInteractability-False]

6 failures because eager pageLoadStrategy is not yet implemented, issue 1902:
* create_alwaysMatch.py and create_firstMatch.py: test_valid[pageLoadStrategy-eager]
* page_load_strategy.py: test_pageLoadStrategy
* merge.py: test_merge_platformName and test_merge_browserName
* default_values.py: test_valid_but_unmatchable_key

2 failures due to difficulty with large timeout value,  issue 2679 :
* create_alwaysMatch.py and create_firstMatch.py: test_valid[timeouts-value12]

6 failures because ChromeDriver defaults to legacy mode, issue 2536:
* invalid_capabilities.py: 4 test_invalid_capabilites subtests
* default_values.py: test_no_capabilites and test_desired

1 failure because ChromeDriver supports multiple sessions. W3C spec [1] allows each endpoint node to support only 1 session, but ChromeDriver has long supported multiple sessions, limited only by the amount of available system resources. Technically ChromeDriver is violating the spec here, but I don't see any need to disable ChromeDriver's multi-session support. Intentional Won't Fix:
* default_values.py: test_repeat_new_session

[1] https://www.w3.org/TR/2018/REC-webdriver1-20180605/#dfn-maximum-active-sessions
Project Member

Comment 14 by johnchen@chromium.org, Dec 11

Labels: -ToBeReleased -ChromeDriver-2.45
Project Member

Comment 15 by johnchen@chromium.org, Dec 11

Labels: ChromeDriver-2.45
Is it really fixed assuming that it's impossible to start ChromeDriver without "desiredCapabilities"? An attempt to start a new session with the following payload:

    {"capabilities":{"alwaysMatch":[{"browserName":"chrome"}]}

results in the following errored response:

    {"sessionId":"c86df74881446df202eb8f9c75e2f52b","status":33,"value":{"message":"session not created: Missing or invalid capabilities\n  (Driver info: chromedriver=2.45.615355 (d5698f682d8b2742017df6c81e0bd8e6a3063189),platform=Mac OS X 10.14.2 x86_64)"}
Project Member

Comment 17 by johnchen@chromium.org, Jan 8

ChromeDriver still operates in OSS mode by default, pending resolution of issue 2536. Until that is done, it is still necessary to explicitly select W3C mode, e.g.,

curl -XPOST http://localhost:9515/session -d '{"capabilities":{"alwaysMatch":{"goog:chromeOptions":{"w3c":true}}}}'
Oh, I wasn't aware of "w3c" option. Thank you for explanation!

Sign in to add a comment