If we are able to make test preferences available this way, we can also use it to bypass document blocking for file URLs when universal access for files is allowed.

I think this can be solved by

1. disabling CORB altogether when a test calls testRunner.addOriginAccessWhitelistEntry(...) or (testRunner.overridePreference("WebKitWebSecurityEnabled", false)

    - I think an easiest way would be to communicate this via
      test_runner::TestRunner::
      -> test_runner::WebTestRunner/content::BlinkTestRunner::DisableBrowserSideSecurityChecks()
      -> IPC
      -> content::BlinkTestController::OnDisableBrowserSideSecurityChecks
      -> content::CrossSiteDocumentResourceHandler::DisableForTesting(...)

2. unconditionally enabling CORB back again in BlinkTestController::PrepareForLayoutTest

(obviously #1 and #2 would still be subject to content::SiteIsolationPolicy::IsCrossSiteDocumentBlockingEnabled).


In theory we could be more granular/selective with the disabling logic (e.g. disable CORB only for the origin specified in testRunner.addOriginAccessWhitelistEntry, but it seems like an overkill at this point).

WIP CL @ https://chromium-review.googlesource.com/#/c/chromium/src/+/939532

Hmmm... I guess before starting to work on this bug, I should have verified that the tests actually fail without any changes.  It turns out that they don't fail at ToT (e.g. at r540191, see also https://crrev.com/c/942316).  And actually, I can't repro the test failures even at the initial CORB revision - r522016 (358baf47af51727d835c322b572d377d9be494f2):

$ git log --oneline | head -1
358baf47af51 Block cross-site document responses in Site Isolation modes.

$ DISPLAY=:20 third_party/WebKit/Tools/Scripts/run-webkit-tests -t gn --no-retry --additional-driver-flag=--site-per-process --additional-driver-flag=--isolate-origins=http://www.web-platform.test:8001/,http://www1.web-platform.test:8001/,http://www2.web-platform.test:8001/,http://xn--n8j6ds53lwwkrqhv28a.web-platform.test:8001/,http://xn--lve-6lad.web-platform.test:8001/,http://www.web-platform.test:8081/,http://www1.web-platform.test:8081/,http://www2.web-platform.test:8081/,http://xn--n8j6ds53lwwkrqhv28a.web-platform.test:8081/,http://xn--lve-6lad.web-platform.test:8081/,https://www.web-platform.test:8444/,https://www1.web-platform.test:8444/,https://www2.web-platform.test:8444/,https://xn--n8j6ds53lwwkrqhv28a.web-platform.test:8444/,https://xn--lve-6lad.web-platform.test:8444/ http/tests/xmlhttprequest
Using port 'linux-trusty'
Test configuration: <trusty, x86_64, release>
View the test results at file:///usr/local/google/home/lukasza/src/chromium4/src/out/gn/layout-test-results/results.html
Using random order with seed: 1519923737
Baseline search path: linux -> win -> site-per-process -> linux -> win -> generic
Using Release build
Pixel tests enabled
Regular timeout: 6000, slow test timeout: 30000
Command line: /usr/local/google/home/lukasza/src/chromium4/src/out/gn/content_shell --site-per-process --isolate-origins=http://www.web-platform.test:8001/,http://www1.web-platform.test:8001/,http://www2.web-platform.test:8001/,http://xn--n8j6ds53lwwkrqhv28a.web-platform.test:8001/,http://xn--lve-6lad.web-platform.test:8001/,http://www.web-platform.test:8081/,http://www1.web-platform.test:8081/,http://www2.web-platform.test:8081/,http://xn--n8j6ds53lwwkrqhv28a.web-platform.test:8081/,http://xn--lve-6lad.web-platform.test:8081/,https://www.web-platform.test:8444/,https://www1.web-platform.test:8444/,https://www2.web-platform.test:8444/,https://xn--n8j6ds53lwwkrqhv28a.web-platform.test:8444/,https://xn--lve-6lad.web-platform.test:8444/ --run-layout-test --ignore-certificate-errors-spki-list=Nxvaj3+bY3oVrTc+Jp7m3E3sB1n3lXtnMDCyBsqEXiY= --user-data-dir --enable-crash-reporter --crash-dumps-dir=/usr/local/google/home/lukasza/src/chromium4/src/out/gn/crash-dumps -

Found 326 tests; running 326, skipping 0.
Running 6 content_shells in parallel.                                                        

[92/326] http/tests/xmlhttprequest/workers/xmlhttprequest-allowed-with-disabled-web-security.html passed unexpectedly
[172/326] http/tests/xmlhttprequest/origin-whitelisting-removal.html passed unexpectedly          
[179/326] http/tests/xmlhttprequest/xmlhttprequest-allowed-with-disabled-web-security.html passed unexpectedly
[207/326] http/tests/xmlhttprequest/origin-whitelisting-subdomains.html passed unexpectedly
[305/326] http/tests/xmlhttprequest/origin-whitelisting-exact-match.html passed unexpectedly                      
                                                                                                   
321 tests ran as expected (319 passed, 2 didn't), 5 didn't.

Some ad-hoc, printf-style debugging from running the tests seems to indicate that XSDB/CORB 1) kicks in when expected, but 2) doesn't block the responses because they don't sniff as HTML/JSON/XML:


http/tests/xmlhttprequest/xmlhttprequest-allowed-with-disabled-web-security.html:

[88383:88458:0301/113637.184232:ERROR:cross_site_document_resource_handler.cc(731)] ShouldBlockBasedOnHeaders=false (3); request()->url() = http://127.0.0.1:8000/js-test-resources/js-test.js; initiator = http://127.0.0.1:8000
[88383:88458:0301/113637.608594:ERROR:cross_site_document_resource_handler.cc(731)] ShouldBlockBasedOnHeaders=false (3); request()->url() = http://127.0.0.1:8000/xmlhttprequest/resources/insecure-xhr.js; initiator = http://127.0.0.1:8000
[88383:88458:0301/113637.631791:ERROR:cross_site_document_resource_handler.cc(864)] ShouldBlockBasedOnHeaders=TRUE (12); request()->url() = http://localhost:8000/xmlhttprequest/resources/echo-request-origin.php; initiator = http://127.0.0.1:8000; canonical_mime_type_ = 3; needs_sniffing_ = 1; non_stylesheet_mime_type_ = 1
[88383:88458:0301/113637.631951:ERROR:cross_site_document_resource_handler.cc(517)] OnReadCompleted - after sniffing: ; confirmed_blockable = 0; data = Origin: <missing>


http/tests/xmlhttprequest/origin-whitelisting-exact-match.html:

[90471:90486:0301/114015.111838:ERROR:cross_site_document_resource_handler.cc(864)] ShouldBlockBasedOnHeaders=TRUE (12); request()->url() = http://localhost:8000/xmlhttprequest/resources/get.txt; initiator = http://127.0.0.1:8000; canonical_mime_type_ = 3; needs_sniffing_ = 1; non_stylesheet_mime_type_ = 1
[90471:90486:0301/114015.112081:ERROR:cross_site_document_resource_handler.cc(517)] OnReadCompleted - after sniffing: ; confirmed_blockable = 0; data = PASS
[90471:90486:0301/114015.129669:ERROR:cross_site_document_resource_handler.cc(864)] ShouldBlockBasedOnHeaders=TRUE (12); request()->url() = http://localhost:8000/xmlhttprequest/resources/get.txt; initiator = http://127.0.0.1:8000; canonical_mime_type_ = 3; needs_sniffing_ = 1; non_stylesheet_mime_type_ = 1
[90471:90486:0301/114015.129829:ERROR:cross_site_document_resource_handler.cc(517)] OnReadCompleted - after sniffing: ; confirmed_blockable = 0; data = PASS
[90471:90486:0301/114015.167048:ERROR:cross_site_document_resource_handler.cc(864)] ShouldBlockBasedOnHeaders=TRUE (12); request()->url() = http://localhost:8000/xmlhttprequest/resources/get.txt; initiator = http://127.0.0.1:8000; canonical_mime_type_ = 3; needs_sniffing_ = 1; non_stylesheet_mime_type_ = 1
[90471:90486:0301/114015.167186:ERROR:cross_site_document_resource_handler.cc(517)] OnReadCompleted - after sniffing: ; confirmed_blockable = 0; data = PASS
[90471:90486:0301/114015.178440:ERROR:cross_site_document_resource_handler.cc(864)] ShouldBlockBasedOnHeaders=TRUE (12); request()->url() = http://localhost:8000/xmlhttprequest/resources/get.txt; initiator = http://127.0.0.1:8000; canonical_mime_type_ = 3; needs_sniffing_ = 1; non_stylesheet_mime_type_ = 1
[90471:90486:0301/114015.178562:ERROR:cross_site_document_resource_handler.cc(517)] OnReadCompleted - after sniffing: ; confirmed_blockable = 0; data = PASS

Comments 5-6: Interesting!  I definitely saw those tests fail when working on my CL, such as these results from patch set 9 of https://chromium-review.googlesource.com/c/chromium/src/+/783826:

https://ci.chromium.org/buildbot/tryserver.chromium.linux/linux_site_isolation/11435
https://storage.googleapis.com/chromium-layout-test-archives/linux_site_isolation/11435/layout-test-results/results.html

Unexpected Failures:
* http/tests/xmlhttprequest/origin-whitelisting-exact-match.html
* http/tests/xmlhttprequest/origin-whitelisting-removal.html
* http/tests/xmlhttprequest/origin-whitelisting-subdomains.html
* http/tests/xmlhttprequest/workers/xmlhttprequest-allowed-with-disabled-web-security.html
* http/tests/xmlhttprequest/xmlhttprequest-allowed-with-disabled-web-security.html
* typedcssom/stylevalue-subclasses/numeric-objects/parse.html
* virtual/mojo-loading/http/tests/xmlhttprequest/origin-whitelisting-exact-match.html
* virtual/mojo-loading/http/tests/xmlhttprequest/origin-whitelisting-removal.html
* virtual/mojo-loading/http/tests/xmlhttprequest/origin-whitelisting-subdomains.html
* virtual/mojo-loading/http/tests/xmlhttprequest/workers/xmlhttprequest-allowed-with-disabled-web-security.html
* virtual/mojo-loading/http/tests/xmlhttprequest/xmlhttprequest-allowed-with-disabled-web-security.html

However, you raise a good point in comment 6-- I was seeing those failures and disabled the tests *prior* to adding content sniffing to the CL, and I didn't check to see if the tests would pass after adding that!  So I guess they were working at the time.  Feel free to re-enable them!

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/16295fe01a1264e488dc0f7837f53e6f01e74743

commit 16295fe01a1264e488dc0f7837f53e6f01e74743
Author: Lukasz Anforowicz <lukasza@chromium.org>
Date: Mon Mar 05 22:18:14 2018

Remove test exceptions for tests that have "healed" themselves

Bug: 477150,  789781 ,  769508 ,  661725 
Bug: 611232,  745881 ,  801992 
Change-Id: If90fb04e97513b99716afd844a2a77ca0905ab3d
Reviewed-on: https://chromium-review.googlesource.com/942316
Reviewed-by: Alex Moshchuk <alexmos@chromium.org>
Reviewed-by: Charlie Reis <creis@chromium.org>
Commit-Queue: Łukasz Anforowicz <lukasza@chromium.org>
Cr-Commit-Position: refs/heads/master@{#540958}
[modify] https://crrev.com/16295fe01a1264e488dc0f7837f53e6f01e74743/third_party/WebKit/LayoutTests/FlagExpectations/site-per-process