New issue
Advanced search Search tips

Issue 811460 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jun 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Feature

Blocked on:
issue 824576

Blocking:
issue 696822



Sign in to add a comment

DNR: Add ability to whitelist top level urls.

Project Member Reported by karandeepb@chromium.org, Feb 12 2018

Issue description

Add an extension API which allows the extensions to whitelist top level URLs for declarative Net Request. The API would look as follows:

   - chrome.declarativeNetRequest.whitelistPage([array of URL match patterns])
   - chrome.declarativeNetRequest.removeWhitelistedPages([array of URL match patterns])
   - chrome.declarativeNetRequest.getWhitelistedPages() 

As per discussion, the current plan is to not support dynamic rules. Allowing extensions to whitelist pages/sites should suffice for most of the important use cases of dynamic rules.
 
>- chrome.declarativeNetRequest.whitelistPage([array of URL match patterns])

nit: maybe addWhitelistedPages(), since it takes an array, and for consistency with the other methods?
Yeah that sg as well.
Blockedon: 824576
Project Member

Comment 4 by bugdroid1@chromium.org, Mar 22 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5325a952741fffc94b07ef324266cdadea3fa999

commit 5325a952741fffc94b07ef324266cdadea3fa999
Author: Karan Bhatia <karandeepb@chromium.org>
Date: Thu Mar 22 20:17:51 2018

Allow NavigationHandle::GetRenderFrameHost to always be called.

While the NavigationHandle in WebContentsObserver::ReadyToCommitNavigation is
always accessible, the same is not true for uncommitted navigations in
WebContentsObserver::DidFinishNavigation. This CL removes this inconsistency in
the WebContentsObserver interface by allowing
NavigationHandle::GetRenderFrameHost to always be called.

BUG= 824576 ,  811460 

Change-Id: I4f37a5fd4372eb505b065368312ebfc0c5cd730e
Reviewed-on: https://chromium-review.googlesource.com/972667
Reviewed-by: Camille Lamy <clamy@chromium.org>
Commit-Queue: Karan Bhatia <karandeepb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#545208}
[modify] https://crrev.com/5325a952741fffc94b07ef324266cdadea3fa999/content/browser/frame_host/navigation_handle_impl.cc
[modify] https://crrev.com/5325a952741fffc94b07ef324266cdadea3fa999/content/public/browser/navigation_handle.h
[modify] https://crrev.com/5325a952741fffc94b07ef324266cdadea3fa999/content/test/web_contents_observer_sanity_checker.cc

Project Member

Comment 5 by bugdroid1@chromium.org, Mar 22 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5070af30d42677c1e2b06052d189bdf493bd4b4c

commit 5070af30d42677c1e2b06052d189bdf493bd4b4c
Author: Karan Bhatia <karandeepb@chromium.org>
Date: Thu Mar 22 21:27:07 2018

Revert "Allow NavigationHandle::GetRenderFrameHost to always be called."

This reverts commit 5325a952741fffc94b07ef324266cdadea3fa999.

Reason for revert: There isn't consensus currently whether this is the correct thing to do from an API standpoint. Reverting temporarily till we can ensure the same.

Original change's description:
> Allow NavigationHandle::GetRenderFrameHost to always be called.
> 
> While the NavigationHandle in WebContentsObserver::ReadyToCommitNavigation is
> always accessible, the same is not true for uncommitted navigations in
> WebContentsObserver::DidFinishNavigation. This CL removes this inconsistency in
> the WebContentsObserver interface by allowing
> NavigationHandle::GetRenderFrameHost to always be called.
> 
> BUG= 824576 ,  811460 
> 
> Change-Id: I4f37a5fd4372eb505b065368312ebfc0c5cd730e
> Reviewed-on: https://chromium-review.googlesource.com/972667
> Reviewed-by: Camille Lamy <clamy@chromium.org>
> Commit-Queue: Karan Bhatia <karandeepb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#545208}

TBR=clamy@chromium.org,karandeepb@chromium.org

Change-Id: I5a06c842a0a87cbc1b8a87c18db1ec313b83ac2f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug:  824576 ,  811460 
Reviewed-on: https://chromium-review.googlesource.com/976604
Reviewed-by: Karan Bhatia <karandeepb@chromium.org>
Commit-Queue: Karan Bhatia <karandeepb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#545255}
[modify] https://crrev.com/5070af30d42677c1e2b06052d189bdf493bd4b4c/content/browser/frame_host/navigation_handle_impl.cc
[modify] https://crrev.com/5070af30d42677c1e2b06052d189bdf493bd4b4c/content/public/browser/navigation_handle.h
[modify] https://crrev.com/5070af30d42677c1e2b06052d189bdf493bd4b4c/content/test/web_contents_observer_sanity_checker.cc

Status: Started (was: Assigned)
Project Member

Comment 7 by bugdroid1@chromium.org, Mar 27 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a2b3aa8df0cfbec673b199e51cf6ff3e8021ec3f

commit a2b3aa8df0cfbec673b199e51cf6ff3e8021ec3f
Author: Karan Bhatia <karandeepb@chromium.org>
Date: Tue Mar 27 23:44:32 2018

DNR: Add page whitelisting API.

This CL implements part of the page whitelisting API for the Declarative Net
Request API. Three new extension functions are introduced:
  - chrome.declarativeNetRequest.addWhitelistedPages
  - chrome.declarativeNetRequest.removeWhitelistedPages
  - chrome.declarativeNetRequest.getWhitelistedPages

These functions are used to add, remove and get the current set of whitelisted
pages for an extension. Whitelisted pages are specified as match patterns. If
the main frame URL of a request matches the current set of whitelisted pages,
the extension ruleset won't act on the request.

This CL implements the extension functions and persists the set of whitelisted
pages as part of Extension preferences. In a subsequent CL, the whitelisted
pages set would be passed to the RulesetManager on the IO thread, which will use
it to whitelist requests.

BUG= 811460 
Doc=https://docs.google.com/document/d/1Fhm-t0JCc3dcmwyBX_i7TGyuJ2QjqDechv1-VCRBo5o/edit?usp=sharing

Change-Id: Ic8e449449748dbb3c89b24bab68ab6a55367fee0
Reviewed-on: https://chromium-review.googlesource.com/965564
Commit-Queue: Karan Bhatia <karandeepb@chromium.org>
Reviewed-by: Ilya Sherman <isherman@chromium.org>
Reviewed-by: Istiaque Ahmed <lazyboy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#546316}
[add] https://crrev.com/a2b3aa8df0cfbec673b199e51cf6ff3e8021ec3f/chrome/browser/extensions/api/declarative_net_request/declarative_net_request_apitest.cc
[modify] https://crrev.com/a2b3aa8df0cfbec673b199e51cf6ff3e8021ec3f/chrome/browser/extensions/api/declarative_net_request/declarative_net_request_browsertest.cc
[modify] https://crrev.com/a2b3aa8df0cfbec673b199e51cf6ff3e8021ec3f/chrome/test/BUILD.gn
[add] https://crrev.com/a2b3aa8df0cfbec673b199e51cf6ff3e8021ec3f/chrome/test/data/extensions/api_test/declarative_net_request/background.js
[add] https://crrev.com/a2b3aa8df0cfbec673b199e51cf6ff3e8021ec3f/chrome/test/data/extensions/api_test/declarative_net_request/manifest.json
[add] https://crrev.com/a2b3aa8df0cfbec673b199e51cf6ff3e8021ec3f/chrome/test/data/extensions/api_test/declarative_net_request/rules_file_empty.json
[modify] https://crrev.com/a2b3aa8df0cfbec673b199e51cf6ff3e8021ec3f/extensions/browser/api/declarative_net_request/BUILD.gn
[add] https://crrev.com/a2b3aa8df0cfbec673b199e51cf6ff3e8021ec3f/extensions/browser/api/declarative_net_request/declarative_net_request_api.cc
[add] https://crrev.com/a2b3aa8df0cfbec673b199e51cf6ff3e8021ec3f/extensions/browser/api/declarative_net_request/declarative_net_request_api.h
[modify] https://crrev.com/a2b3aa8df0cfbec673b199e51cf6ff3e8021ec3f/extensions/browser/extension_function_histogram_value.h
[modify] https://crrev.com/a2b3aa8df0cfbec673b199e51cf6ff3e8021ec3f/extensions/browser/extension_prefs.cc
[modify] https://crrev.com/a2b3aa8df0cfbec673b199e51cf6ff3e8021ec3f/extensions/browser/extension_prefs.h
[modify] https://crrev.com/a2b3aa8df0cfbec673b199e51cf6ff3e8021ec3f/extensions/common/api/declarative_net_request.idl
[modify] https://crrev.com/a2b3aa8df0cfbec673b199e51cf6ff3e8021ec3f/extensions/common/api/declarative_net_request/test_utils.cc
[modify] https://crrev.com/a2b3aa8df0cfbec673b199e51cf6ff3e8021ec3f/extensions/common/api/declarative_net_request/test_utils.h
[modify] https://crrev.com/a2b3aa8df0cfbec673b199e51cf6ff3e8021ec3f/tools/metrics/histograms/enums.xml

Project Member

Comment 8 by bugdroid1@chromium.org, Mar 29 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c3a610ca9d349de254050ea58ac2735db7df7088

commit c3a610ca9d349de254050ea58ac2735db7df7088
Author: Karan Bhatia <karandeepb@chromium.org>
Date: Thu Mar 29 18:57:39 2018

Extensions: Add main frame url info. to extension FrameData.

This CL adds "last_committed_main_frame_url" and "pending_main_frame_url" to the
extension FrameData. "last_committed_main_frame_url" is used to track the the
last committed url of the frames's web contents and "pending_main_frame_url" is
used to track an active main frame navigation. "pending_main_frame_url" will be
required for main frame subresource loads which cannot be reliably mapped to a
single main frame url currently.

In a subsequent CL, a page whitelisting API will be introduced for the
Declarative Net Request API and this information related to the main frame url
will be used to check whether a network request has been whitelisted as per the
API.

BUG= 811460 , 696822
Doc=https://docs.google.com/document/d/1Fhm-t0JCc3dcmwyBX_i7TGyuJ2QjqDechv1-VCRBo5o/edit?usp=sharing

Change-Id: Ie6588955ca5105af0bca4ed4408a141b0d586e44
Reviewed-on: https://chromium-review.googlesource.com/936325
Reviewed-by: Nasko Oskov <nasko@chromium.org>
Reviewed-by: Devlin <rdevlin.cronin@chromium.org>
Commit-Queue: Karan Bhatia <karandeepb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#546910}
[modify] https://crrev.com/c3a610ca9d349de254050ea58ac2735db7df7088/chrome/browser/extensions/web_contents_browsertest.cc
[modify] https://crrev.com/c3a610ca9d349de254050ea58ac2735db7df7088/extensions/browser/extension_api_frame_id_map.cc
[modify] https://crrev.com/c3a610ca9d349de254050ea58ac2735db7df7088/extensions/browser/extension_api_frame_id_map.h
[modify] https://crrev.com/c3a610ca9d349de254050ea58ac2735db7df7088/extensions/browser/extension_api_frame_id_map_unittest.cc
[modify] https://crrev.com/c3a610ca9d349de254050ea58ac2735db7df7088/extensions/browser/extension_navigation_ui_data.cc
[modify] https://crrev.com/c3a610ca9d349de254050ea58ac2735db7df7088/extensions/browser/extension_web_contents_observer.cc
[modify] https://crrev.com/c3a610ca9d349de254050ea58ac2735db7df7088/extensions/browser/extension_web_contents_observer.h

Project Member

Comment 9 by bugdroid1@chromium.org, Apr 3 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/158b85420b90f2c325a6c71caba992c40b34610e

commit 158b85420b90f2c325a6c71caba992c40b34610e
Author: Karan Bhatia <karandeepb@chromium.org>
Date: Tue Apr 03 07:07:15 2018

DNR: Ensure page whitelisting API can't be called by extensions with no rulesets.

This CL ensures that all the page whitelisting API functions
(chrome.declarativeNetRequest.*) can't be called by extensions which don't have
a declarative ruleset. It doesn't make sense for extensions to whitelist page
patterns if they don't have a registered declarative ruleset.

BUG= 811460 , 696822
Doc=https://docs.google.com/document/d/1Fhm-t0JCc3dcmwyBX_i7TGyuJ2QjqDechv1-VCRBo5o/edit?usp=sharing

Change-Id: Id2822af5f258a32889e9217a3c63e13879984312
Reviewed-on: https://chromium-review.googlesource.com/987314
Commit-Queue: Karan Bhatia <karandeepb@chromium.org>
Reviewed-by: Istiaque Ahmed <lazyboy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#547636}
[modify] https://crrev.com/158b85420b90f2c325a6c71caba992c40b34610e/chrome/browser/extensions/api/declarative_net_request/declarative_net_request_apitest.cc
[add] https://crrev.com/158b85420b90f2c325a6c71caba992c40b34610e/chrome/test/data/extensions/api_test/declarative_net_request/extension_with_no_ruleset/background.js
[add] https://crrev.com/158b85420b90f2c325a6c71caba992c40b34610e/chrome/test/data/extensions/api_test/declarative_net_request/extension_with_no_ruleset/manifest.json
[rename] https://crrev.com/158b85420b90f2c325a6c71caba992c40b34610e/chrome/test/data/extensions/api_test/declarative_net_request/page_whitelisting_api/background.js
[rename] https://crrev.com/158b85420b90f2c325a6c71caba992c40b34610e/chrome/test/data/extensions/api_test/declarative_net_request/page_whitelisting_api/manifest.json
[rename] https://crrev.com/158b85420b90f2c325a6c71caba992c40b34610e/chrome/test/data/extensions/api_test/declarative_net_request/page_whitelisting_api/rules_file_empty.json
[modify] https://crrev.com/158b85420b90f2c325a6c71caba992c40b34610e/extensions/browser/api/declarative_net_request/declarative_net_request_api.cc
[modify] https://crrev.com/158b85420b90f2c325a6c71caba992c40b34610e/extensions/browser/api/declarative_net_request/declarative_net_request_api.h
[modify] https://crrev.com/158b85420b90f2c325a6c71caba992c40b34610e/extensions/browser/api/declarative_net_request/rules_monitor_service.cc
[modify] https://crrev.com/158b85420b90f2c325a6c71caba992c40b34610e/extensions/browser/api/declarative_net_request/rules_monitor_service.h

Project Member

Comment 10 by bugdroid1@chromium.org, Apr 4 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/72e12fdc05df67024f7adda2aa8319a33653261f

commit 72e12fdc05df67024f7adda2aa8319a33653261f
Author: Karan Bhatia <karandeepb@chromium.org>
Date: Wed Apr 04 05:04:38 2018

DNR: Impose limit on the max. no. of whitelisted page patterns per extension.

This CL imposes a limit on the maximum number of page patterns an extension can
whitelist using the chrome.declarativeNetRequest.addWhitelistedPages API. This
limit is currently set to 100. A new property MAX_NUMBER_OF_WHITELISTED_PAGES is
introduced in the declarativeNetRequest API for the same.

It is important to limit this since the whitelisted patterns are stored in the
browser memory (and also persisted across sessions using Extension preferences).

BUG= 811460 , 696822
Doc=https://docs.google.com/document/d/1Fhm-t0JCc3dcmwyBX_i7TGyuJ2QjqDechv1-VCRBo5o/edit?usp=sharing

Change-Id: I674da154107ac06269827272576f1af911f13338
Reviewed-on: https://chromium-review.googlesource.com/987392
Commit-Queue: Karan Bhatia <karandeepb@chromium.org>
Reviewed-by: Istiaque Ahmed <lazyboy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#547983}
[modify] https://crrev.com/72e12fdc05df67024f7adda2aa8319a33653261f/chrome/test/data/extensions/api_test/declarative_net_request/page_whitelisting_api/background.js
[modify] https://crrev.com/72e12fdc05df67024f7adda2aa8319a33653261f/extensions/browser/api/declarative_net_request/declarative_net_request_api.cc
[modify] https://crrev.com/72e12fdc05df67024f7adda2aa8319a33653261f/extensions/common/api/declarative_net_request.idl

Project Member

Comment 12 by bugdroid1@chromium.org, May 5 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e7ba8040dc25a423896c82428dd268b992cb100f

commit e7ba8040dc25a423896c82428dd268b992cb100f
Author: Karan Bhatia <karandeepb@chromium.org>
Date: Sat May 05 07:31:00 2018

DNR: Introduce helpers for the page whitelisting API in tests.

This CL introduces helpers for the page whitelisting API in the
DeclarativeNetRequestBrowserTest test fixture. Existing test code is modified to
use these helpers.

BUG= 811460 , 696822

Change-Id: I7f9ec32d2ec8edc17dc6bd546bb3d804390efbd8
Reviewed-on: https://chromium-review.googlesource.com/1039007
Commit-Queue: Karan Bhatia <karandeepb@chromium.org>
Reviewed-by: Istiaque Ahmed <lazyboy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#556321}
[modify] https://crrev.com/e7ba8040dc25a423896c82428dd268b992cb100f/chrome/browser/extensions/api/declarative_net_request/declarative_net_request_browsertest.cc

Project Member

Comment 13 by bugdroid1@chromium.org, May 31 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5ef1aba10cff27faa2724eb1bca6b01924ec5526

commit 5ef1aba10cff27faa2724eb1bca6b01924ec5526
Author: Karan Bhatia <karandeepb@chromium.org>
Date: Thu May 31 03:36:55 2018

DNR: Evaluate the set of whitelisted pages for network requests.

This is the final CL implementing the DNR page whitelisting API. This does the
following:

- The set of whitelisted pages for an extension are passed to the RulesetManager
  which lives on the IO thread. This happens whenever an extension is loaded
  (through RulesMonitorService) or when the set of whitelisted pages is updated
  (through a call to chrome.declarativeNetRequest.[add/remove]WhitelistedPages).
- For each pair of (network request, extension ruleset), RulesetManager checks
  if the request is whitelisted as per the page whitelisting API. This happens
  if the top level frame url corresponding to the request has been whitelisted
  by the extension using the page whitelisting API.

BUG= 811460 , 696822
Doc=https://docs.google.com/document/d/1Fhm-t0JCc3dcmwyBX_i7TGyuJ2QjqDechv1-VCRBo5o/edit?usp=sharing

Cq-Include-Trybots: master.tryserver.chromium.linux:linux_mojo
Change-Id: I910c5b481dc8f0b3283374a37c1351a7d7574372
Reviewed-on: https://chromium-review.googlesource.com/1041279
Commit-Queue: Karan Bhatia <karandeepb@chromium.org>
Reviewed-by: Devlin <rdevlin.cronin@chromium.org>
Reviewed-by: Istiaque Ahmed <lazyboy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#563141}
[modify] https://crrev.com/5ef1aba10cff27faa2724eb1bca6b01924ec5526/chrome/browser/extensions/api/declarative_net_request/declarative_net_request_browsertest.cc
[modify] https://crrev.com/5ef1aba10cff27faa2724eb1bca6b01924ec5526/chrome/browser/extensions/api/declarative_net_request/ruleset_manager_unittest.cc
[add] https://crrev.com/5ef1aba10cff27faa2724eb1bca6b01924ec5526/chrome/test/data/extensions/declarative_net_request/child_frame_with_subresources.html
[add] https://crrev.com/5ef1aba10cff27faa2724eb1bca6b01924ec5526/chrome/test/data/extensions/declarative_net_request/iframe_subresources/image.png
[add] https://crrev.com/5ef1aba10cff27faa2724eb1bca6b01924ec5526/chrome/test/data/extensions/declarative_net_request/iframe_subresources/ping.mp3
[add] https://crrev.com/5ef1aba10cff27faa2724eb1bca6b01924ec5526/chrome/test/data/extensions/declarative_net_request/iframe_subresources/script.js
[add] https://crrev.com/5ef1aba10cff27faa2724eb1bca6b01924ec5526/chrome/test/data/extensions/declarative_net_request/iframe_subresources/style.css
[add] https://crrev.com/5ef1aba10cff27faa2724eb1bca6b01924ec5526/chrome/test/data/extensions/declarative_net_request/iframe_subresources/subframe.html
[add] https://crrev.com/5ef1aba10cff27faa2724eb1bca6b01924ec5526/chrome/test/data/extensions/declarative_net_request/whitelisting_api.html
[modify] https://crrev.com/5ef1aba10cff27faa2724eb1bca6b01924ec5526/extensions/browser/api/declarative_net_request/declarative_net_request_api.cc
[modify] https://crrev.com/5ef1aba10cff27faa2724eb1bca6b01924ec5526/extensions/browser/api/declarative_net_request/declarative_net_request_api.h
[modify] https://crrev.com/5ef1aba10cff27faa2724eb1bca6b01924ec5526/extensions/browser/api/declarative_net_request/rules_monitor_service.cc
[modify] https://crrev.com/5ef1aba10cff27faa2724eb1bca6b01924ec5526/extensions/browser/api/declarative_net_request/ruleset_manager.cc
[modify] https://crrev.com/5ef1aba10cff27faa2724eb1bca6b01924ec5526/extensions/browser/api/declarative_net_request/ruleset_manager.h
[modify] https://crrev.com/5ef1aba10cff27faa2724eb1bca6b01924ec5526/extensions/browser/api/web_request/web_request_info.cc
[modify] https://crrev.com/5ef1aba10cff27faa2724eb1bca6b01924ec5526/testing/buildbot/filters/mash.browser_tests.filter
[modify] https://crrev.com/5ef1aba10cff27faa2724eb1bca6b01924ec5526/testing/buildbot/filters/mojo.fyi.network_browser_tests.filter

Project Member

Comment 14 by bugdroid1@chromium.org, Jun 8 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f3ae11fcae5502cf26cadf3224728db6345b0c28

commit f3ae11fcae5502cf26cadf3224728db6345b0c28
Author: Karan Bhatia <karandeepb@chromium.org>
Date: Fri Jun 08 00:06:41 2018

DNR: Add UMA for initiator checks performed by the page whitelisting API.

The Declarative Net Request Page Whitelisting API uses initiator checks to guess
the main frame url for a main-frame subresource request. This CL adds UMA to
enumerate how often the different cases occur. This should help reason about the
correctness of these checks.

BUG= 811460 

Change-Id: Id72c05041477d05825fbe7e0ff7fecbf39fda861
Reviewed-on: https://chromium-review.googlesource.com/1089475
Reviewed-by: Ilya Sherman <isherman@chromium.org>
Reviewed-by: Istiaque Ahmed <lazyboy@chromium.org>
Commit-Queue: Karan Bhatia <karandeepb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#565473}
[modify] https://crrev.com/f3ae11fcae5502cf26cadf3224728db6345b0c28/extensions/browser/api/declarative_net_request/ruleset_manager.cc
[modify] https://crrev.com/f3ae11fcae5502cf26cadf3224728db6345b0c28/tools/metrics/histograms/enums.xml
[modify] https://crrev.com/f3ae11fcae5502cf26cadf3224728db6345b0c28/tools/metrics/histograms/histograms.xml

Status: Fixed (was: Started)

Sign in to add a comment