Enable Declarative Web Request API to use subresource_filter's data structure for storing a set/index of URL patterns and querying them.
Enable Declarative Net Request API to use subresource_filter's data structure for storing a set/index of URL patterns and querying them.
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7a70ae843511b3e6d4f8a3924904fd3f7d1e0cba commit 7a70ae843511b3e6d4f8a3924904fd3f7d1e0cba Author: pkalinnikov <pkalinnikov@chromium.org> Date: Fri May 05 10:19:36 2017 Factor out UrlPatternIndex from IndexedRuleset. This CL extracts the UrlPatternIndex class out of IndexedRuleset, and keeps the latter's implementation use the former as a building block. The new class can serve as a general-purpose tool for matching network requests against a set of URL rules. For example, it can be used by Declarative WebRequest API implementation. The follow-up step would be to cover UrlPatternIndex with tests and move it out from subresource_filter component to some generic place, e.g., components/url_matcher or components/url_pattern_index. BUG= 713774 Review-Url: https://codereview.chromium.org/2844293003 Cr-Commit-Position: refs/heads/master@{#469618} [modify] https://crrev.com/7a70ae843511b3e6d4f8a3924904fd3f7d1e0cba/components/subresource_filter/core/common/BUILD.gn [modify] https://crrev.com/7a70ae843511b3e6d4f8a3924904fd3f7d1e0cba/components/subresource_filter/core/common/PRESUBMIT.py [modify] https://crrev.com/7a70ae843511b3e6d4f8a3924904fd3f7d1e0cba/components/subresource_filter/core/common/document_subresource_filter_unittest.cc [modify] https://crrev.com/7a70ae843511b3e6d4f8a3924904fd3f7d1e0cba/components/subresource_filter/core/common/flat/BUILD.gn [add] https://crrev.com/7a70ae843511b3e6d4f8a3924904fd3f7d1e0cba/components/subresource_filter/core/common/flat/indexed_ruleset.fbs [rename] https://crrev.com/7a70ae843511b3e6d4f8a3924904fd3f7d1e0cba/components/subresource_filter/core/common/flat/url_pattern_index.fbs [modify] https://crrev.com/7a70ae843511b3e6d4f8a3924904fd3f7d1e0cba/components/subresource_filter/core/common/indexed_ruleset.cc [modify] https://crrev.com/7a70ae843511b3e6d4f8a3924904fd3f7d1e0cba/components/subresource_filter/core/common/indexed_ruleset.h [modify] https://crrev.com/7a70ae843511b3e6d4f8a3924904fd3f7d1e0cba/components/subresource_filter/core/common/indexed_ruleset_unittest.cc [modify] https://crrev.com/7a70ae843511b3e6d4f8a3924904fd3f7d1e0cba/components/subresource_filter/core/common/url_pattern.cc [add] https://crrev.com/7a70ae843511b3e6d4f8a3924904fd3f7d1e0cba/components/subresource_filter/core/common/url_pattern_index.cc [add] https://crrev.com/7a70ae843511b3e6d4f8a3924904fd3f7d1e0cba/components/subresource_filter/core/common/url_pattern_index.h
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4d49107dce95a5491acebfab01023a377ae03b58 commit 4d49107dce95a5491acebfab01023a377ae03b58 Author: pkalinnikov <pkalinnikov@chromium.org> Date: Fri May 05 11:14:34 2017 Clean up IndexedRuleset tests; factor out test_support. The new test_support header will be used by UrlPatternIndex tests when these get introduced. BUG= 713774 Review-Url: https://codereview.chromium.org/2865643002 Cr-Commit-Position: refs/heads/master@{#469624} [modify] https://crrev.com/4d49107dce95a5491acebfab01023a377ae03b58/components/subresource_filter/core/common/BUILD.gn [modify] https://crrev.com/4d49107dce95a5491acebfab01023a377ae03b58/components/subresource_filter/core/common/indexed_ruleset_unittest.cc [add] https://crrev.com/4d49107dce95a5491acebfab01023a377ae03b58/components/subresource_filter/core/common/url_rule_test_support.cc [add] https://crrev.com/4d49107dce95a5491acebfab01023a377ae03b58/components/subresource_filter/core/common/url_rule_test_support.h
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/09721eaab9e4d8dd62796f991da68dc2b89a325c commit 09721eaab9e4d8dd62796f991da68dc2b89a325c Author: pkalinnikov <pkalinnikov@chromium.org> Date: Fri May 05 15:15:56 2017 Move UrlPatternIndex-specific tests from indexed_ruleset_unittest. BUG= 713774 Review-Url: https://codereview.chromium.org/2862853002 Cr-Commit-Position: refs/heads/master@{#469653} [modify] https://crrev.com/09721eaab9e4d8dd62796f991da68dc2b89a325c/components/subresource_filter/core/common/BUILD.gn [modify] https://crrev.com/09721eaab9e4d8dd62796f991da68dc2b89a325c/components/subresource_filter/core/common/indexed_ruleset_unittest.cc [modify] https://crrev.com/09721eaab9e4d8dd62796f991da68dc2b89a325c/components/subresource_filter/core/common/url_pattern_index.cc [modify] https://crrev.com/09721eaab9e4d8dd62796f991da68dc2b89a325c/components/subresource_filter/core/common/url_pattern_index.h [add] https://crrev.com/09721eaab9e4d8dd62796f991da68dc2b89a325c/components/subresource_filter/core/common/url_pattern_index_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d27b3ffe038eba9c733337eccec5dae067118521 commit d27b3ffe038eba9c733337eccec5dae067118521 Author: pkalinnikov <pkalinnikov@chromium.org> Date: Mon May 08 09:47:14 2017 Add new UrlPatternIndex test. The new test verifies that FindMatch returns meaningful data. BUG= 713774 Review-Url: https://codereview.chromium.org/2861063004 Cr-Commit-Position: refs/heads/master@{#469941} [modify] https://crrev.com/d27b3ffe038eba9c733337eccec5dae067118521/components/subresource_filter/core/common/url_pattern_index_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/387b2a94017449569f0f5b2f865d3331e3646a3a commit 387b2a94017449569f0f5b2f865d3331e3646a3a Author: pkalinnikov <pkalinnikov@chromium.org> Date: Tue Jun 06 10:47:36 2017 Update FlatBuffers to include multiple improvements. This CL updates FlatBuffers to the latest version 1.6.x. This is necessary to include the following changes to the library: - d7ac378: Offset<T>::IsNull method. - 93c0960: --keep-prefix compiler option. - c559eb4: Generate files for empty schemas. - aaf5598: Fix path comparison on Windows. - fb87c0d: Improve allocator handling (fixes test). - 22743ca: Fix --keep-prefix bugs. - 642254b: Fix one more --keep-prefix issue. - dd05f32: Add tests for nested FlatBuffers. - 01c50d5: Get rid of move semantics in VectorIterator. BUG= 611351 , 713774 Review-Url: https://codereview.chromium.org/2883063002 Cr-Commit-Position: refs/heads/master@{#477254} [modify] https://crrev.com/387b2a94017449569f0f5b2f865d3331e3646a3a/DEPS [modify] https://crrev.com/387b2a94017449569f0f5b2f865d3331e3646a3a/third_party/flatbuffers/BUILD.gn [modify] https://crrev.com/387b2a94017449569f0f5b2f865d3331e3646a3a/third_party/flatbuffers/README.chromium [modify] https://crrev.com/387b2a94017449569f0f5b2f865d3331e3646a3a/third_party/flatbuffers/flatbuffer.gni
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/10a266981d83baf81c3eb24873c8732699ed087e commit 10a266981d83baf81c3eb24873c8732699ed087e Author: pkalinnikov <pkalinnikov@chromium.org> Date: Tue Jun 06 11:32:49 2017 Revert of Update FlatBuffers to include multiple improvements. (patchset #5 id:80001 of https://codereview.chromium.org/2883063002/ ) Reason for revert: Fails "Win" build: https://build.chromium.org/p/chromium/builders/Win/builds/55836 Original issue's description: > Update FlatBuffers to include multiple improvements. > > This CL updates FlatBuffers to the latest version 1.6.x. This is necessary to > include the following changes to the library: > - d7ac378: Offset<T>::IsNull method. > - 93c0960: --keep-prefix compiler option. > - c559eb4: Generate files for empty schemas. > - aaf5598: Fix path comparison on Windows. > - fb87c0d: Improve allocator handling (fixes test). > - 22743ca: Fix --keep-prefix bugs. > - 642254b: Fix one more --keep-prefix issue. > - dd05f32: Add tests for nested FlatBuffers. > - 01c50d5: Get rid of move semantics in VectorIterator. > > BUG= 611351 , 713774 > > Review-Url: https://codereview.chromium.org/2883063002 > Cr-Commit-Position: refs/heads/master@{#477254} > Committed: https://chromium.googlesource.com/chromium/src/+/387b2a94017449569f0f5b2f865d3331e3646a3a TBR=palmer@chromium.org,engedy@chromium.org # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true BUG= 611351 , 713774 Review-Url: https://codereview.chromium.org/2926563002 Cr-Commit-Position: refs/heads/master@{#477258} [modify] https://crrev.com/10a266981d83baf81c3eb24873c8732699ed087e/DEPS [modify] https://crrev.com/10a266981d83baf81c3eb24873c8732699ed087e/third_party/flatbuffers/BUILD.gn [modify] https://crrev.com/10a266981d83baf81c3eb24873c8732699ed087e/third_party/flatbuffers/README.chromium [modify] https://crrev.com/10a266981d83baf81c3eb24873c8732699ed087e/third_party/flatbuffers/flatbuffer.gni
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/488ae5249bcea6c46af5e832f8be87dcc8df56e1 commit 488ae5249bcea6c46af5e832f8be87dcc8df56e1 Author: pkalinnikov <pkalinnikov@chromium.org> Date: Thu Jun 08 12:09:44 2017 Reland: Update FlatBuffers to include multiple improvements. This CL updates FlatBuffers to the latest version 1.6.x. This is necessary to include the following changes to the library: - d7ac378: Offset<T>::IsNull method. - 93c0960: --keep-prefix compiler option. - c559eb4: Generate files for empty schemas. - aaf5598: Fix path comparison on Windows. - fb87c0d: Improve allocator handling (fixes test). - 22743ca: Fix --keep-prefix bugs. - 642254b: Fix one more --keep-prefix issue. - dd05f32: Add tests for nested FlatBuffers. - 01c50d5: Get rid of move semantics in VectorIterator. This is a reland of https://codereview.chromium.org/2883063002/ with a compile flag added to Windows build suppressing the alignment warning. BUG= 611351 , 713774 TBR=palmer@chromium.org Review-Url: https://codereview.chromium.org/2923203002 Cr-Commit-Position: refs/heads/master@{#477947} [modify] https://crrev.com/488ae5249bcea6c46af5e832f8be87dcc8df56e1/DEPS [modify] https://crrev.com/488ae5249bcea6c46af5e832f8be87dcc8df56e1/third_party/flatbuffers/BUILD.gn [modify] https://crrev.com/488ae5249bcea6c46af5e832f8be87dcc8df56e1/third_party/flatbuffers/README.chromium [modify] https://crrev.com/488ae5249bcea6c46af5e832f8be87dcc8df56e1/third_party/flatbuffers/flatbuffer.gni
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7975d5b8c41a7f2239f3cf9f537549795a74a815 commit 7975d5b8c41a7f2239f3cf9f537549795a74a815 Author: pkalinnikov <pkalinnikov@chromium.org> Date: Fri Jun 09 00:10:45 2017 [flatbuffers] Enable including by absolute path. BUG= 713774 Review-Url: https://codereview.chromium.org/2928573002 Cr-Commit-Position: refs/heads/master@{#478130} [modify] https://crrev.com/7975d5b8c41a7f2239f3cf9f537549795a74a815/components/subresource_filter/core/common/flat/indexed_ruleset.fbs [modify] https://crrev.com/7975d5b8c41a7f2239f3cf9f537549795a74a815/third_party/flatbuffers/flatbuffer.gni
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d797063ae13856497560195cc6a7cf5dcd7c2667 commit d797063ae13856497560195cc6a7cf5dcd7c2667 Author: Pavel Kalinnikov <pkalinnikov@chromium.org> Date: Wed Jun 21 23:49:46 2017 Create a separate component for UrlPatternIndex. Bug: 713774 Change-Id: I12a70bc0b5caa37470ecf53568d90304c05095eb Reviewed-on: https://chromium-review.googlesource.com/527445 Commit-Queue: Pavel Kalinnikov <pkalinnikov@chromium.org> Reviewed-by: Jochen Eisinger <jochen@chromium.org> Reviewed-by: Charlie Harrison <csharrison@chromium.org> Cr-Commit-Position: refs/heads/master@{#481360} [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/chrome/browser/subresource_filter/subresource_filter_browsertest.cc [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/BUILD.gn [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/DEPS [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/content/browser/activation_state_computing_navigation_throttle_unittest.cc [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/content/browser/async_document_subresource_filter.cc [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/content/browser/async_document_subresource_filter_unittest.cc [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/content/browser/content_subresource_filter_throttle_manager_unittest.cc [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/content/browser/subresource_filter_safe_browsing_activation_throttle_unittest.cc [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/content/renderer/web_document_subresource_filter_impl.cc [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/content/renderer/web_document_subresource_filter_impl.h [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/core/browser/BUILD.gn [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/core/browser/ruleset_service.cc [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/core/browser/ruleset_service_unittest.cc [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/core/common/BUILD.gn [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/core/common/PRESUBMIT.py [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/core/common/document_subresource_filter.cc [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/core/common/document_subresource_filter.h [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/core/common/document_subresource_filter_unittest.cc [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/core/common/flat/BUILD.gn [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/core/common/flat/indexed_ruleset.fbs [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/core/common/indexed_ruleset.cc [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/core/common/indexed_ruleset.h [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/core/common/indexed_ruleset_unittest.cc [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/core/common/test_ruleset_creator.cc [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/core/common/test_ruleset_creator.h [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/core/common/test_ruleset_utils.cc [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/subresource_filter/core/common/test_ruleset_utils.h [add] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/BUILD.gn [add] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/DEPS [add] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/OWNERS [add] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/README [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/closed_hash_map.h [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/closed_hash_map_unittest.cc [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/copying_file_stream.cc [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/copying_file_stream.h [copy] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/flat/BUILD.gn [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/flat/url_pattern_index.fbs [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/fuzzy_pattern_matching.cc [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/fuzzy_pattern_matching.h [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/fuzzy_pattern_matching_unittest.cc [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/ngram_extractor.h [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/ngram_extractor_unittest.cc [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/proto/BUILD.gn [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/proto/rules.proto [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/string_splitter.h [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/string_splitter_unittest.cc [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/uint64_hasher.h [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/unindexed_ruleset.cc [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/unindexed_ruleset.h [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/unindexed_ruleset_unittest.cc [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/url_pattern.cc [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/url_pattern.h [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/url_pattern_index.cc [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/url_pattern_index.h [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/url_pattern_index_unittest.cc [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/url_pattern_unittest.cc [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/url_rule_test_support.cc [rename] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/components/url_pattern_index/url_rule_test_support.h [modify] https://crrev.com/d797063ae13856497560195cc6a7cf5dcd7c2667/third_party/flatbuffers/flatbuffer.gni
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d712e89aba395a568ba802bd25fe42bbf54ceccd commit d712e89aba395a568ba802bd25fe42bbf54ceccd Author: Karan Bhatia <karandeepb@chromium.org> Date: Wed Jul 05 21:07:02 2017 UrlPatternIndex: Introduce ElementType and ActivationType enums to url_pattern_index schema. Currently, the UrlPatternIndex flatbuffer structure stores the element types and activation types as bitmasks of the corresponding proto types. This CL introduces ElementType and ActivationType enums to the UrlPatternIndex flatbuffer schema, hence removing the dependence on the corresponding proto types. This should allow the url_pattern_index clients to work without a proto dependency. BUG= 713774 Change-Id: I95e16b85dd9c15bc8cbf5aeced82224900fb384c Reviewed-on: https://chromium-review.googlesource.com/554138 Commit-Queue: Karan Bhatia <karandeepb@chromium.org> Reviewed-by: Pavel Kalinnikov <pkalinnikov@chromium.org> Reviewed-by: Charlie Harrison <csharrison@chromium.org> Cr-Commit-Position: refs/heads/master@{#484349} [modify] https://crrev.com/d712e89aba395a568ba802bd25fe42bbf54ceccd/components/subresource_filter/core/common/indexed_ruleset.cc [modify] https://crrev.com/d712e89aba395a568ba802bd25fe42bbf54ceccd/components/url_pattern_index/flat/url_pattern_index.fbs [modify] https://crrev.com/d712e89aba395a568ba802bd25fe42bbf54ceccd/components/url_pattern_index/url_pattern_index.cc [modify] https://crrev.com/d712e89aba395a568ba802bd25fe42bbf54ceccd/components/url_pattern_index/url_pattern_index.h
Karan, I am reassigning this to you because I don't work on Chrome anymore.
Closing this, URLPatternIndex is being shared b/w subresouce_filter and Declarative Net Request.
Comment 1 by pkalinnikov@chromium.org
, Apr 20 2017