The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a5ae4fabb7a3bbe5871cd7704edf947067c08204

commit a5ae4fabb7a3bbe5871cd7704edf947067c08204
Author: Istiaque Ahmed <lazyboy@chromium.org>
Date: Fri Jan 19 04:27:29 2018

Refactor content verification hashes into ContentHash class.

There are two hash files that are involved for content
verification: verified_contents.json and computed_hashes.json.
The new class aims to pull all operations related to these
files into ContentHash to improve content verification code
and also facilitates further improvements to the code.

This CL moves both hash files' reading code into ContentHash
class avoiding the need to read verified_contents.json from
multiple places. This CL also moves computed_hashes.json file
generation/writing to ContentHash. The Cancel() behavior of
ContentHashFetcherJob is retained in this CL, and is moved
primarily to ContentHash. ContentHash uses CancellationStatus
interface to consult ContentHashFetcherJob about cross thread
cancellation of ContentHashFetcherJob.

Update ContentHashReader and ContentHashFetcher accordingly
to use ContentHash to deal with the hashes. Fetching
verified_contents.json from network is still kept in
ContentHashFetcher to keep the change simpler.

Behavior changes:
We used to write computed_hashes.json after computing hashes but
didn't read that file at that step, this CL reads them back to
ContentHash::Reader right after writing it. This avoids special
cases and makes ContentHash always have both hashes ready. Other
than that, expect no behavior change in this CL.

Bug: 796395
Change-Id: I00e03cf208982e1cbd8be7951e06da97173a3ffc
Reviewed-on: https://chromium-review.googlesource.com/831341
Commit-Queue: Istiaque Ahmed <lazyboy@chromium.org>
Reviewed-by: Devlin <rdevlin.cronin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#530422}
[modify] https://crrev.com/a5ae4fabb7a3bbe5871cd7704edf947067c08204/extensions/browser/BUILD.gn
[modify] https://crrev.com/a5ae4fabb7a3bbe5871cd7704edf947067c08204/extensions/browser/content_hash_fetcher.cc
[modify] https://crrev.com/a5ae4fabb7a3bbe5871cd7704edf947067c08204/extensions/browser/content_hash_reader.cc
[add] https://crrev.com/a5ae4fabb7a3bbe5871cd7704edf947067c08204/extensions/browser/content_verifier/content_hash.cc
[add] https://crrev.com/a5ae4fabb7a3bbe5871cd7704edf947067c08204/extensions/browser/content_verifier/content_hash.h

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/72816eaa19b0eca08b6ccbfd642148690a55922a

commit 72816eaa19b0eca08b6ccbfd642148690a55922a
Author: Istiaque Ahmed <lazyboy@chromium.org>
Date: Tue Jan 30 22:37:06 2018

Remove ContentVerifyJob::TestDelegate.

The delegate used to create separate code paths in ContentVerifyJob,
which makes the code harder to read and also creates a parallel
reality in tests. Remove it entirely since tests using the delegate
can work with the TestObserver in ContentVerifyJob.

Remove two tests: ContentVerifierTest:FailOnRead/FailOnDone, that
heavily used the TestDelegate. These tests were *artificially*
(different from how production code does) creating error conditions
from ContentVerifyJob::BytesRead and ContentVerifyJob::DoneReading.

Extract test utilities related to content verification to a
separate file (e/b/content_verifier/test_utils.h)

Bug: 796395
Change-Id: Ib59e9f0c7a4aac9b6a883d7f42647cecc603fa79
Reviewed-on: https://chromium-review.googlesource.com/888327
Commit-Queue: Istiaque Ahmed <lazyboy@chromium.org>
Reviewed-by: Devlin <rdevlin.cronin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#533050}
[modify] https://crrev.com/72816eaa19b0eca08b6ccbfd642148690a55922a/chrome/browser/extensions/content_verifier_browsertest.cc
[modify] https://crrev.com/72816eaa19b0eca08b6ccbfd642148690a55922a/chrome/browser/extensions/extension_protocols_unittest.cc
[modify] https://crrev.com/72816eaa19b0eca08b6ccbfd642148690a55922a/extensions/BUILD.gn
[add] https://crrev.com/72816eaa19b0eca08b6ccbfd642148690a55922a/extensions/browser/content_verifier/test_utils.cc
[add] https://crrev.com/72816eaa19b0eca08b6ccbfd642148690a55922a/extensions/browser/content_verifier/test_utils.h
[modify] https://crrev.com/72816eaa19b0eca08b6ccbfd642148690a55922a/extensions/browser/content_verify_job.cc
[modify] https://crrev.com/72816eaa19b0eca08b6ccbfd642148690a55922a/extensions/browser/content_verify_job.h
[modify] https://crrev.com/72816eaa19b0eca08b6ccbfd642148690a55922a/extensions/browser/content_verify_job_unittest.cc
[modify] https://crrev.com/72816eaa19b0eca08b6ccbfd642148690a55922a/extensions/browser/scoped_ignore_content_verifier_for_test.cc
[modify] https://crrev.com/72816eaa19b0eca08b6ccbfd642148690a55922a/extensions/browser/scoped_ignore_content_verifier_for_test.h

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1dec1d48f30d478251a971b2b5c26bf63ca80301

commit 1dec1d48f30d478251a971b2b5c26bf63ca80301
Author: Istiaque Ahmed <lazyboy@chromium.org>
Date: Fri Feb 02 00:01:36 2018

Add tests to cover the ordering of asynchronous steps in ContentVerifyJob.

ContentVerifyJob has two steps that can run independently of each
other: 1. Reading content hash (OnHashesReady) and 2. Receiving content
bytes (BytesReady and DoneReading). Add tests to make sure the ordering
of those two steps doesn't matter (and avoid future regressions around
this).
Expand existing tests to run these steps without any ordering
guarantee (as is), with step #1 completing before step #2 runs
and vice versa.

Bug: 796395
Test: None
Change-Id: I4c3d9597b030dcf36626feb1bec251aab61bf069
Reviewed-on: https://chromium-review.googlesource.com/891656
Reviewed-by: Devlin <rdevlin.cronin@chromium.org>
Commit-Queue: Istiaque Ahmed <lazyboy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#533857}
[modify] https://crrev.com/1dec1d48f30d478251a971b2b5c26bf63ca80301/chrome/browser/extensions/content_verifier_browsertest.cc
[modify] https://crrev.com/1dec1d48f30d478251a971b2b5c26bf63ca80301/chrome/browser/extensions/extension_protocols_unittest.cc
[modify] https://crrev.com/1dec1d48f30d478251a971b2b5c26bf63ca80301/extensions/browser/content_verifier/test_utils.cc
[modify] https://crrev.com/1dec1d48f30d478251a971b2b5c26bf63ca80301/extensions/browser/content_verifier/test_utils.h
[modify] https://crrev.com/1dec1d48f30d478251a971b2b5c26bf63ca80301/extensions/browser/content_verify_job.cc
[modify] https://crrev.com/1dec1d48f30d478251a971b2b5c26bf63ca80301/extensions/browser/content_verify_job.h
[modify] https://crrev.com/1dec1d48f30d478251a971b2b5c26bf63ca80301/extensions/browser/content_verify_job_unittest.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c16191af09e394cebaa1b551352e5c4c889b0c4f

commit c16191af09e394cebaa1b551352e5c4c889b0c4f
Author: Istiaque Ahmed <lazyboy@chromium.org>
Date: Fri Feb 16 18:51:31 2018

Extract ContentVerifierKey to its own file.

This removes the necessity of including content_verifier_delegate.h
where only ContentVerifierKey would be required.

Bug: 796395
Change-Id: Icf18f2957291c454c6e8a6971f98be7f9fd2fb78
Reviewed-on: https://chromium-review.googlesource.com/922925
Commit-Queue: Istiaque Ahmed <lazyboy@chromium.org>
Reviewed-by: Karan Bhatia <karandeepb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#537363}
[modify] https://crrev.com/c16191af09e394cebaa1b551352e5c4c889b0c4f/extensions/browser/BUILD.gn
[modify] https://crrev.com/c16191af09e394cebaa1b551352e5c4c889b0c4f/extensions/browser/content_verifier/content_hash.h
[add] https://crrev.com/c16191af09e394cebaa1b551352e5c4c889b0c4f/extensions/browser/content_verifier/content_verifier_key.h
[modify] https://crrev.com/c16191af09e394cebaa1b551352e5c4c889b0c4f/extensions/browser/content_verifier_delegate.h

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/545e59b4d718e55d2fa80a0a6c104de9236a0eda

commit 545e59b4d718e55d2fa80a0a6c104de9236a0eda
Author: Istiaque Ahmed <lazyboy@chromium.org>
Date: Fri Feb 23 02:52:34 2018

Move mock ContentVerifierDelegate to test_utils.

This will be used in other tests in the future, so extract it to
test_utils.

Bug: 796395
Change-Id: Iddc6a764d56acdc41b65a4ca1424f1ad124d8f1a
Reviewed-on: https://chromium-review.googlesource.com/932910
Reviewed-by: Devlin <rdevlin.cronin@chromium.org>
Commit-Queue: Istiaque Ahmed <lazyboy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#538677}
[modify] https://crrev.com/545e59b4d718e55d2fa80a0a6c104de9236a0eda/extensions/browser/content_hash_fetcher_unittest.cc
[modify] https://crrev.com/545e59b4d718e55d2fa80a0a6c104de9236a0eda/extensions/browser/content_verifier/test_utils.cc
[modify] https://crrev.com/545e59b4d718e55d2fa80a0a6c104de9236a0eda/extensions/browser/content_verifier/test_utils.h

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e63b37066f5016fa3b8b2d0b56a905a56f65f80c

commit e63b37066f5016fa3b8b2d0b56a905a56f65f80c
Author: Istiaque Ahmed <lazyboy@chromium.org>
Date: Sat Feb 24 05:10:06 2018

Simplify ContentHashReader lifetime and creation.

Instead of creating CHR and then calling CHR::Init(), create it
in one go (in a blocking allowed thread). The instance is immuatable and
stored inside ContentVerifyJob, hence CHR doens't need to be
RefCountedThreadSafe anymore.

ContentHashReader only needs extension related information on creation,
so move all such params from the class members (to ContentVerifyJob).

Bug: 796395
Change-Id: Ia65938d96e99d5c95562c75920cd614935026240
Reviewed-on: https://chromium-review.googlesource.com/917007
Reviewed-by: Istiaque Ahmed <lazyboy@chromium.org>
Reviewed-by: Devlin <rdevlin.cronin@chromium.org>
Commit-Queue: Istiaque Ahmed <lazyboy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#538994}
[modify] https://crrev.com/e63b37066f5016fa3b8b2d0b56a905a56f65f80c/extensions/browser/content_hash_reader.cc
[modify] https://crrev.com/e63b37066f5016fa3b8b2d0b56a905a56f65f80c/extensions/browser/content_hash_reader.h
[modify] https://crrev.com/e63b37066f5016fa3b8b2d0b56a905a56f65f80c/extensions/browser/content_verifier.cc
[modify] https://crrev.com/e63b37066f5016fa3b8b2d0b56a905a56f65f80c/extensions/browser/content_verify_job.cc
[modify] https://crrev.com/e63b37066f5016fa3b8b2d0b56a905a56f65f80c/extensions/browser/content_verify_job.h
[modify] https://crrev.com/e63b37066f5016fa3b8b2d0b56a905a56f65f80c/extensions/browser/content_verify_job_unittest.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4dacae9837240e8c9f74f7db10662cb8eab5bba3

commit 4dacae9837240e8c9f74f7db10662cb8eab5bba3
Author: Istiaque Ahmed <lazyboy@chromium.org>
Date: Sat Mar 03 07:01:42 2018

Add two tests for content hash fetch failure.

This CL adds two tests to exercise scenarios where verified_contents.json fetch
can fail. The tests make sure that the fetch fails and ContentHashFetcher
responds correctly.

FetchInvalidVerifiedContents: Fetched hash contains invalid signature content.
Fetch404VerifiedContents: Hash fetch results in 404.

Bug: 818315, 796395
Change-Id: I3606bf504bf1e710705d2020bdf0ac3afa84d77a
Reviewed-on: https://chromium-review.googlesource.com/942182
Commit-Queue: Istiaque Ahmed <lazyboy@chromium.org>
Reviewed-by: Devlin <rdevlin.cronin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#540747}
[modify] https://crrev.com/4dacae9837240e8c9f74f7db10662cb8eab5bba3/extensions/browser/content_hash_fetcher_unittest.cc
[modify] https://crrev.com/4dacae9837240e8c9f74f7db10662cb8eab5bba3/extensions/test/data/content_hash_fetcher/missing_verified_contents/README.txt
[add] https://crrev.com/4dacae9837240e8c9f74f7db10662cb8eab5bba3/extensions/test/data/content_hash_fetcher/missing_verified_contents/invalid_verified_contents.json

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/584fe1448a33b2df382d910d0c2e156836099f6b

commit 584fe1448a33b2df382d910d0c2e156836099f6b
Author: Istiaque Ahmed <lazyboy@chromium.org>
Date: Tue Mar 13 09:19:04 2018

Add rigorous tests for content hash fetch behavior.

For better or worse, there are lots of subtleties around content hash
fetching code and how computed hashes are handled. This CL aims to
document those with tests.

The intention is to make sure that these don't regress unknowingly
while code is refactored in the future.

Bug: 796395
Change-Id: Iecc106a992596955497a1d02b6610c85c1d64364
Reviewed-on: https://chromium-review.googlesource.com/947712
Commit-Queue: Istiaque Ahmed <lazyboy@chromium.org>
Reviewed-by: Devlin <rdevlin.cronin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#542748}
[modify] https://crrev.com/584fe1448a33b2df382d910d0c2e156836099f6b/chrome/browser/extensions/content_verifier_browsertest.cc
[add] https://crrev.com/584fe1448a33b2df382d910d0c2e156836099f6b/chrome/browser/extensions/content_verifier_hash_fetch_behavior_browsertest.cc
[modify] https://crrev.com/584fe1448a33b2df382d910d0c2e156836099f6b/chrome/browser/extensions/extension_protocols_unittest.cc
[modify] https://crrev.com/584fe1448a33b2df382d910d0c2e156836099f6b/chrome/test/BUILD.gn
[add] https://crrev.com/584fe1448a33b2df382d910d0c2e156836099f6b/chrome/test/data/extensions/content_verifier/has_default_resource.crx
[add] https://crrev.com/584fe1448a33b2df382d910d0c2e156836099f6b/chrome/test/data/extensions/content_verifier/has_default_resource.crx.INFO
[add] https://crrev.com/584fe1448a33b2df382d910d0c2e156836099f6b/chrome/test/data/extensions/content_verifier/no_default_resources.crx
[add] https://crrev.com/584fe1448a33b2df382d910d0c2e156836099f6b/chrome/test/data/extensions/content_verifier/no_default_resources.crx.INFO
[modify] https://crrev.com/584fe1448a33b2df382d910d0c2e156836099f6b/extensions/browser/content_verifier.cc
[modify] https://crrev.com/584fe1448a33b2df382d910d0c2e156836099f6b/extensions/browser/content_verifier.h
[modify] https://crrev.com/584fe1448a33b2df382d910d0c2e156836099f6b/extensions/browser/content_verifier/test_utils.cc
[modify] https://crrev.com/584fe1448a33b2df382d910d0c2e156836099f6b/extensions/browser/content_verifier/test_utils.h
[modify] https://crrev.com/584fe1448a33b2df382d910d0c2e156836099f6b/extensions/browser/content_verify_job_unittest.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3641f523a0f6e38b2de6d28b1366c7f44d6a9a42

commit 3641f523a0f6e38b2de6d28b1366c7f44d6a9a42
Author: Istiaque Ahmed <lazyboy@chromium.org>
Date: Fri Mar 23 23:27:41 2018

Always pass ContentHash creation through IO thread.

This will allow ContentHash caching to be (trivially)
implemented in the future in ContentVerifier::GetContentHash().
This CL also makes extension load at startup to queue all inflight
requests to load verified_contents and computed_hashes reading
from disk. This is because inflight requests will be served by
single file read.

Summary of changes:
The primary method is ContentVerifier::GetContentHash(). This
method is responsible for retrieving ContentHash on IO thread. It
uses ContentVerifier::HashHelper to manage thread hops and such.
HashHelper is not thread safe and it uses static method + weak
pointers to safely call ContentHash methods on blocking thread.

ContentHash class is responsible for fetching verified_contents.json
from network, using ContentHashFetcher. ContentHashFetcher
manages its own lifetime and is no longer ref-counted. Also,
ContentHashFetcherJob is no longer required.
ContentHashFetcher is currently an implementation detail of
ContentHash, and can be further restricted so that it cannot be
instantiated outside of ContentHash.
Because of async nature requirement of net fetch, ContentHash::Create
is now async.

Since ContentHash instance can travel between sequences, they are
RefCountedThreadSafe now.

ContentVerifier::VerifyFailed used to be (also) called when we fail to
retrieve hashes on disk and it would fall back to ContentHashFetcher
for fetching the hashes. As GetContentHash currently takes care of
this through ContentHash::Create, ContentVerifier is only made aware
of failures on when hash mismatch occurs, simplifying things further.

Bug: 796395
Change-Id: I7eb6bf93daba908258ae604a8cd45ea643b4e00d
Reviewed-on: https://chromium-review.googlesource.com/933863
Commit-Queue: Istiaque Ahmed <lazyboy@chromium.org>
Reviewed-by: Devlin <rdevlin.cronin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#545623}
[modify] https://crrev.com/3641f523a0f6e38b2de6d28b1366c7f44d6a9a42/chrome/browser/extensions/content_verifier_browsertest.cc
[modify] https://crrev.com/3641f523a0f6e38b2de6d28b1366c7f44d6a9a42/extensions/browser/content_hash_fetcher.cc
[modify] https://crrev.com/3641f523a0f6e38b2de6d28b1366c7f44d6a9a42/extensions/browser/content_hash_fetcher.h
[modify] https://crrev.com/3641f523a0f6e38b2de6d28b1366c7f44d6a9a42/extensions/browser/content_hash_fetcher_unittest.cc
[modify] https://crrev.com/3641f523a0f6e38b2de6d28b1366c7f44d6a9a42/extensions/browser/content_hash_reader.cc
[modify] https://crrev.com/3641f523a0f6e38b2de6d28b1366c7f44d6a9a42/extensions/browser/content_hash_reader.h
[modify] https://crrev.com/3641f523a0f6e38b2de6d28b1366c7f44d6a9a42/extensions/browser/content_verifier.cc
[modify] https://crrev.com/3641f523a0f6e38b2de6d28b1366c7f44d6a9a42/extensions/browser/content_verifier.h
[modify] https://crrev.com/3641f523a0f6e38b2de6d28b1366c7f44d6a9a42/extensions/browser/content_verifier/content_hash.cc
[modify] https://crrev.com/3641f523a0f6e38b2de6d28b1366c7f44d6a9a42/extensions/browser/content_verifier/content_hash.h
[modify] https://crrev.com/3641f523a0f6e38b2de6d28b1366c7f44d6a9a42/extensions/browser/content_verifier/test_utils.cc
[modify] https://crrev.com/3641f523a0f6e38b2de6d28b1366c7f44d6a9a42/extensions/browser/content_verifier/test_utils.h
[modify] https://crrev.com/3641f523a0f6e38b2de6d28b1366c7f44d6a9a42/extensions/browser/content_verify_job.cc
[modify] https://crrev.com/3641f523a0f6e38b2de6d28b1366c7f44d6a9a42/extensions/browser/content_verify_job.h
[modify] https://crrev.com/3641f523a0f6e38b2de6d28b1366c7f44d6a9a42/extensions/browser/content_verify_job_unittest.cc
[modify] https://crrev.com/3641f523a0f6e38b2de6d28b1366c7f44d6a9a42/extensions/browser/extension_protocols.cc
[modify] https://crrev.com/3641f523a0f6e38b2de6d28b1366c7f44d6a9a42/extensions/browser/extension_user_script_loader.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b1091b415d1a4965ce112aa900e04d38252f9bf8

commit b1091b415d1a4965ce112aa900e04d38252f9bf8
Author: Dominic Battre <battre@chromium.org>
Date: Mon Mar 26 13:53:07 2018

Disable flaky ContentVerifierTest.DotSlashPaths

See  crbug.com/825540  for details.

TBR=lazyboy@chromium.org,rdevlin.cronin@chromium.org
notry=true

Bug:  825540 ,796395
Change-Id: I70e628fd0b5f7fc98358be225ef0fd856a155359
Reviewed-on: https://chromium-review.googlesource.com/980256
Reviewed-by: Dominic Battré <battre@chromium.org>
Commit-Queue: Dominic Battré <battre@chromium.org>
Cr-Commit-Position: refs/heads/master@{#545778}
[modify] https://crrev.com/b1091b415d1a4965ce112aa900e04d38252f9bf8/chrome/browser/extensions/content_verifier_browsertest.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/51f716d8f187f5b31d332054249de792c1ff81c8

commit 51f716d8f187f5b31d332054249de792c1ff81c8
Author: Istiaque Ahmed <lazyboy@chromium.org>
Date: Tue Nov 06 23:32:18 2018

Implement ContentHash caching.

Add caching for ContentHash instances for a particular version of
an extension. Since ContentHash is required for all extension
resource requests when content verification applies, this should
speed up ContentVerifyJob.

Note that we should expect gains in
"ExtensionContentVerifyJob.TimeSpentUS" UMA.

Bug: 796395
Test: No visible behavior changes expected.
Change-Id: I4d03bcd95f00215628eb9b3f143e15e7bd2c84d5
Reviewed-on: https://chromium-review.googlesource.com/c/1266604
Commit-Queue: Istiaque Ahmed <lazyboy@chromium.org>
Reviewed-by: Devlin <rdevlin.cronin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#605875}
[modify] https://crrev.com/51f716d8f187f5b31d332054249de792c1ff81c8/extensions/browser/content_hash_fetcher_unittest.cc
[modify] https://crrev.com/51f716d8f187f5b31d332054249de792c1ff81c8/extensions/browser/content_verifier.cc
[modify] https://crrev.com/51f716d8f187f5b31d332054249de792c1ff81c8/extensions/browser/content_verifier.h
[modify] https://crrev.com/51f716d8f187f5b31d332054249de792c1ff81c8/extensions/browser/content_verifier/content_hash.h
[modify] https://crrev.com/51f716d8f187f5b31d332054249de792c1ff81c8/extensions/browser/content_verify_job.cc
[modify] https://crrev.com/51f716d8f187f5b31d332054249de792c1ff81c8/extensions/browser/content_verify_job.h