Docs: https://docs.google.com/document/d/16_igCI15Gfzb6UYqeuJTmJPrzEtawz6Y1tVOKNtYgiU/edit#heading=h.7nki9mck5t64 https://docs.google.com/document/d/1tAOTGradXLdLvFi-M0aIh71h3fG49Z4J99KrkBMfZmM/edit#
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/3493a0df4536fb42b5273318db56e9fece2bc0fc commit 3493a0df4536fb42b5273318db56e9fece2bc0fc Author: Daniel Murphy <dmurph@chromium.org> Date: Tue Aug 07 23:04:54 2018 [LevelDBScopes] Creating lock manager This is the first component of LevelDB Scopes. The lock manager is a basic leveled lock system with shared and exclusive locks. This change contains an implementation which requires disjoint lock ranges, which is what the IndexedDB system will be using. This also includes a README with the implementation plan. This will be updated as more code is submitted. Bug: 862456 Change-Id: Ib4300ae8c59035c06772625323b3de0cc9ef3f6a Reviewed-on: https://chromium-review.googlesource.com/1134407 Commit-Queue: Daniel Murphy <dmurph@chromium.org> Reviewed-by: John Abd-El-Malek <jam@chromium.org> Reviewed-by: Victor Costan <pwnall@chromium.org> Cr-Commit-Position: refs/heads/master@{#581385} [modify] https://crrev.com/3493a0df4536fb42b5273318db56e9fece2bc0fc/content/browser/BUILD.gn [modify] https://crrev.com/3493a0df4536fb42b5273318db56e9fece2bc0fc/content/browser/dom_storage/storage_area_impl_unittest.cc [add] https://crrev.com/3493a0df4536fb42b5273318db56e9fece2bc0fc/content/browser/indexed_db/scopes/README.md [add] https://crrev.com/3493a0df4536fb42b5273318db56e9fece2bc0fc/content/browser/indexed_db/scopes/disjoint_range_lock_manager.cc [add] https://crrev.com/3493a0df4536fb42b5273318db56e9fece2bc0fc/content/browser/indexed_db/scopes/disjoint_range_lock_manager.h [add] https://crrev.com/3493a0df4536fb42b5273318db56e9fece2bc0fc/content/browser/indexed_db/scopes/disjoint_range_lock_manager_unittest.cc [add] https://crrev.com/3493a0df4536fb42b5273318db56e9fece2bc0fc/content/browser/indexed_db/scopes/scopes_lock_manager.cc [add] https://crrev.com/3493a0df4536fb42b5273318db56e9fece2bc0fc/content/browser/indexed_db/scopes/scopes_lock_manager.h [add] https://crrev.com/3493a0df4536fb42b5273318db56e9fece2bc0fc/content/browser/indexed_db/scopes/scopes_lock_manager_unittest.cc [modify] https://crrev.com/3493a0df4536fb42b5273318db56e9fece2bc0fc/content/test/BUILD.gn [add] https://crrev.com/3493a0df4536fb42b5273318db56e9fece2bc0fc/content/test/barrier_builder.cc [add] https://crrev.com/3493a0df4536fb42b5273318db56e9fece2bc0fc/content/test/barrier_builder.h
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/90b3bcf7e7e22e921647b09ab38abebfe1571574 commit 90b3bcf7e7e22e921647b09ab38abebfe1571574 Author: Victor Costan <pwnall@chromium.org> Date: Thu Aug 16 02:37:17 2018 Roll src/third_party/leveldatabase/src/ 186839815..f7b0e1d90 (3 commits) https://chromium.googlesource.com/external/leveldb.git/+log/18683981505d..f7b0e1d901da $ git log 186839815..f7b0e1d90 --date=short --no-merges --format='%ad %ae %s' 2018-08-14 costan Expose WriteBatch::Append(). 2018-06-04 costan Clean up Iterator. 2018-05-21 cmumford Corrected typo in docs: "cache" to "block_cache". Created with: roll-dep src/third_party/leveldatabase/src Bug: 862456 Change-Id: I8c618f5311d0f13fc59449be3c228b4aa63403ef Reviewed-on: https://chromium-review.googlesource.com/1174981 Commit-Queue: Daniel Murphy <dmurph@chromium.org> Reviewed-by: Daniel Murphy <dmurph@chromium.org> Cr-Commit-Position: refs/heads/master@{#583510} [modify] https://crrev.com/90b3bcf7e7e22e921647b09ab38abebfe1571574/DEPS
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/65a3081df8ebf3744ae00c4cd05d825f551c057c commit 65a3081df8ebf3744ae00c4cd05d825f551c057c Author: Daniel Murphy <dmurph@chromium.org> Date: Wed Dec 12 22:06:46 2018 [LevelDBScopes] Extract Range & Lock classes, add multi-level acquire. This patch * extracts ScopeLockRange and ScopeLock classes into separate files, * implements a multi-level lock acquisition algorithm to acquire a collection of locks together * clarifies the invariants of DisjointRangeLockManager Bug: 862456 Change-Id: I2702237dfa7e5213e7cd395d3c8fcfeeaf86898e Reviewed-on: https://chromium-review.googlesource.com/c/1258218 Commit-Queue: Daniel Murphy <dmurph@chromium.org> Reviewed-by: Victor Costan <pwnall@chromium.org> Reviewed-by: Chase Phillips <cmp@chromium.org> Cr-Commit-Position: refs/heads/master@{#616070} [modify] https://crrev.com/65a3081df8ebf3744ae00c4cd05d825f551c057c/content/browser/BUILD.gn [modify] https://crrev.com/65a3081df8ebf3744ae00c4cd05d825f551c057c/content/browser/indexed_db/scopes/disjoint_range_lock_manager.cc [modify] https://crrev.com/65a3081df8ebf3744ae00c4cd05d825f551c057c/content/browser/indexed_db/scopes/disjoint_range_lock_manager.h [modify] https://crrev.com/65a3081df8ebf3744ae00c4cd05d825f551c057c/content/browser/indexed_db/scopes/disjoint_range_lock_manager_unittest.cc [add] https://crrev.com/65a3081df8ebf3744ae00c4cd05d825f551c057c/content/browser/indexed_db/scopes/scope_lock.cc [add] https://crrev.com/65a3081df8ebf3744ae00c4cd05d825f551c057c/content/browser/indexed_db/scopes/scope_lock.h [add] https://crrev.com/65a3081df8ebf3744ae00c4cd05d825f551c057c/content/browser/indexed_db/scopes/scope_lock_range.cc [add] https://crrev.com/65a3081df8ebf3744ae00c4cd05d825f551c057c/content/browser/indexed_db/scopes/scope_lock_range.h [modify] https://crrev.com/65a3081df8ebf3744ae00c4cd05d825f551c057c/content/browser/indexed_db/scopes/scopes_lock_manager.cc [modify] https://crrev.com/65a3081df8ebf3744ae00c4cd05d825f551c057c/content/browser/indexed_db/scopes/scopes_lock_manager.h [modify] https://crrev.com/65a3081df8ebf3744ae00c4cd05d825f551c057c/content/browser/indexed_db/scopes/scopes_lock_manager_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/1a4a225edc5b98e3c8662c5f9cff13fce168d907 commit 1a4a225edc5b98e3c8662c5f9cff13fce168d907 Author: Daniel Murphy <dmurph@chromium.org> Date: Tue Jan 08 21:28:29 2019 [IndexedDB] Refactor & clarify leveldb database creation & ownership This refactors the way the LevelDB database is opened for IndexedDB. Classes like IndexedDBBackingStore and LevelDBDatabase now accept their dependencies on construction, instead of constructing/initializing their dependencies on creation. This allows for much easier testing and reasoning about state in the code. TBR: jam@chromium.org Bug: 862456 Change-Id: I4202bd1e75915ef7eed86e3a2cc0ad7d4fdc5d06 Reviewed-on: https://chromium-review.googlesource.com/c/1357015 Commit-Queue: Daniel Murphy <dmurph@chromium.org> Reviewed-by: Mark Pearson <mpearson@chromium.org> Reviewed-by: Victor Costan <pwnall@chromium.org> Reviewed-by: Chase Phillips <cmp@chromium.org> Cr-Commit-Position: refs/heads/master@{#620885} [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/BUILD.gn [add] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/docs/open_and_verify_leveldb_database.code2flow [add] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/docs/open_and_verify_leveldb_database.pdf [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_backing_store.cc [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_backing_store.h [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_backing_store_unittest.cc [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_cleanup_on_io_error_unittest.cc [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_context_impl.cc [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_context_impl.h [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_dispatcher_host_unittest.cc [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_factory.h [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_factory_impl.cc [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_factory_impl.h [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_factory_unittest.cc [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_fake_backing_store.cc [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_leveldb_coding.cc [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_leveldb_operations.cc [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_leveldb_operations.h [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_quota_client_unittest.cc [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_reporting.cc [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_reporting.h [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_tombstone_sweeper_unittest.cc [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/indexed_db_unittest.cc [add] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/leveldb/fake_leveldb_factory.cc [add] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/leveldb/fake_leveldb_factory.h [add] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/leveldb/leveldb_comparator.cc [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/leveldb/leveldb_comparator.h [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/leveldb/leveldb_database.cc [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/leveldb/leveldb_database.h [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/leveldb/leveldb_env.cc [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/leveldb/leveldb_env.h [add] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/leveldb/leveldb_env_unittest.cc [delete] https://crrev.com/06775e39722d5413e4a8fb8b1f31523602378425/content/browser/indexed_db/leveldb/leveldb_factory.h [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/leveldb/leveldb_transaction_unittest.cc [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/leveldb/leveldb_unittest.cc [delete] https://crrev.com/06775e39722d5413e4a8fb8b1f31523602378425/content/browser/indexed_db/leveldb/mock_leveldb_factory.cc [delete] https://crrev.com/06775e39722d5413e4a8fb8b1f31523602378425/content/browser/indexed_db/leveldb/mock_leveldb_factory.h [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/mock_indexed_db_factory.h [add] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/scopes/leveldb_state.cc [add] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/indexed_db/scopes/leveldb_state.h [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/browser/storage_partition_impl.cc [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/content/test/BUILD.gn [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/tools/metrics/histograms/enums.xml [modify] https://crrev.com/1a4a225edc5b98e3c8662c5f9cff13fce168d907/tools/metrics/histograms/histograms.xml
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/1ea0b73893618f670ab9e441f2813f4f226892ee commit 1ea0b73893618f670ab9e441f2813f4f226892ee Author: Daniel Murphy <dmurph@chromium.org> Date: Fri Jan 11 01:56:53 2019 [LevelDBScopes] Making all scopes lock ranges bytewise compared The locking system of LevelDB scopes doesn't need to match the key system of the backing leveldb database. Tieing them together means that the leveldb comparator has to handle both leveldb and lock keys, and these aren't necessarily the same. To avoid making the comparator slower for leveldb by adding more logic, make the ScopesLockManager locks always be bytewise-compared. See this doc for more context: https://docs.google.com/document/d/1jX6OOQJ0aLrWAeqYLtARXPFawUlJPbHNIKAK-0vmFHU/edit#heading=h.yifj0a13oqdj Bug: 862456 Change-Id: I56ec1f7328eadacda91c68a64047e0d84e458376 Reviewed-on: https://chromium-review.googlesource.com/c/1406288 Commit-Queue: Daniel Murphy <dmurph@chromium.org> Commit-Queue: Victor Costan <pwnall@chromium.org> Reviewed-by: Victor Costan <pwnall@chromium.org> Cr-Commit-Position: refs/heads/master@{#621860} [modify] https://crrev.com/1ea0b73893618f670ab9e441f2813f4f226892ee/content/browser/indexed_db/scopes/disjoint_range_lock_manager.cc [modify] https://crrev.com/1ea0b73893618f670ab9e441f2813f4f226892ee/content/browser/indexed_db/scopes/disjoint_range_lock_manager.h [modify] https://crrev.com/1ea0b73893618f670ab9e441f2813f4f226892ee/content/browser/indexed_db/scopes/disjoint_range_lock_manager_unittest.cc [modify] https://crrev.com/1ea0b73893618f670ab9e441f2813f4f226892ee/content/browser/indexed_db/scopes/scope_lock_range.cc [modify] https://crrev.com/1ea0b73893618f670ab9e441f2813f4f226892ee/content/browser/indexed_db/scopes/scope_lock_range.h [modify] https://crrev.com/1ea0b73893618f670ab9e441f2813f4f226892ee/content/browser/indexed_db/scopes/scopes_lock_manager.h
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a6d1c4f92e1d82c25dc3877be43808a400f6d39a commit a6d1c4f92e1d82c25dc3877be43808a400f6d39a Author: Daniel Murphy <dmurph@chromium.org> Date: Wed Jan 16 18:41:10 2019 [IndexedDB] Replacing TransactionCoordinator with ScopesLockManager This change removes the old TransactionCoordinator and uses the new DisjoinRangeLockManager from the scopes system. This is a prerequisite for using Scopes within IndexedDB, and replaces a pretty untested part of the system with a very well tested class. This change also means that the transaction scheduling will be subtly different. In the past, read-write transactions could actually start (and finish!) while an intersecting read-only transaction is executing. This is not the case in any other browser. Tracking bug to put this in the spec is here: https://crbug.com/921193 See here for more context: https://docs.google.com/document/d/1jX6OOQJ0aLrWAeqYLtARXPFawUlJPbHNIKAK-0vmFHU/edit#heading=h.5v1ggclt33i R=pwnall@chromium.org Bug: 862456 Change-Id: I41a116a3a2e9227eedf3a2337db21ff9c80a3e0b Reviewed-on: https://chromium-review.googlesource.com/c/1407783 Commit-Queue: Daniel Murphy <dmurph@chromium.org> Reviewed-by: Victor Costan <pwnall@chromium.org> Cr-Commit-Position: refs/heads/master@{#623307} [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/BUILD.gn [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/database_impl.cc [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/indexed_db_class_factory.cc [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/indexed_db_class_factory.h [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/indexed_db_connection.h [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/indexed_db_context_impl.cc [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/indexed_db_database.cc [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/indexed_db_database.h [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/indexed_db_database_unittest.cc [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/indexed_db_factory_impl.cc [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/indexed_db_factory_impl.h [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/indexed_db_leveldb_coding.cc [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/indexed_db_leveldb_coding.h [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/indexed_db_transaction.cc [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/indexed_db_transaction.h [delete] https://crrev.com/5253c803e3e60330a0f444fe61b97ac8fc82ff46/content/browser/indexed_db/indexed_db_transaction_coordinator.cc [delete] https://crrev.com/5253c803e3e60330a0f444fe61b97ac8fc82ff46/content/browser/indexed_db/indexed_db_transaction_coordinator.h [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/indexed_db_transaction_unittest.cc [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/mock_browsertest_indexed_db_class_factory.cc [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/mock_browsertest_indexed_db_class_factory.h [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/scopes/scope_lock_range.cc [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/scopes/scope_lock_range.h [modify] https://crrev.com/a6d1c4f92e1d82c25dc3877be43808a400f6d39a/content/browser/indexed_db/scopes/scopes_lock_manager_unittest.cc [delete] https://crrev.com/5253c803e3e60330a0f444fe61b97ac8fc82ff46/third_party/blink/web_tests/storage/indexeddb/resources/transaction-starvation.js [delete] https://crrev.com/5253c803e3e60330a0f444fe61b97ac8fc82ff46/third_party/blink/web_tests/storage/indexeddb/transaction-starvation-expected.txt [delete] https://crrev.com/5253c803e3e60330a0f444fe61b97ac8fc82ff46/third_party/blink/web_tests/storage/indexeddb/transaction-starvation.html
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b59f907772d869861a97bb2ade5553e2327b3555 commit b59f907772d869861a97bb2ade5553e2327b3555 Author: Victor Costan <pwnall@chromium.org> Date: Wed Jan 16 21:35:09 2019 Roll src/third_party/leveldatabase/src/ b70493ca8..fe4494804 (2 commits) https://chromium.googlesource.com/external/leveldb.git/+log/b70493ca8586..fe4494804f5e $ git log b70493ca8..fe4494804 --date=short --no-merges --format='%ad %ae %s' 2019-01-15 costan leveldb: Make WriteBatch::ApproximateSize() const. 2019-01-09 costan leveldb: Fix PosixWritableFile::Sync() on Apple systems. Created with: roll-dep src/third_party/leveldatabase/src Bug: 862456 Change-Id: I9b4d1b7c047bf67888e72f94a4b7834bef77e02f Reviewed-on: https://chromium-review.googlesource.com/c/1413793 Reviewed-by: Chris Mumford <cmumford@google.com> Commit-Queue: Victor Costan <pwnall@chromium.org> Cr-Commit-Position: refs/heads/master@{#623377} [modify] https://crrev.com/b59f907772d869861a97bb2ade5553e2327b3555/DEPS
Comment 1 by dmu...@chromium.org
, Jul 11Labels: -Type-Bug -Pri-3 Pri-1 Type-Feature