Find alternative to Meta for the sql::Connection mmap code to store status. |
|||||||
Issue descriptionsql::Connection does some evaluation before enabling mmap on a database, because there are various broken results which can occur with corrupt systems (see issue 489784). It stores status info in the Meta table so that it only have to evaluate once. Some databases have no Meta table, so they disable mmap. These sub-systems know who they are, they could set an alternative place to store the data.
,
Oct 10 2016
Status: components/sync/syncable/directory_backing_store.cc is blocking me. The related test code uses ad-hoc corruption injection, but the way it is done doesn't manifest the same for mmap versus POSIX I/O, so my change causes spurious test failures (the production code is right, but the test is wrong). I'm grinding through solutions for that.
,
Oct 10 2016
Grr. Chromium compiles with SQLITE_ENABLE_MEMORY_MANAGEMENT, which should enable sqlite3_release_memory() to release all of the pcache buffers, which should work well enough as a stand-in for sqlite3_db_release_memory(), problem solved. BUT, we patch SQLite to add SQLITE_SEPARATE_CACHE_POOLS which defeats sqlite3_release_memory() because it takes precedence over SQLITE_ENABLE_MEMORY_MANAGEMENT. I need to ponder. sqlite3_release_memory() was basically a way to dump the page caches without having to wire things through between the test and the originating code. Maybe things just need to be wired through.
,
Oct 14 2016
,
Oct 14 2016
,
Oct 24 2016
shess@, would you be able to fix this in Q4?
,
Oct 28 2016
https://codereview.chromium.org/2397753005 is the prospective CL for this. The problem in #2 has been solved for enough time that I think it will stick, so moving forward.
,
Nov 7 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a62504d228eedeed490044b79403a9de9eb8a4cb commit a62504d228eedeed490044b79403a9de9eb8a4cb Author: shess <shess@chromium.org> Date: Mon Nov 07 19:26:12 2016 [sql] Allow storing mmap status in a VIEW instead of meta table. Some databases do not use the sql::MetaTable to handle versioning. The mmap verification pass uses the [meta] table to store progress for larger databases, and it uses detection of the [meta] table to distinguish new/empty databases from existing databases. For non-[meta] databases, store the status in an MmapStatus view. BUG= 652359 Review-Url: https://codereview.chromium.org/2397753005 Cr-Commit-Position: refs/heads/master@{#430343} [modify] https://crrev.com/a62504d228eedeed490044b79403a9de9eb8a4cb/chrome/browser/extensions/activity_log/activity_database.cc [modify] https://crrev.com/a62504d228eedeed490044b79403a9de9eb8a4cb/chrome/browser/predictors/predictor_database.cc [modify] https://crrev.com/a62504d228eedeed490044b79403a9de9eb8a4cb/components/sync/syncable/directory_backing_store.cc [modify] https://crrev.com/a62504d228eedeed490044b79403a9de9eb8a4cb/content/browser/dom_storage/dom_storage_database.cc [modify] https://crrev.com/a62504d228eedeed490044b79403a9de9eb8a4cb/sql/connection.cc [modify] https://crrev.com/a62504d228eedeed490044b79403a9de9eb8a4cb/sql/connection.h [modify] https://crrev.com/a62504d228eedeed490044b79403a9de9eb8a4cb/sql/connection_unittest.cc
,
Nov 10 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e28e2e598746a19c53447f8d54c1927aba6e5b59 commit e28e2e598746a19c53447f8d54c1927aba6e5b59 Author: shess <shess@chromium.org> Date: Thu Nov 10 02:12:34 2016 [sql] Histogram for mmap status VIEW. https://codereview.chromium.org/2397753005/ landed a change to how sql::Connection handles mmap tracking. I neglected to add the histograms.xml entry to mirror an enum change. BUG= 652359 Review-Url: https://codereview.chromium.org/2483783006 Cr-Commit-Position: refs/heads/master@{#431143} [modify] https://crrev.com/e28e2e598746a19c53447f8d54c1927aba6e5b59/tools/metrics/histograms/histograms.xml
,
Jan 3 2017
Should this be marked fixed?
,
Mar 17 2017
AFAICT, this is working fine. Lots of new successes registered in histograms, a few failures, the failures at first glance appear in line with the failures for the meta-table storage, so I think they just indicate cases where the database is somewhat broken, rather than any new issues. I did have a moment where I worried that EVENT_MMAP_META_FAILURE_UPDATE and EVENT_MMAP_STATUS_FAILURE_UPDATE were untested and thus perhaps just wrong, but, unfortunately, after verifying that they were untested and testing them, the code seems right. So I'll land tests for that shortly, but I think the bug can be closed.
,
Mar 20 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a7b07acd99ebfd40b319dfd49103081a8c538ec4 commit a7b07acd99ebfd40b319dfd49103081a8c538ec4 Author: shess <shess@chromium.org> Date: Sun Mar 19 23:59:38 2017 [sql] Tests for updates to mmap state. Histograms indicate a bit more EVENT_MMAP_STATUS_FAILURE_UPDATE and EVENT_MMAP_META_FAILURE_UPDATE than I was hoping for, and I realized that that code was untested. Add some tests for that case. Unfortunately for the histograms, the untested code appears to work. BUG= 652359 Review-Url: https://codereview.chromium.org/2756843003 Cr-Commit-Position: refs/heads/master@{#458007} [modify] https://crrev.com/a7b07acd99ebfd40b319dfd49103081a8c538ec4/sql/connection_unittest.cc |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by sh...@chromium.org
, Oct 3 2016