Project: chromium Issues People Development process History Sign in
New issue
Advanced search Search tips
Issue 125641 IndexedDB transactions not always persisted
Starred by 12 users Project Member Reported by pweis@google.com, Apr 30, 2012 Back to list
Status: Fixed
Owner: dgro...@chromium.org
Closed: Jan 2013
Components:
OS: Linux
Pri: 2
Type: Bug


Sign in to add a comment
Chrome Versions : 19.0.1084.[15,24,30,36} and 20.0.1105.0
OS Version: Linux

We've observered several instances of IndexedDB transactions completing without the committed data being seen in subsequent transactions in the same tab. In some reports of this, Chrome crashed shortly after being in this state, and restarting Chrome always seems to put the database back in a sane state where the data from completed transactions is persisted.

 
Comment 1 by jsb...@chromium.org, Apr 30, 2012
Labels: -Area-Undefined Area-WebKit Feature-IndexedDB
Labels: Hotlist-GoogleApps
Comment 3 by dgro...@chromium.org, May 11, 2012
Status: Started
Comment 4 by jsb...@chromium.org, May 16, 2012
Labels: WebKit-ID-85841
Quick status update for those watching the bug:

* The trigger is an I/O error when leveldb is writing to disk. A common cause is e.g. NFS-mounted homedirs becoming inaccessible after Kerberos tickets expire, but other causes (e.g. unplugging drives, etc) can have the same effect.

* At this point, IndexedDB transactions may fail to write through to disk, but the error is silently ignored.

* If leveldb attempts to do a background compaction at this point it will enter a pathological state where it spins a thread retrying. An error state is also set that will fail all future Write calls. This results in the behavior of IndexedDB transactions completing but the data not being present on read-back.

* For added fun, if an attempt to open a database fails due to I/O error, IDB will end up spinning trying to decode bad data. The browser process enters a 100% CPU loop and allocates memory until it crashes.

* http://webkit.org/b/85841 changes the behavior of WebKit's IndexedDB so that if the writes to disk fail the transaction will be aborted. 

More bugs/fixes will be forthcoming as we fully understand the issue and decide how to address it.
Project Member Comment 5 by bugdroid1@chromium.org, May 17, 2012
Labels: -WebKit-ID-85841 WebKit-ID-85841-RESOLVED WebKit-Rev-116562
https://bugs.webkit.org/show_bug.cgi?id=85841
http://trac.webkit.org/changeset/116562
Comment 6 by wiltzius@chromium.org, Jun 14, 2012
@jsbell when do we want to call this bug done? I know there were a whole host of changes that landed; are we waiting on particular ones?
Comment 7 by jsb...@chromium.org, Jun 14, 2012
Blockedon: 131818
Owner: alecflett@chromium.org
Two things left to do:

* crbug.com/131818 which is a LevelDB roll to pick up the fix for the "pathological state" above
* land a change that will detect I/O errors, abort in-flight transactions, and attempt to re-open the database.

Assigning to alecflett@ since he was working on the patch for the latter.

Comment 8 by alecflett@google.com, Jun 15, 2012
I have been keeping that patch alive, but I haven't written the unit tests yet. 
Comment 9 by jsb...@chromium.org, Jun 22, 2012
Blockedon: -chromium:131818 chromium:131818
Note that we need to ensure we surface an IDB error (or recover silently with no data loss from) all leveldb I/O errors. That includes not returning dummy data on a failed read as we do today.
Hey guys,

Quick check on this while I'm combing Chrome/Apps bugs. Any word?

Thanks!
Tom
Quick ping again-- Alec are you still working on this / on hold / done?
No updates on my end - still on hold to write some tests to see if it actually does what it claims.
Hey Alec, any word? Just going through our Chrome/Apps hotlist. Thanks!
David and I have each taken some time to try to write tests for this but still no luck

(The real danger here is that if we do this wrong, that we run the risk of crashing chrome, since we're running in the browser process for this) 
My usual ping :)

Any movement? Thanks!
Hey guys-- again poking this (we sweep the Chrome/Apps issues every two weeks). If you no longer think this is high priority enough to warrant being on there (because the part affects Apps is done with and its just tests on our side now), lemme know!
I think if there are workarounds on the docs side, then it might be worth taking this off the hotlist...
Comment 18 by jo...@ronomon.com, Oct 3, 2012
Have any fixes been made or are all the issues listed above still a problem? It is difficult to recommend Chrome for intensive use, if bugs like these are not addressed.
joran, have you experienced these problems around corruption? We have not heard of this occurring outside of google but we'd love to find a testcase.
Comment 20 by jo...@ronomon.com, Oct 4, 2012
Alec, have been working on an app in production for a group of users storing around 1GB per machine.

Does Chrome remove a database if it detects corruption? If so, we may have seen it a few times already, with users having their database removed by Chrome for no known reason.

And then some related notes:

We had a case of many writes to IndexedDB causing an older Windows machine to crash and reboot deterministically.

Poor write throughput (readers block writers).

Our keys and values are binary, for performance and space efficiency, but IndexedDB has no concept of binary keys, which is strange for a storage layer.

No way for users to grant storage permissions without signing up to a proprietary third party web store.

Blocked events should be passed through the onerror chain, that would be less surprising.

If LevelDB could be exposed directly, that would be more useful.
> Does Chrome remove a database if it detects corruption? If so, we may have seen it a few times already, with 
> users having their database removed by Chrome for no known reason.

Yes, it does

And then some related notes:

> We had a case of many writes to IndexedDB causing an older Windows machine to crash and reboot 
> deterministically.

It would be great if you had a test case for this as this is obviously quite surprising, and this is the first I've heard of it.

> Poor write throughput (readers block writers).

These are two different issues (poor throughput, and transaction blocking) but the transaction issue is:
http://code.google.com/p/chromium/issues/detail?id=64076
(basically all transactions are run serially right now) but that is unrelated to this bug about corruption.

> Our keys and values are binary, for performance and space efficiency, but IndexedDB has no concept of binary keys, which is strange for a storage layer.

Also no connection to this bug but I agree this is a problem.

> No way for users to grant storage permissions without signing up to a proprietary third party web store.

Again, not related to corruption, and again I agree this is a problem. Please file a bug.

> Blocked events should be passed through the onerror chain, that would be less surprising.

Again, not an issue with corruption, but an issue with the spec. Please take this up on the W3C lists.

> If LevelDB could be exposed directly, that would be more useful.

LevelDB is an implementation detail - Firefox uses SQLLite I think under the hood and IE uses its own storage system. If you're concerned about proprietary third party issues, then leveldb is not the answer here as you'll be even less cross-browser, and you'd still have to deal with recognizing corruption.

This bug is about recovery from corrupt databases, lets stay focused on that. It sounds like you've had issues with it but we'd still need descriptions of how to produce corruption. I welcome discussion of our implementation issues on chromium-html5@chromium.org.
Comment 22 by jo...@ronomon.com, Oct 4, 2012
We tried it several times on that machine and when the writing started it just rebooted. It was out in the field and that was awhile ago. They just switched to another machine.

Regarding the unrelated notes, hope they were at least helpful if you weren't aware of them already through other forums.
Comment 23 by laforge@google.com, Oct 17, 2012
Labels: -Feature-IndexedDB WebKit-Storage-IndexedDB
Comment 24 by jsb...@chromium.org, Jan 22, 2013
Owner: dgro...@chromium.org
Comment 25 by dgro...@chromium.org, Jan 23, 2013
Status: Fixed
I did a pretty thorough sweep of the IDB code, it should now fire an abort event in all cases where data might not have been persisted. I think we can call this done.

Note that there are still open issues around corruption (issue 146284) and how to recover from it (issue 171561).

Blockedon: chromium:171561
Project Member Comment 27 by bugdroid1@chromium.org, Mar 11, 2013
Labels: -Area-WebKit -WebKit-Storage-IndexedDB Cr-Content Cr-Content-Storage-IndexedDB
Project Member Comment 28 by bugdroid1@chromium.org, Apr 6, 2013
Labels: -Cr-Content Cr-Blink
Project Member Comment 29 by bugdroid1@chromium.org, Apr 6, 2013
Labels: -Cr-Content-Storage-IndexedDB Cr-Blink-Storage-IndexedDB
Sign in to add a comment