New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 261623 link

Starred by 266 users

Comments by non-members will not trigger notification emails to users who starred this issue.

Issue metadata

Status: Fixed
Owner:
Closed: Mar 2014
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 2
Type: Bug

Blocking:
issue tampermonkey:168


Show other hotlists

Hotlists containing this issue:
Hotlist-1
Hotlist-1


Sign in to add a comment

Unrecoverable chrome.storage.sync database corruption

Reported by bean...@gmail.com, Jul 18 2013

Issue description

UserAgent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1566.2 Safari/537.36

Steps to reproduce the problem:
I'm not certain how this happened, but was probably the result of my browser and/or computer crashing during a write to chrome.storage.sync.

What is the expected behavior?
I should be able to use chrome.storage.sync.get, or at the very least chrome.storage.sync.clear.

What went wrong?
My Chrome App uses chrome.storage.sync to store a session token. Right now, I can't open my app. When it attempts to call chrome.storage.sync.set, I get the following:

storage.get: Failed to open database: Corruption: 1 missing files; e.g.: .../000005.sst
    at Function.ue.init (chrome-extension://djejicklhojeokkfmdelnempiecmdomj/chromechart/background_lib.js:393:694)
    at we.init (chrome-extension://djejicklhojeokkfmdelnempiecmdomj/chromechart/background_lib.js:395:166)
    at chrome-extension://djejicklhojeokkfmdelnempiecmdomj/chromechart/background_lib.js:395:479
    at Array.forEach (native)
    at chrome-extension://djejicklhojeokkfmdelnempiecmdomj/chromechart/background_lib.js:395:459
    at auth (chrome-extension://djejicklhojeokkfmdelnempiecmdomj/chromechart/background_lib.js:7953:5)
    at chrome-extension://djejicklhojeokkfmdelnempiecmdomj/chromechart/background_lib.js:8162:3 

The worst part is that it calls the callback that I give to chrome.storage.sync.get, but it passes in null rather than an empty object. If I ask for a non-existent key, it is supposed to give me an empty object rather than null. So when I try to check "if('mykey' in storageData)", I get an exception that I can't use "in" on null.

Uninstalling and reinstalling the app does not help. chrome.storage.sync.clear() results in the same exception, and does not work. Is there some workaround for clearing a corrupted storage database? I'm not sure how many of our users this will affect, but it happened to me in the first few weeks of using our app.

I expected that if the database became corrupted, it would be replaced with a working, empty database at the very least.

Did this work before? N/A 

Chrome version: 30.0.1566.2  Channel: dev
OS Version: 6.1 (Windows 7, Windows Server 2008 R2)
Flash Version: Shockwave Flash 11.8 r800
 
Showing comments 15 - 114 of 114 Older

Comment 15 by sc...@fujan.name, Aug 9 2013

My anecdotal experience implies that it may be related to the hard drive being too full. This has been the majority of error reports to me for the last month.
That could definitely be a cause of this. Leveldb tries to writes its sst file, out of disk space, fails to write, corrupted.

@dgrogan is there a way to make leveldb survive this sort of things transactionally?

Comment 17 by bean...@gmail.com, Aug 9 2013

I can't speak for everyone, but in my case, my hard drive was definitely not full when this corruption occurred.

Comment 18 by ngy...@gmail.com, Aug 9 2013

My own extension is also suffering from this same issue, and I ended up writing a fallback method to the extension's localStorage when chrome.storage.local fails.

In fact, the situation is so bad for one other extension I volunteer for support that the support team wrote a document to tell people to manually delete folders in Chrome User Data: https://docs.google.com/document/d/1tenz6elEJxEOUNvBHl4_zWbhj_F4vLRLwD0c4IVwL6A/edit?pli=1

There's one clue from user reports that might solve the mystery: This seems to ALWAYS happen right after a Windows System Restore.

Comment 19 by bean...@gmail.com, Aug 9 2013

That's interesting that they mention Windows System Restore. I don't remember if this happened to me at that exact time, but I definitely did a System Restore once or twice around the time that this happened to me.
What ngyikp said, most user reports told me that it happeneds after a system restore.

Also I wrote the following in another issue (  Issue 270665  ) but I should have asked it here instead:
When the leveldb will get reset, will Chrome re-fetch synced data?
If full disk causes leveldb to lose an sst file, there's a bug in leveldb. Which isn't out of the question. What's supposed to happen is that leveldb won't record the sst file's existence to its internal manifest until the sst file has been written successfully to disk. The "n missing files; e.g..." error message occurs when leveldb's internal manifest points to a file that can't be found. It's the leading cause of corruption in IndexedDB: http://goto.google.com/poduu (link will only work for googlers, apologies to others)

@ngyikp thanks for the tip about Windows System Restore. I will try to reproduce on my windows vm when I'm back at my desk in a few weeks.
That is very interesting about the system restore.

If you delete the directory without Chrome's involvement then sync should re-fetch it next time Chrome starts, yes.

Comment 23 by ngy...@gmail.com, Aug 10 2013

I also noticed that if the user uninstalls the extension, the extension's settings folder in "Local Extension Settings" or "Sync Extension Settings" don't get deleted.

Ideally, when the user tells Chrome to uninstall an extension, it SHOULD remove all traces of the extension including settings. This isn't a problem for extensions that still use localStorage.

At least make it so that uninstalling an extension will wipe out the settings, so we don't need to tell people to go digging through their Chrome User Data.
There is code to delete it:

https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/extensions/data_deleter.cc&q=data_deleter.c&sq=package:chromium&type=cs&l=63

perhaps file a separate issue if you can repro chrome failing to clear the database?
Our extension have more than 400k users and we are getting around 10-20 messages/day related on this issue. The majority of users that reported the issue are on stable channel / win. In the last week it became a mass phenomenon. Note: we don't use sync at all, only chrome.storage.local.
If storage.sync is corrupted, storage.local is as well (and likely vice versa).
System restore caused leveldb corruption (missing files) for me the first time I tried it. Will investigate more and think on a solution or workaround.
Project Member

Comment 28 by bugdroid1@chromium.org, Sep 10 2013

------------------------------------------------------------------------
r222178 | kalman@chromium.org | 2013-09-10T01:54:30.486528Z

Changed paths:
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_storage_quota_enforcer.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_sync_unittest.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/storage_api.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/sync/test/integration/extension_settings_helper.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_frontend_unittest.cc?r1=222178&r2=222177&pathrev=222178
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/value_store_util.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/leveldb_value_store.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/testing_value_store.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/value_store_frontend_unittest.cc?r1=222178&r2=222177&pathrev=222178
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/value_store_util.h?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/value_store_unittest.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/value_store_frontend.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/leveldb_value_store.h?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/testing_value_store.h?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/syncable_settings_storage.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/value_store.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_browser.gypi?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/value_store.h?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/policy_value_store_unittest.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_backend.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/policy_value_store.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_quota_unittest.cc?r1=222178&r2=222177&pathrev=222178

Propagate more information about ValueStore errors to callers, notably an
error code in addition to the error message. This turned out to be a
surprisingly invasive yak shave and is accompanied by a few very small fixes
to inefficient and overly complex code I noticed along the way.

This is the first stage in making the chrome.storage extension/app API able to
gracefully recover from leveldb corruption.

BUG= 261623 
R=mpcomplete@chromium.org
TBR=tim@chromium.org
NOTRY=true

Review URL: https://chromiumcodereview.appspot.com/24021002
------------------------------------------------------------------------
I am seeing this issue on customer where the browser update has some issues.  The browser update fails or takes a long time to finish.  
LocalStorage Issue.png
64.8 KB View Download
Project Member

Comment 30 by bugdroid1@chromium.org, Sep 20 2013

------------------------------------------------------------------------
r224433 | dgrogan@chromium.org | 2013-09-20T18:13:50.413641Z

Changed paths:
   M http://src.chromium.org/viewvc/chrome/trunk/src/DEPS?r1=224433&r2=224432&pathrev=224433
   M http://src.chromium.org/viewvc/chrome/trunk/src/third_party/leveldatabase/README.chromium?r1=224433&r2=224432&pathrev=224433

Roll leveldb to r77.

Biggest change for chromium is no longer using .sst as the sstable file
extension. The .sst extension caused windows system restore to corrupt
databases (error message: Corruption: 1 missing files; e.g...).

The only other change that could affect chromium is a bugfix, where
changing directions on an iterator could yield the wrong value.
(https://code.google.com/p/leveldb/issues/detail?id=200)

BUG= 261623 
R=jsbell@chromium.org

Review URL: https://codereview.chromium.org/23976008
------------------------------------------------------------------------
^ Fantastic.

Now to finish my part of the work and write the recovery logic. Probably isn't going to happen by Chrome 31.
Labels: M-32

Comment 33 by derj...@gmail.com, Oct 2 2013

Hi, Tampermonkey (~800k users) dev here. I recently switched over to chrome.storage.local and now some users report that their scripts are gone after a Chrome restart. 

This is the error message that the user reported:
storage.set: Failed to open database: Corruption: 1 missing files; e.g.: .../000005.sst

This is the user-agent of that user:
Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.66 Safari/537.36

Three questions:
1) This issue was reported for chrome.storage.sync. Does my issue have the same root cause?
2) What can I advise to my user to recover from that?
3) Can  issue 294409  be related?

Thanks. :)
@derjanb - is it possible to check with the user(s) who reported the problem to see if they had run a Windows system restore?

Yes, the problem exists for *.local, not just *.sync. This is an ongoing (and very annoying) problem for many of my users. They even give my extension 1 star in the Web Store because they think it "stopped working", despite it being a Chrome problem.

QUESTION - Is there a good way to duplicate this problem? I've tried a system restore and it didn't corrupt my system. I would like to simulate it so I can try to recover from it in my code and still display something useful to my users, if possible.
matthew.a.kruse, I can reproduce this problem easily. You have to write a bunch of data after creating the restore point.
1) Using beta or stable channel chrome, write to a leveldb database via your favorite javascript api (chrome.storage.* or indexeddb).
2) Create a system restore point.
3) Write a bunch more data to leveldb such at least 1 new .sst file is generated.
4) Restore the restore point created in (2).
5) The .sst files that were created in (3) will be missing.

Comment 37 by sif...@gmail.com, Oct 2 2013

Matt, in my testing, closing the browser and deleting one or more .sst files in the extension's folder caused the problem.
Thanks for the tips! The part I was missing was writing a bunch of data, to create the sst files. Once I did that I could recreate the problem, trap the call in try/catch, and at least carry on with running the extension in a prefs-less mode.

Users will still lose their preferences, but at least they won't complain that the extension disappeared completely.

Comment 39 by sc...@fujan.name, Oct 2 2013

Matthew, can you provide a sample for detecting the corruption? I assume that you try calling chrome.storage.local.get and the error is thrown immediately? No async error detection is needed?
Actually, I don't need the try/catch at all. This code detects the problem, and the async return function is called as usual, but with an argument of undefined:

chrome.storage.local.get(defaults,function(ret) {
	if (chrome&&chrome.extension&&chrome.extension.lastError) {
		console.log("Chrome error: "+chrome.extension.lastError.message)
		// ret == undefined !
	}
...
 
My code was always assuming a value would be returned, even if it was null or empty. Bad assumption to make, apparently. 

I assume that .set() will fail similarly, and it will just appear to the extension that nothing is ever being persisted, which is a condition I need to account for.

Comment 41 by derj...@gmail.com, Oct 2 2013

@jsbell yes, he run a Windows system restore.
It is related to the chrome update, today a new build was released, please see http://googlechromereleases.blogspot.com/. I have had similar issues with a chrome installable app when chrome updates.  The develdb for local storage ,indexeddb, and file system gets corrupted.  I know this is not reasonable for most, but I have advised clients to disable the automatic chrome update.  Currently a backup of the google user data folder is recommended and create regular data backup exports of the application data.  I am currently looking at options to use an external database like couchdb instead of the local indexeddb, I am using a js cache service instead of local storage.
A more reasonable workaround is to JSON encode your data and use the File System API from the HTML5 specification (a good tutorial at www.html5rocks.com/en/tutorials/file/filesystem/).

You can store everything in file, or use a different file for each key (be aware that some characters will be illegal in the file name).

The only significant difference is that chrome.storage will correctly serialize circular references, while JSON.stringify will not. 
Notes:

(a) Don't disable chrome autoupdate. This is not autoupdate related.

(b) We don't actually correctly serialized circular references sadly, we just don't break if they exist.

Comment 45 by omn...@gmail.com, Oct 4 2013

File System Api in Chrome is actually based on same, not very robust, leveldb technology (why so complicated by the way?). And the biggest problem with File System Api is that it just impossible to backup and restore this data, for users(as this is possible with IndexedDB or WebSQL, or local storage) - because all folders names is just random 000, 001, 002.... instead of the extension/application id, nor it is not easily possible to help users find the data on a filesystem in case of some problems. All of this is completely not maintainable.

Comment 46 Deleted

Comment 47 by cde...@gmail.com, Oct 6 2013

I'm a tampermonkey user and I got the problem with the disappearing scripts too, I don't have much info to give and you're all talking chinese for me here. I just wanted to say that I did NOT do a system restore, I'l running a fresh install and restore is disabled
Cc: haibinlu@chromium.org dbbrooks@chromium.org markdavidscott@google.com
The Cast Extension is also experiencing this issue (~500K users, not sure how many it  affects).

Sample from JS console:

storage.get: Failed to open database: Corruption: 2 missing files; e.g.: .../000035.sst
    at d.IC (chrome-extension://fmfcbgogabcbclcofgocippekhfcmgfj/background_script.js:1653:88)
    at HTMLDocument.<anonymous> (chrome-extension://fmfcbgogabcbclcofgocippekhfcmgfj/background_script.js:14258:58) 

Happens on Windows 31.0.1566.69

Comment 49 by cde...@gmail.com, Oct 8 2013

In the meanwhile i got it sorted by creating a new chrome profile;

Comment 50 by derj...@gmail.com, Oct 12 2013

Blocking: tampermonkey:168
Tampermonkey user; notified by TM script that "chrome profile was broken", and pointed to this page. 
Dual booting Win7 x64 /   SLES 11,   dual 1TB hard drives, no raid - more than 800GB available in Windows partition. 
Yes, I have used System Restore.. but last use was about 3 weeks ago;   several- average daily - 'normal' shutdowns/ restarts since then. 
Using TM for gaming,   other Extensions active include Google Music, Chrome to Phone, Google drawings. 
Will post other diagnostics if asked. 

Comment 52 by Deleted ...@, Oct 25 2013

Chrome has really messed up my new computer; never ever download 8.1 WINDOWS either.

My extension began experiencing a "magic number" error following a System Restore on a Win 7 machine. My extension was using the chrome.storage.local property of the Chrome storage API.

I ended up having to manually delete my extension's subfolder from the Local Extension Settings folder in order to clear the issue. I have switched my users over to HTML5's synchronous localStorage, which does not seem to be similarly affected.

Comment 54 by Deleted ...@, Oct 26 2013

 

Reported by Bean...@gmail.com, Jul 18, 2013 
UserAgent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1566.2 Safari/537.36

Steps to reproduce the problem:
I'm not certain how this happened, but was probably the result of my browser and/or computer crashing during a write to chrome.storage.sync.

What is the expected behavior?
I should be able to use chrome.storage.sync.get, or at the very least chrome.storage.sync.clear.

What went wrong?
My Chrome App uses chrome.storage.sync to store a session token. Right now, I can't open my app. When it attempts to call chrome.storage.sync.set, I get the following:

storage.get: Failed to open database: Corruption: 1 missing files; e.g.: .../000005.sst
    at Function.ue.init (chrome-extension://djejicklhojeokkfmdelnempiecmdomj/chromechart/background_lib.js:393:694)
    at we.init (chrome-extension://djejicklhojeokkfmdelnempiecmdomj/chromechart/background_lib.js:395:166)
    at chrome-extension://djejicklhojeokkfmdelnempiecmdomj/chromechart/background_lib.js:395:479
    at Array.forEach (native)
    at chrome-extension://djejicklhojeokkfmdelnempiecmdomj/chromechart/background_lib.js:395:459
    at auth (chrome-extension://djejicklhojeokkfmdelnempiecmdomj/chromechart/background_lib.js:7953:5)
    at chrome-extension://djejicklhojeokkfmdelnempiecmdomj/chromechart/background_lib.js:8162:3 

The worst part is that it calls the callback that I give to chrome.storage.sync.get, but it passes in null rather than an empty object. If I ask for a non-existent key, it is supposed to give me an empty object rather than null. So when I try to check "if('mykey' in storageData)", I get an exception that I can't use "in" on null.

Uninstalling and reinstalling the app does not help. chrome.storage.sync.clear() results in the same exception, and does not work. Is there some workaround for clearing a corrupted storage database? I'm not sure how many of our users this will affect, but it happened to me in the first few weeks of using our app.

I expected that if the database became corrupted, it would be replaced with a working, empty database at the very least.

Did this work before? N/A 

Chrome version: 30.0.1566.2  Channel: dev
OS Version: 6.1 (Windows 7, Windows Server 2008 R2)
Flash Version: Shockwave Flash 11.8 r800



 Jul 18, 2013  
#1 mpcomplete@chromium.org (No comment was entered for this change.)






Status: Assigned 
Owner: kal...@chromium.org 






 Jul 18, 2013  
#2 kal...@chromium.org You would need to rm -r that directory. It sounds like I should add fallback code to do this automatically though.

The null-in-callback thing probably implies that chrome.runtime.lastError has been set.



 Jul 18, 2013  
#3 micha...@chromium.org +jsbell +dgrogan for additional leveldb lore






Cc: jsbell@chromium.org dgrogan@chromium.org 






 Jul 22, 2013  
#4 stepien....@gmail.com I personally have never experienced it but a *lot* of my users have:
https://github.com/MayhemYDG/4chan-x/issues/1077
https://github.com/MayhemYDG/4chan-x/issues/1121
https://github.com/MayhemYDG/4chan-x/issues/1132
https://github.com/MayhemYDG/4chan-x/issues/1170



 Jul 25, 2013  
#5 Kyle0...@gmail.com I just want to add that this seems to be a major issue with my extension as well. It seems to be happening to a small set of users, but enough that it's an issue.

https://chrome.google.com/webstore/detail/lol-stream-browser/edidfaijmhpefkbnobdcepampbncgejp?hl=en-US



 Jul 25, 2013  
#6 kal...@chromium.org (No comment was entered for this change.)






Status: Started 






 Jul 25, 2013  
#7 Kyle0...@gmail.com Here's the logs from my extension with a user who's having issues. A major note is that this call failing doesn't fall back to local storage as specified in the Chrome docs, so storage doesn't work at all.

storage.get: Failed to open database: Corruption: 1 missing files; e.g.: .../001310.sst at StreamBrowser.Host.SyncStorage.get (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/host_chrome.js:18:37) at new Store (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/storage.js:56:14) at new DualStore (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/storage.js:78:17) at new <anonymous> (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/storage.js:88:21) at chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/storage.js:86:25 lastError:29 error getting sync storage: [object Object] host_chrome.js:20 storage: settings.version: 1.2.0.6 storage.js:64

storage.get: Failed to open database: Corruption: 1 missing files; e.g.: .../001310.sst at StreamBrowser.Host.SyncStorage.get (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/host_chrome.js:18:37) at new Store (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/storage.js:56:14) at new DualStore (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/storage.js:78:17) at new <anonymous> (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/storage.js:89:17) at chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/storage.js:86:25 lastError:29 error getting sync storage: [object Object] host_chrome.js:20

storage.set: Failed to open database: Corruption: 1 missing files; e.g.: .../001310.sst at StreamBrowser.Host.SyncStorage.set (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/host_chrome.js:30:41) at Store.set (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/storage.js:49:18) at migrateVersion (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/streambg.js:210:55) at Object.$.ajax.success (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/streambg.js:227:21) at l (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/jquery-2.0.0.min.js:4:24584) at Object.c.fireWith [as resolveWith] (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/jquery-2.0.0.min.js:4:25405) at k (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/jquery-2.0.0.min.js:6:4690) lastError:29

storage.set: Failed to open database: Corruption: 1 missing files; e.g.: .../001310.sst at StreamBrowser.Host.SyncStorage.set (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/host_chrome.js:30:41) at Store.set (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/storage.js:49:18) at migrateVersion (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/streambg.js:211:55) at Object.$.ajax.success (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/streambg.js:227:21) at l (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/jquery-2.0.0.min.js:4:24584) at Object.c.fireWith [as resolveWith] (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/jquery-2.0.0.min.js:4:25405) at k (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/jquery-2.0.0.min.js:6:4690) lastError:29

storage.set: Failed to open database: Corruption: 1 missing files; e.g.: .../001310.sst at StreamBrowser.Host.SyncStorage.set (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/host_chrome.js:30:41) at Store.set (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/storage.js:49:18) at migrateVersion (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/streambg.js:215:51) at Object.$.ajax.success (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/streambg.js:227:21) at l (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/jquery-2.0.0.min.js:4:24584) at Object.c.fireWith [as resolveWith] (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/jquery-2.0.0.min.js:4:25405) at k (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/jquery-2.0.0.min.js:6:4690) lastError:29

storage.set: Failed to open database: Corruption: 1 missing files; e.g.: .../001310.sst at StreamBrowser.Host.SyncStorage.set (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/host_chrome.js:30:41) at Store.set (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/storage.js:49:18) at $.fn.settingsCheckbox (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/streamfg.js:25:49) at HTMLDocument.<anonymous> (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/streamfg.js:64:22) at l (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/jquery-2.0.0.min.js:4:24584) at Object.c.fireWith [as resolveWith] (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/jquery-2.0.0.min.js:4:25405) at Function.x.extend.ready (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/jquery-2.0.0.min.js:4:2897) lastError:29

storage.set: Failed to open database: Corruption: 1 missing files; e.g.: .../001310.sst at StreamBrowser.Host.SyncStorage.set (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/host_chrome.js:30:41) at Store.set (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/storage.js:49:18) at $.fn.settingsCheckbox (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/streamfg.js:25:49) at HTMLDocument.<anonymous> (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/streamfg.js:65:22) at l (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/jquery-2.0.0.min.js:4:24584) at Object.c.fireWith [as resolveWith] (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/jquery-2.0.0.min.js:4:25405) at Function.x.extend.ready (chrome-extension://edidfaijmhpefkbnobdcepampbncgejp/js/jquery-2.0.0.min.js:4:2897) lastError:29



 Jul 25, 2013  
#8 kal...@chromium.org Storage doesn't fall back to local storage. Where in the docs does it say that?



 Jul 25, 2013  
#9 stepien....@gmail.com >Storage doesn't fall back to local storage. Where in the docs does it say that?

see

>Even if a user disables syncing, storage.sync will still work. In this case, it will behave identically to storage.local.



One of my user deleted the sync folder and claims the issue persists:
https://github.com/MayhemYDG/4chan-x/issues/1205#issuecomment-21550154



 Jul 25, 2013  
#10 Kyle0...@gmail.com https://developer.chrome.com/extensions/storage.html#using-sync

Specifically:

"When Chrome is offline, Chrome stores the data locally. The next time the browser is online, Chrome syncs the data. Even if a user disables syncing, storage.sync will still work. In this case, it will behave identically to storage.local."

My understanding from this was that if sync storage doesn't work, local storage will be used. I guess that's not an entirely accurate understanding though.



 Jul 25, 2013  
#11 kal...@chromium.org Maybe we need to clarify the docs - there is no fallback, the data store isn't shared in any way - sync just doesn't sync if it's offline. Thanks for the feedback there.

I can't explain that user failing each time, perhaps their hard drive or profile is busted in some other way, or they're clearing from the wrong profile or something... or leveldb is super busted.



 Jul 25, 2013  
#12 Kyle0...@gmail.com I unfortunately don't have a repro, but I've had a lot of different users experience this issue, with a lot of different setups. The issues seems to happen regardless of a user being signed into chrome, and even if they've disabled sync. I wish I had more information, though I have added logging to gather more information (as you can see above), and I'll try to report any other information I can obtain (assuming it's helpful).



 Jul 29, 2013  
#13 DHNishi (No comment was entered for this change.)






Labels: Cr-Platform-Extensions 






 Aug 9, 2013  
#14 sc...@fujan.name FYI, I've been noticing error reports like this since June 18th. The user claimed to be on 27.0.1453.116. So it would be at least that early of a bug.



 Aug 9, 2013  
#15 sc...@fujan.name My anecdotal experience implies that it may be related to the hard drive being too full. This has been the majority of error reports to me for the last month.



 Aug 9, 2013  
#16 kal...@chromium.org That could definitely be a cause of this. Leveldb tries to writes its sst file, out of disk space, fails to write, corrupted.

@dgrogan is there a way to make leveldb survive this sort of things transactionally?



 Aug 9, 2013  
#17 Bean...@gmail.com I can't speak for everyone, but in my case, my hard drive was definitely not full when this corruption occurred.



 Aug 9, 2013  
#18 ngyikp My own extension is also suffering from this same issue, and I ended up writing a fallback method to the extension's localStorage when chrome.storage.local fails.

In fact, the situation is so bad for one other extension I volunteer for support that the support team wrote a document to tell people to manually delete folders in Chrome User Data: https://docs.google.com/document/d/1tenz6elEJxEOUNvBHl4_zWbhj_F4vLRLwD0c4IVwL6A/edit?pli=1

There's one clue from user reports that might solve the mystery: This seems to ALWAYS happen right after a Windows System Restore.



 Aug 9, 2013  
#19 Bean...@gmail.com That's interesting that they mention Windows System Restore. I don't remember if this happened to me at that exact time, but I definitely did a System Restore once or twice around the time that this happened to me.



 Aug 9, 2013  
#20 stepien....@gmail.com What ngyikp said, most user reports told me that it happeneds after a system restore.

Also I wrote the following in another issue (  Issue 270665  ) but I should have asked it here instead:
When the leveldb will get reset, will Chrome re-fetch synced data?



 Aug 9, 2013  
#21 dgrogan@chromium.org If full disk causes leveldb to lose an sst file, there's a bug in leveldb. Which isn't out of the question. What's supposed to happen is that leveldb won't record the sst file's existence to its internal manifest until the sst file has been written successfully to disk. The "n missing files; e.g..." error message occurs when leveldb's internal manifest points to a file that can't be found. It's the leading cause of corruption in IndexedDB: http://goto.google.com/poduu (link will only work for googlers, apologies to others)

@ngyikp thanks for the tip about Windows System Restore. I will try to reproduce on my windows vm when I'm back at my desk in a few weeks.



 Aug 9, 2013  
#22 kal...@chromium.org That is very interesting about the system restore.

If you delete the directory without Chrome's involvement then sync should re-fetch it next time Chrome starts, yes.



 Aug 9, 2013  
#23 ngyikp I also noticed that if the user uninstalls the extension, the extension's settings folder in "Local Extension Settings" or "Sync Extension Settings" don't get deleted.

Ideally, when the user tells Chrome to uninstall an extension, it SHOULD remove all traces of the extension including settings. This isn't a problem for extensions that still use localStorage.

At least make it so that uninstalling an extension will wipe out the settings, so we don't need to tell people to go digging through their Chrome User Data.



 Aug 12, 2013  
#24 kal...@chromium.org There is code to delete it:

https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/extensions/data_deleter.cc&q=data_deleter.c&sq=package:chromium&type=cs&l=63

perhaps file a separate issue if you can repro chrome failing to clear the database?



 Aug 14, 2013  
#25 serge.st...@googlemail.com Our extension have more than 400k users and we are getting around 10-20 messages/day related on this issue. The majority of users that reported the issue are on stable channel / win. In the last week it became a mass phenomenon. Note: we don't use sync at all, only chrome.storage.local.



 Aug 19, 2013  
#26 p...@lucidchart.com If storage.sync is corrupted, storage.local is as well (and likely vice versa).



 Aug 20, 2013  
#27 dgrogan@chromium.org System restore caused leveldb corruption (missing files) for me the first time I tried it. Will investigate more and think on a solution or workaround.



 Sep 9, 2013  
#28 bugdro...@chromium.org ------------------------------------------------------------------------
r222178 | kalman@chromium.org | 2013-09-10T01:54:30.486528Z

Changed paths:
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_storage_quota_enforcer.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_sync_unittest.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/storage_api.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/sync/test/integration/extension_settings_helper.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_frontend_unittest.cc?r1=222178&r2=222177&pathrev=222178
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/value_store_util.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/leveldb_value_store.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/testing_value_store.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/value_store_frontend_unittest.cc?r1=222178&r2=222177&pathrev=222178
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/value_store_util.h?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/value_store_unittest.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/value_store_frontend.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/leveldb_value_store.h?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/testing_value_store.h?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/syncable_settings_storage.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/value_store.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_browser.gypi?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/value_store.h?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/policy_value_store_unittest.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_backend.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/policy_value_store.cc?r1=222178&r2=222177&pathrev=222178
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_quota_unittest.cc?r1=222178&r2=222177&pathrev=222178

Propagate more information about ValueStore errors to callers, notably an
error code in addition to the error message. This turned out to be a
surprisingly invasive yak shave and is accompanied by a few very small fixes
to inefficient and overly complex code I noticed along the way.

This is the first stage in making the chrome.storage extension/app API able to
gracefully recover from leveldb corruption.

BUG= 261623 
R=mpcomplete@chromium.org
TBR=tim@chromium.org
NOTRY=true

Review URL: https://chromiumcodereview.appspot.com/24021002
------------------------------------------------------------------------



 Sep 18, 2013  
#29 vijayphi...@gmail.com I am seeing this issue on customer where the browser update has some issues.  The browser update fails or takes a long time to finish.  



   LocalStorage Issue.png 
 64.8 KB   View   Download  
   


 Sep 20, 2013  
#30 bugdro...@chromium.org ------------------------------------------------------------------------
r224433 | dgrogan@chromium.org | 2013-09-20T18:13:50.413641Z

Changed paths:
   M http://src.chromium.org/viewvc/chrome/trunk/src/DEPS?r1=224433&r2=224432&pathrev=224433
   M http://src.chromium.org/viewvc/chrome/trunk/src/third_party/leveldatabase/README.chromium?r1=224433&r2=224432&pathrev=224433

Roll leveldb to r77.

Biggest change for chromium is no longer using .sst as the sstable file
extension. The .sst extension caused windows system restore to corrupt
databases (error message: Corruption: 1 missing files; e.g...).

The only other change that could affect chromium is a bugfix, where
changing directions on an iterator could yield the wrong value.
(https://code.google.com/p/leveldb/issues/detail?id=200)

BUG= 261623 
R=jsbell@chromium.org

Review URL: https://codereview.chromium.org/23976008
------------------------------------------------------------------------



 Sep 20, 2013  
#31 kal...@chromium.org ^ Fantastic.

Now to finish my part of the work and write the recovery logic. Probably isn't going to happen by Chrome 31.



 Sep 20, 2013  
#32 kal...@chromium.org (No comment was entered for this change.)






Labels: M-32 






 Oct 2, 2013  
#33 derj...@googlemail.com Hi, Tampermonkey (~800k users) dev here. I recently switched over to chrome.storage.local and now some users report that their scripts are gone after a Chrome restart. 

This is the error message that the user reported:
storage.set: Failed to open database: Corruption: 1 missing files; e.g.: .../000005.sst

This is the user-agent of that user:
Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.66 Safari/537.36

Three questions:
1) This issue was reported for chrome.storage.sync. Does my issue have the same root cause?
2) What can I advise to my user to recover from that?
3) Can  issue 294409  be related?

Thanks. :)



 Oct 2, 2013  
#34 jsbell@chromium.org @derjanb - is it possible to check with the user(s) who reported the problem to see if they had run a Windows system restore?




 Oct 2, 2013  
#35 matthew....@gmail.com Yes, the problem exists for *.local, not just *.sync. This is an ongoing (and very annoying) problem for many of my users. They even give my extension 1 star in the Web Store because they think it "stopped working", despite it being a Chrome problem.

QUESTION - Is there a good way to duplicate this problem? I've tried a system restore and it didn't corrupt my system. I would like to simulate it so I can try to recover from it in my code and still display something useful to my users, if possible.



 Oct 2, 2013  
#36 dgrogan@chromium.org matthew.a.kruse, I can reproduce this problem easily. You have to write a bunch of data after creating the restore point.
1) Using beta or stable channel chrome, write to a leveldb database via your favorite javascript api (chrome.storage.* or indexeddb).
2) Create a system restore point.
3) Write a bunch more data to leveldb such at least 1 new .sst file is generated.
4) Restore the restore point created in (2).
5) The .sst files that were created in (3) will be missing.



 Oct 2, 2013  
#37 sif...@gmail.com Matt, in my testing, closing the browser and deleting one or more .sst files in the extension's folder caused the problem.



 Oct 2, 2013  
#38 matthew....@gmail.com Thanks for the tips! The part I was missing was writing a bunch of data, to create the sst files. Once I did that I could recreate the problem, trap the call in try/catch, and at least carry on with running the extension in a prefs-less mode.

Users will still lose their preferences, but at least they won't complain that the extension disappeared completely.



 Oct 2, 2013  
#39 sc...@fujan.name Matthew, can you provide a sample for detecting the corruption? I assume that you try calling chrome.storage.local.get and the error is thrown immediately? No async error detection is needed?



 Oct 2, 2013  
#40 matthew....@gmail.com Actually, I don't need the try/catch at all. This code detects the problem, and the async return function is called as usual, but with an argument of undefined:

chrome.storage.local.get(defaults,function(ret) {
	if (chrome&&chrome.extension&&chrome.extension.lastError) {
		console.log("Chrome error: "+chrome.extension.lastError.message)
		// ret == undefined !
	}
...
 
My code was always assuming a value would be returned, even if it was null or empty. Bad assumption to make, apparently. 

I assume that .set() will fail similarly, and it will just appear to the extension that nothing is ever being persisted, which is a condition I need to account for.



 Oct 2, 2013  
#41 derj...@googlemail.com @jsbell yes, he run a Windows system restore.



 Oct 2, 2013  
#42 vijayphi...@gmail.com It is related to the chrome update, today a new build was released, please see http://googlechromereleases.blogspot.com/. I have had similar issues with a chrome installable app when chrome updates.  The develdb for local storage ,indexeddb, and file system gets corrupted.  I know this is not reasonable for most, but I have advised clients to disable the automatic chrome update.  Currently a backup of the google user data folder is recommended and create regular data backup exports of the application data.  I am currently looking at options to use an external database like couchdb instead of the local indexeddb, I am using a js cache service instead of local storage.



 Oct 2, 2013  
#43 p...@lucidchart.com A more reasonable workaround is to JSON encode your data and use the File System API from the HTML5 specification (a good tutorial at www.html5rocks.com/en/tutorials/file/filesystem/).

You can store everything in file, or use a different file for each key (be aware that some characters will be illegal in the file name).

The only significant difference is that chrome.storage will correctly serialize circular references, while JSON.stringify will not. 



 Oct 3, 2013  
#44 kal...@chromium.org Notes:

(a) Don't disable chrome autoupdate. This is not autoupdate related.

(b) We don't actually correctly serialized circular references sadly, we just don't break if they exist.



 Oct 4, 2013  
#45 omn...@gmail.com File System Api in Chrome is actually based on same, not very robust, leveldb technology (why so complicated by the way?). And the biggest problem with File System Api is that it just impossible to backup and restore this data, for users(as this is possible with IndexedDB or WebSQL, or local storage) - because all folders names is just random 000, 001, 002.... instead of the extension/application id, nor it is not easily possible to help users find the data on a filesystem in case of some problems. All of this is completely not maintainable.



 Oct 6, 2013  
#47 cde...@gmail.com I'm a tampermonkey user and I got the problem with the disappearing scripts too, I don't have much info to give and you're all talking chinese for me here. I just wanted to say that I did NOT do a system restore, I'l running a fresh install and restore is disabled



 Oct 8, 2013  
#48 mfo...@chromium.org The Cast Extension is also experiencing this issue (~500K users, not sure how many it  affects).

Sample from JS console:

storage.get: Failed to open database: Corruption: 2 missing files; e.g.: .../000035.sst
    at d.IC (chrome-extension://fmfcbgogabcbclcofgocippekhfcmgfj/background_script.js:1653:88)
    at HTMLDocument.<anonymous> (chrome-extension://fmfcbgogabcbclcofgocippekhfcmgfj/background_script.js:14258:58) 

Happens on Windows 31.0.1566.69







Cc: haibi...@chromium.org markdavi...@google.com dbbro...@chromium.org 






 Oct 8, 2013  
#49 cde...@gmail.com In the meanwhile i got it sorted by creating a new chrome profile;



 Oct 12, 2013  
#50 derj...@googlemail.com (No comment was entered for this change.)






Blocking: tampermonkey:168 






 Yesterday (19 hours ago)  
#51 bill.hil...@gmail.com Tampermonkey user; notified by TM script that "chrome profile was broken", and pointed to this page. 
Win7 Professional SP1 x64 /  81GB free on 500BG disk drive; latest update of TamperMonkey.  
Yes, I have used System Restore.. in the past week. An update to unrelated software was not ready so I backed it out via Windows System Restore. normal shutdowns and restarts since then. 
I use TamperMonkey for gaming and have other extensions active - Lucifers MOd and Spockholm Toolbar.  
Will post screenshots if needed. 
 

Comment 55 by kareng@google.com, Nov 8 2013

Labels: -M-32 M-33 MovedFrom-32
Moving all non essential bugs to the next Milestone.

Comment 56 by Deleted ...@, Nov 8 2013

Chrome pleeeease fix this. It's silly that every time I restart my computer I have to redownload scripts
I'm using chrome.storage.local to store actual information, not just preferences, and I'm getting furious reviews from people who've lost their data. I would like to create a fallback for my users(using the suggested chrome.extension.lastError method) where they can at least extract all data from a recent backup... but what storage technique to use if everything based on leveldb is unreliable?

Comment 58 by ngy...@gmail.com, Nov 12 2013

#57: The extension's localStorage

Comment 59 by cbu...@gmail.com, Nov 12 2013

Just FYI:  I used Windows 8.0 native utilities and took a full system image backup and then immediately restored it. Running Chrome after the restore I got an immediate error from one of the Chrome plugins, Tampermonkey I think, like: Chrome has damaged the chrome.storage.local area, please create the user profile.  (Not the real message.)

Point is though, a full system backup (which uses WSS) followed by a full restore effectively shouldn't have changed anything from the user side. (Internal WSS restore points are gone, files has been defragged I guess, and maybe the inodes  have changed numbers.  Don't know about that two -- but from the user standpoint "nothing's changed, everything is exactly the same as it was."

Things shouldn't break after a full backup/restore cycle, OR there should be some way to repair the damage and relink things.  An automatic detect/repair would be great, a manual repair procedure would work, and even a manual "try to extract, initialize, and then reimport my data" would be OK - but just a "He's Dead, Jim" isn't nice at all.
Project Member

Comment 60 by bugdroid1@chromium.org, Dec 17 2013

Labels: -M-33 MovedFrom-33
This issue has already been moved once and is lower than Priority 1,therefore removing mstone.
Cc: smckay@chromium.org tbreisacher@chromium.org
This is causing issues for some users of the Chrome Platform Analytics library.

https://github.com/GoogleChrome/chrome-platform-analytics/issues/13

We'd really appreciate seeing this fixed in M33. If folks don't have cycles, please point us in the right direction and we might be able to take a pass at this.
I started working on this and r222178 was the first patch in a series that I never had the chance to finish. From there, my intention was to make chrome.storage.local clear its database on deletion and chrome.storage.sync recover from the sync database.

Comment 63 by knee...@gmail.com, Dec 21 2013

Sounds great, thank you! :)

Merry Christmas!
I'll take up the work on this again.

Comment 65 Deleted

Reported:

"""
The behavior changed on Chrome Version 32.0.1700.76 m:

DB corruption now doesn't throw an error anymore but silently fails.

[3256:2244:0116/170053:WARNING:leveldb_value_store.cc(55)] Corruption: 1 missing files; e.g.: .../000005.ldb
[3256:2244:0116/170053:WARNING:settings_storage_quota_enforcer.cc(92)] Failed to get initial settings for quota: Corruption: 1 missing files; e.g.: .../000005.ldb

Is this intended?
"""

No, it will warn but it should also fail (as in chrome.runtime.lastError)? I wouldn't have expected this to change.

Comment 67 by derj...@gmail.com, Jan 16 2014

Nevermind, I found the error. :) I was confused because when this error can be seen then additionally to chrome.storage the WebSQL storage also won't work anymore.

Should I create a new issue for that or will this be fixed during the work for this issue?
Fixing this overall bug implies fixing #66.

Comment 69 Deleted

Comment 70 Deleted

Comment 71 Deleted

Comment 72 Deleted

Comment 73 Deleted

Comment 74 by Deleted ...@, Feb 11 2014

all the times make me angry!!!!
I'm working on this one; soon we should be able to do a sane recovery.  Of course, if the database is totally corrupted, all data will be lost - but at least it won't remain permanently broken.
Owner: rdevlin....@chromium.org
Hooray!

Comment 78 by Deleted ...@, Feb 17 2014

Google, please fix this error asap. I'm no longer able to scrobble my music from Google Music.

Comment 79 by niahi...@gmail.com, Feb 18 2014

unsubscribe

Comment 80 by Deleted ...@, Feb 21 2014

All of a sudden my chrome has this error! It is un-usable and I use it purposefully for work. 
Project Member

Comment 81 by bugdroid1@chromium.org, Feb 21 2014

------------------------------------------------------------------------
r252617 | rdevlin.cronin@chromium.org | 2014-02-21T19:39:55.663587Z

Changed paths:
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/policy_value_store.cc?r1=252617&r2=252616&pathrev=252617
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/sync_storage_backend.cc?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_quota_unittest.cc?r1=252617&r2=252616&pathrev=252617
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/sync_storage_backend.h?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/policy_value_store.h?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/leveldb_settings_storage_factory.h?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_storage_factory.h?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_storage_quota_enforcer.cc?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/sync_or_local_value_store_cache.cc?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_tests_unit.gypi?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/sync_or_local_value_store_cache.h?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_browser_extensions.gypi?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_sync_unittest.cc?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/storage_api.cc?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_test_util.h?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/storage_api.h?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_frontend.cc?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/leveldb_value_store_unittest.cc?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/test_extension_system.h?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/syncable_settings_storage.cc?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/weak_unlimited_settings_storage.cc?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/value_store.h?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_backend.cc?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/weak_unlimited_settings_storage.h?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/leveldb_settings_storage_factory.cc?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_backend.h?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_storage_quota_enforcer.h?r1=252617&r2=252616&pathrev=252617
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/local_storage_backend.cc?r1=252617&r2=252616&pathrev=252617
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/local_storage_backend.h?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_test_util.cc?r1=252617&r2=252616&pathrev=252617
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/storage_api_unittest.cc?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/leveldb_value_store.cc?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/testing_value_store.cc?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/test_extension_system.cc?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/leveldb_value_store.h?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/testing_value_store.h?r1=252617&r2=252616&pathrev=252617
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/syncable_settings_storage.h?r1=252617&r2=252616&pathrev=252617

Add a Restore() method to ValueStore and make StorageAPI use it
Take the first steps in preventing the Storage from getting and staying
corrupted. This lets us Restore() a ValueStore when it's corrputed, and tries to
do so in the StorageAPI calls.
This also refactors so that there's a Local and Sync SettingsBackend - this has
the added advantage of meaning we don't decorate the Local stores with Sync
behavior.
BUG= 261623 

Review URL: https://codereview.chromium.org/165223003
------------------------------------------------------------------------
Project Member

Comment 82 by bugdroid1@chromium.org, Feb 21 2014

------------------------------------------------------------------------
r252652 | rdevlin.cronin@chromium.org | 2014-02-21T21:44:35.686913Z

Changed paths:
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_storage_quota_enforcer.h?r1=252652&r2=252651&pathrev=252652
   D http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/local_storage_backend.h?r1=252652&r2=252651&pathrev=252652
   D http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/storage_api_unittest.cc?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_test_util.cc?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/storage_api.cc?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_test_util.h?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_frontend.cc?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/leveldb_value_store_unittest.cc?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/leveldb_value_store.cc?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/testing_value_store.cc?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/syncable_settings_storage.cc?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/weak_unlimited_settings_storage.cc?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/syncable_settings_storage.h?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/value_store.h?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/policy_value_store.cc?r1=252652&r2=252651&pathrev=252652
   D http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/sync_storage_backend.cc?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_backend.h?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_tests_unit.gypi?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_browser_extensions.gypi?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/sync_or_local_value_store_cache.h?r1=252652&r2=252651&pathrev=252652
   D http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/local_storage_backend.cc?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_sync_unittest.cc?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/storage_api.h?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/test_extension_system.cc?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/testing_value_store.h?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/leveldb_value_store.h?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/test_extension_system.h?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/weak_unlimited_settings_storage.h?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_backend.cc?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/leveldb_settings_storage_factory.cc?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_quota_unittest.cc?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/policy_value_store.h?r1=252652&r2=252651&pathrev=252652
   D http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/sync_storage_backend.h?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/leveldb_settings_storage_factory.h?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_storage_factory.h?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_storage_quota_enforcer.cc?r1=252652&r2=252651&pathrev=252652
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/sync_or_local_value_store_cache.cc?r1=252652&r2=252651&pathrev=252652

Revert of Add a Restore() method to ValueStore and make StorageAPI use it (https://codereview.chromium.org/165223003/)

Reason for revert:
Broke win unit_tests with StorageApiUnittest.RestoreStorage

Original issue's description:
> Add a Restore() method to ValueStore and make StorageAPI use it
> Take the first steps in preventing the Storage from getting and staying
> corrupted. This lets us Restore() a ValueStore when it's corrputed, and tries to
> do so in the StorageAPI calls.
> This also refactors so that there's a Local and Sync SettingsBackend - this has
> the added advantage of meaning we don't decorate the Local stores with Sync
> behavior.
> BUG= 261623 
> 
> Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=252617

TBR=mpcomplete@chromium.org,kalman@chromium.org,joaodasilva@chromium.org
NOTREECHECKS=true
NOTRY=true
BUG= 261623 

Review URL: https://codereview.chromium.org/175853002
------------------------------------------------------------------------

Comment 83 by Deleted ...@, Feb 23 2014

I keep getting a message about Pandora, storage and possible corruption. It references this error every time I open up my Chrome browser. It said to re-download the Chrome browser. Which I did. But I keep getting the message  How can I get this to stop? I'm not a computer genius. I've read through the above comments and I have no idea what to do.
Project Member

Comment 84 by bugdroid1@chromium.org, Feb 26 2014

------------------------------------------------------------------------
r253353 | rdevlin.cronin@chromium.org | 2014-02-26T04:57:50.024451Z

Changed paths:
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/weak_unlimited_settings_storage.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_backend.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/leveldb_settings_storage_factory.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_quota_unittest.cc?r1=253353&r2=253352&pathrev=253353
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/sync_storage_backend.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/policy_value_store.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/leveldb_settings_storage_factory.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_storage_factory.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_storage_quota_enforcer.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/sync_or_local_value_store_cache.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_storage_quota_enforcer.h?r1=253353&r2=253352&pathrev=253353
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/local_storage_backend.h?r1=253353&r2=253352&pathrev=253353
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/storage_api_unittest.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_test_util.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/storage_api.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_test_util.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_frontend.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/leveldb_value_store_unittest.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/leveldb_value_store.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/testing_value_store.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/syncable_settings_storage.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/weak_unlimited_settings_storage.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/value_store.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/syncable_settings_storage.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/policy_value_store.cc?r1=253353&r2=253352&pathrev=253353
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/sync_storage_backend.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_backend.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_tests_unit.gypi?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_browser_extensions.gypi?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/sync_or_local_value_store_cache.h?r1=253353&r2=253352&pathrev=253353
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/local_storage_backend.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_sync_unittest.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/storage_api.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/test_extension_system.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/testing_value_store.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/leveldb_value_store.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/test_extension_system.h?r1=253353&r2=253352&pathrev=253353

Resubmit: Add a Restore() method to ValueStore and make StorageAPI use it

Resubmit of: https://codereview.chromium.org/165223003/

TBR=mpcomplete@chromium.org
TBR=joaodasilva@chromium.org

---------------------------------
Add a Restore() method to ValueStore and make StorageAPI use it
Take the first steps in preventing the Storage from getting and staying
corrupted. This lets us Restore() a ValueStore when it's corrputed, and tries to
do so in the StorageAPI calls.
This also refactors so that there's a Local and Sync SettingsBackend - this has
the added advantage of meaning we don't decorate the Local stores with Sync
behavior.
BUG= 261623 

Review URL: https://codereview.chromium.org/176963006
------------------------------------------------------------------------
Project Member

Comment 85 by bugdroid1@chromium.org, Feb 26 2014

------------------------------------------------------------------------
r253353 | rdevlin.cronin@chromium.org | 2014-02-26T04:57:50.024451Z

Changed paths:
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/storage_api.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/test_extension_system.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/testing_value_store.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/leveldb_value_store.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/test_extension_system.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/weak_unlimited_settings_storage.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_backend.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_quota_unittest.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/leveldb_settings_storage_factory.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/policy_value_store.h?r1=253353&r2=253352&pathrev=253353
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/sync_storage_backend.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/leveldb_settings_storage_factory.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_storage_factory.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_storage_quota_enforcer.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/sync_or_local_value_store_cache.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_storage_quota_enforcer.h?r1=253353&r2=253352&pathrev=253353
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/local_storage_backend.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_test_util.cc?r1=253353&r2=253352&pathrev=253353
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/storage_api_unittest.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/storage_api.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_test_util.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_frontend.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/leveldb_value_store_unittest.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/testing_value_store.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/leveldb_value_store.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/syncable_settings_storage.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/weak_unlimited_settings_storage.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/syncable_settings_storage.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/value_store/value_store.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/policy_value_store.cc?r1=253353&r2=253352&pathrev=253353
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/sync_storage_backend.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_backend.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_tests_unit.gypi?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/sync_or_local_value_store_cache.h?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_browser_extensions.gypi?r1=253353&r2=253352&pathrev=253353
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/local_storage_backend.cc?r1=253353&r2=253352&pathrev=253353
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/extensions/api/storage/settings_sync_unittest.cc?r1=253353&r2=253352&pathrev=253353

Resubmit: Add a Restore() method to ValueStore and make StorageAPI use it

Resubmit of: https://codereview.chromium.org/165223003/

TBR=mpcomplete@chromium.org
TBR=joaodasilva@chromium.org

---------------------------------
Add a Restore() method to ValueStore and make StorageAPI use it
Take the first steps in preventing the Storage from getting and staying
corrupted. This lets us Restore() a ValueStore when it's corrputed, and tries to
do so in the StorageAPI calls.
This also refactors so that there's a Local and Sync SettingsBackend - this has
the added advantage of meaning we don't decorate the Local stores with Sync
behavior.
BUG= 261623 

Review URL: https://codereview.chromium.org/176963006
------------------------------------------------------------------------
Status: Fixed
Tentatively marking this one as fixed.

To summarize:
Old behavior let you get into an eternally broken state if the database somehow got corrupted. Only solution was to manually delete Extension's stored files.

New behavior will automatically correct any corruption by removing corrupted parts.  As much of the database as possible will be spared, but this _could_ result in its total deletion. This is decidedly better than the old behavior, because it will prevent any "eternally broken" states, and all extensions should continue to be usable.

Please note that this change will take awhile to percolate out to beta and stable.

Comment 87 by mas...@gmail.com, Mar 28 2014

Similar issue here. Thank you for fixing it!

Any idea how long "a while" might be? 1 month-ish? 3? 6? 12?
@mascip:

It depends on which channel you're in (Canary, Dev, Beta, Stable).  The fix should be in dev already, arrive in beta in mid-April, and in stable in mid-May (I think).
Can't wait for this fix to be in stable.  Thanks loads for working on it.
Out of curiosity, if parts are now discarded as corrupt, maybe there should be a previous instance of config kept from previous run and saved in case of corruption?

e.g: like session backup data, a localstorage.previousrun is created on each startup of chrome.  If corruption found load salvaged data (subtle prompt of data corruption and option of restore from .previousrun), otherwise create a .previousrun file and if the chrome instance fails, all is safe.
I doubt that this fix is really a fix... I can reproduce this "corruption" easily, using two PCs that both run chrome. At some point chrome starts to complain when I open an url from within an application (while chrome is already running), after that also opening chrome on the other PC starts to complain about a broken profile (four pop ups and 'waiting for ... plugins' for MINUTES). Very annoying :/
@nascent: We still have some work to do for figuring out the best ways to prevent corruption and truly recover from it.  We've thought about just backing it all up, but that's just not scalable, unfortunately (for instance, let's say a user has a few large games with the unlimited local storage permission, and we have to back up excessive amounts of data).  This patch was mostly to ensure that no one got left in an eternally broken state.  But we're still thinking about it. :)

@aleric.inglewood
Could you be more specific?  Provide any kind of logs, chrome version info, OS, etc?  A broken profile can be indicative of many different things, not just corrupt extension storage - are you sure it's this bug?

Comment 92 by Deleted ...@, May 2 2014

Can't wait for this fix to be in stable.  Thanks loads for working on it.
As the underlying problem gets addressed, which issue will be used for that conversation?

Comment 94 Deleted

Comment 95 Deleted

Comment 96 Deleted

I came here to add my vote for this issue, and I have done that.
Please send me email regarding the status of this issue when resolved.

Thanks, Sincerely;
Michael L. Burdett
michael.l.burdett@gmail.com

this is a very annoying flaw. I have been the victim of this multiple times already. so what i'm reading is to switch to a dev release?

Comment 99 by Deleted ...@, Sep 11 2014

Please fix this problem already!
I came here to add my vote for this issue.Be badly in need of solution.

Comment 101 by mas...@gmail.com, Sep 11 2014

I've had to change browser because of that, which was more annoying than
you'd think: i don't have all my beloved plugins in Firefox.

Please fix this...
Cc: -dgro...@chromium.org
This issue should be fixed.  If you're still seeing the problem, could you please include logs and chromium version?  Thanks!

Comment 104 by Deleted ...@, Nov 9 2014

 google fix it please. still have same bug.
chrome 38.0.2125.111

Comment 105 by Deleted ...@, Nov 26 2014

fix it, please

Comment 106 by Deleted ...@, Dec 28 2014

Please fix.
Thank you.
Today is 1-28-15, I've been having this problem for several weeks now!  The problem is not fixed!!!!  Please fix!!!!  Thank You!
Not, not even close, keep working. Getting the same error every week.
If you are still seeing this error, can you please include any logs, stack trace, etc?

Comment 110 by s...@woobox.com, Feb 23 2015

@rdevlin Where do you recommend I get these logs from? I have had this error twice since September. The only way I know is that Tampermonkey tells me and I have to restore the scripts.

I am a developer and can probably follow any instruction, but this bug doesn't actually crash the browser.
This bug refers to a crash/bug that an extension author was seeing when trying to use the storage API.  If this is still happening, it should show up in the developer tools of the extension's background page (assuming that's where it's used), which can be seen by inspecting it from the chrome://extensions page.

If you're not actually seeing this as a crash from using the storage api, it's possible it's a separate bug.  What behavior are you seeing?
I use Tampermonkey 3.10.60 and get the "broken profile" error about once a week.  I assume this is the error referred to here; Tampermonkey's FAQ linked me to this page, anyway.  If it is the same error, it is obviously NOT FIXED.  Please fix it!
Cc: -smckay@chromium.org
Due to the policy of not asking the user what to do with the files on his own computer, google just deleted my settings automatically.

I'm very pissed off right now.

Can you ask NSA to send me the most recent copy or perhaps you, google, have taken a copy without asking and can send me it?

You do not fix things by hiding the problem. Why not fix the bug that does the corruption?

Showing comments 15 - 114 of 114 Older

Sign in to add a comment