Project: chromium Issues People Development process History Sign in
New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Issue 58985 Unlimited Storage permission should apply to Local Storage
Starred by 51 users Reported by sethladd@google.com, Oct 13 2010 Back to list
Status: WontFix
Owner: ----
Closed: Dec 2010
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Feature

Restricted
  • Only users with Commit permission may comment.



Sign in to add a comment
The unlimitedStorage permission should apply to Local Storage as well.  The unlimited storage permission is a huge draw for apps and extensions, and is currently only supported by Web SQL DB and App Cache.

Similar to http://code.google.com/p/chromium/issues/detail?id=49993
 
Labels: -Type-Bug Type-Feature
Comment 3 by jorlow@chromium.org, Oct 13 2010
Labels: -Area-Undefined Area-WebKit
The current implementation of LocalStorage doesn't scale very well.  Although this fix wouldn't be too hard, fixing the scaling problem (and thus having larger uses of LocalStorage not be a problem) is non-trivial.  A related (and frankly higher priority item) is probably to make LocalStorage support the structured clone algorithm.
Labels: not-extensions
Comment 5 by karen@chromium.org, Dec 6 2010
Labels: Mstone-X
We, as developers, NEED unlimited storage in localStorage. There are already plenty of localStorage wrappers (along with base64 encoding) that allow developers to store huge blogs of information via JSON. I've personally built entire web apps with just localStorage and wrappers like storageLocker, so i know it's possible, but we need the ability to, at least in apps, allow for unlimited storage OR ask to allow unlimited storage. 

How else are you going to sell this whole "Google Chrome *App* Store" if all the apps *have* to have an external DB or use the overly complicated Web SQL DB? C'mon guys.
We don't want people to store large amounts of information in LocalStorage.  The API can block a page (and any other pages in your same renderer process) while it loads LocalStorage into memory because it's a synchronous API.  Honestly, we'd just plain not support it if we could, but too many developers/sites rely on it.  So this is the compromise.

But clearly we don't want you loading hundreds of megs into memory and thus blocking everything in the web browser for extended periods of time.  

IndexedDB is a bit easier than WebSQLDatabase and will be ready for prime time before long.  WebSQLDatabase isn't _that_ complicated.  FileSystem is really the right place to store big files (which is what it sounds like you're doing) and is available in Chrome 9.  I suggest you look at these alternatives.

I'm tempted to mark this as a WontFix...
Hmm... maybe i read the spec wrong, but localStorage doesn't have to be synchronous, i think thats something browser vendors just decided, not the spec.

And IndexedDB is a lot easier, and im excited for that, but WebSQLDatabase is a little complicated for a front end developer. I personally work with MongoDB & MySQL, but I'm looking out for the majority of developers. MongoDB, for example, is easy and designers and front end devs can figure it out, but do we really want to make something you need to learn SQL to store a couple of images or a video not to mention since we are moving to a more NoSQL world and apps? Seems just plain stupid. But yes, IndexedDB will be easier and I'll be looking for that.

Just a shame that the greatness of what localStorage could be was shot down because of an internal dislikes for it. If im not mistaken, "client-side" just means not on an external server... It's an excellent candidate for storing base64 encoded images for example.

Just too bad we couldn't have gone with a NoSQL style storage system rather than back to SQL. Yes, there are many use cases for SQL, but storing binary data is *not* one of them.

Oh well.
setItem is the only method that can be somewhat synchronous, but there are still some parts of it that need to be.  Clealry something like 'getItem' whose return value is synchronous.  I.e. it can't return and scripts can't continue running until it returns a result.

I don't know what you mean by "internal dislikes".  The fact that it's synchronous is pretty much our entire reason for disliking it.  IndexedDB and WebSQLDatabase are "client-side" too.

IndexedDB is "NoSQL".  And I think you'll see it become fairly widely adopted over the next year.
All i was saying was that they way it's built and how it works isn't up to the spec. 

You *could* have built it to be some more async i.e. getItem scans items and returns for the exact match and returns it. Therefore, if you have some massive string e.g. "abcdefghijklmnopqrstuvwxyz" and a user does a getItem on "1234", once it hits the first character of that "abc..." results it just skips it, sees "1234" and then returns that one. How slow can that possibly be? I guess if it were Gigs of data, but shouldn't that be up to US, the developers?

I mean... it's only one result we are looking for, and it's on the client side. Once one is found, stop searching. This can very much be async. The loading time of a base64 encoded image can't possibly be much slower than storing an image in a folder on my desktop and calling that image from a HTML file on my desktop... can it? Not to mention you can take advantage of some very basic caching, check if it's been modified since last getItem, if not, returned cached result.

And i know IndexedDB, WebSQLDatabase, and localStorage are all client side... that's my entire point. If it's client side, you guys can store it however you want, and it's gotta be ultra fast to grab the data... it's *client* side.

And, also, i know IndexedDB is NoSQL and that's why im excited, but localStorage is more widely supported for one and it's even simpler for basic apps. That's all. I just can't see how slow it can possibly be if it was built in a async fashion.
Status: WontFix
We could build it a bit differently sure, but it'd still be blocking rendering/ui on disk seeks.  Personally I think doing one big block at the beginning vs. many smaller ones as we run isn't really that much worse.  If you want to make this better, feel free to take a shot and send patches.

But either way, I don't think we want to make the 5mb limit any higher under any circumstance.
+ a few others who might be interested in this
Comment 13 by thd...@gmail.com, Dec 16 2010
Many sources that say Chrome stores strings in localStorage as UTF-16. Is this true or disinformation? If it's a fact, then the actual storage limit is only 2.5MB; can the developer have the option to store strings in UTF-8 format?

You say that localStorage doesn't scale well -- is 5MB the threshold when it begins to perform poorly? If not, then can you please increase the limit to the point where it actually performs poorly.

How hard would it be (technically and politically) to change the localstorage API from a synchronous to an asynchronous architecture?
I'd be quite sad if it uses UTF-16 and only gives us 2.5MB...
Comment 15 by jorlow@google.com, Dec 17 2010
Yes, it's UTF-16 and thus only gives you 2.5MB if you're only storing UTF8 strings in it.  I don't think it's practical to change that though.

The performance issue gets worse as more information is stored.  5MB is what the spec suggests and thus far we've been scared to make this less.  If anything I think we should decrease it though.  Especially as IndexedDB become more ready for prime time.

IndexedDB is basically the async version of LocalStorage...with additional features.  It's a LOT more powerful which also means it's less simple, but we can't have a different API for every combination of power vs. simplicity people want.  JavaScript libraries (like Lawnchair) should be able to help in this department.  (If someone else doesn't beat me to it, I'll probably write an IndexedDB backend for Lawnchair at some point.)

I hope that helps and makes sense.
Figures... 2.5MBs. I can now store. That's what, 250,000 characters? At that size it's now almost entirely basically a replacement for cookies.

If only it were built asynchronous instead. Thatd for one fix the "loading issue" you say because:

A) You could simply have anything on initial page load that the script calls *after* or at the same time as the site's "GUI"

B) On scripts loaded on an event, i.e. clicking a button, the user will expect a pause, but again, i dont get why you say it HAS to pause the whole page. If i call something via AJAX or i load an image dynamically it doesnt lock everything up, why does this have to?
2,500,000 characters.

I agree that it having been built async would have worked around the entire problem.  It's a shame no one caught this until it was too late.  :-/
As thdoan originally asked though

"How hard would it be (technically and politically) to change the localstorage API from a synchronous to an asynchronous architecture?"
The web platform does not need yet another storage type.  Use libraries like LawnChair.  There is a 0% chance that we and/or other browser vendors will do another API that's somewhere between LocalStorage and IndexedDB.  There's no point.

Anyway, this is really off topic for this bug.  So please let it rest or bring it up on the chromium-html5 mailing list.
I don't think any of us want another platform... I'm positive none of us asked that.

I think the original asker, me, and thdoan are asking for building Chrome apps. So other browsers wouldnt matter, lawnchair wouldnt make sense since theres no need for it to work across browsers, and the WebSQL API allows it. I don't think it's that crazy to ask for more storage for locally hosted apps.
This is not the right place for this discussion.  chromium-discuss is probably right, actually.  Feel free to bring it up there and cc me.  But this is not the right place.
Clear out the link to this bug on this page:
http://code.google.com/chrome/extensions/manifest.html#permissions

Huh? Thats totally unrelated? We're not talking about extensions at all.
Project Member Comment 24 by bugdroid1@chromium.org, Oct 13 2012
Labels: Restrict-AddIssueComment-Commit
This issue has been closed for some time. No one will pay attention to new comments.
If you are seeing this bug or have new data, please click New Issue to start a new bug.
Project Member Comment 25 by bugdroid1@chromium.org, Mar 11 2013
Labels: -Area-WebKit Cr-Content
Project Member Comment 26 by bugdroid1@chromium.org, Apr 6 2013
Labels: -Cr-Content Cr-Blink
Sign in to add a comment