Attached code that triggers reproduction. To reproduce:

1) Open the attached file in a new tab
2) Open up Dev Tools
3) Go to the "Console" tab
4) Wait for "Starting round 1 / 1000" to show up
5) Go to the "Application" tab, and select the "Clear storage" item
6) Watch the Usage number go up indefinitely

To recover the wasted disk space.

1) Open a new tab (leave the repro tab open)
2) Go to chrome://indexeddb-internals
3) Find the ever-growing database (the origin should be file://, if the repro file was opened)
4) Click "Force close"
5) Go back to the tab that has the repro in it
6) Go to the "Console" tab of dev tools. An error should have shown up.
7) Go to the "Application" tab, and select the "Clear storage" item
8) Scroll to the bottom, click the "Clear site data" button


Note that the situation doesn't reproduce if instead of writing large records, we write small records with attached Blobs. In that case, once the Usage number in the Application tab gets to 2-2.5GB, it falls back down to ~400MB, and the cycle repeats.

Deleted: bug756754.html 2.2 KB

bug756754.html 2.2 KB View Download

The Blobs get leaked when unwrapping values. The (updated) attached repro can be modified to show that. chrome://blob-internals shows many Blobs of type application/vnd.blink-idb-value-wrapper. Some of them come from IndexedDB, some of them are paged out to disk.

Deleted: bug756754.html 2.9 KB

bug756754.html 2.9 KB View Download

I observed this on Chrome Canary and Chromium (master) on Mac and Linux. I did not observe on M60 Linux.

Also does not repro on M60 on Mac. On M60, the Blobs directory stays at ~750 files and 350MB.

This suggests that a more recent change in the Blob subsystem is interacting negatively with the value wrapping code.

Reducing priority, as the problem is not deployed to stable.

(For comparison, on M62 the Blobs directory gets to ~10GB by round 100.)

dmurph@: Any changes in the Blob subsystem between M60 and M62 that might make it easier for us to leak Blobs?

After chatting with dmurph@, I re-did the repro attempts involving Blobs. This entails setting kUseBlobs = true in the repro code, so we use Blobs directly, instead of having large record get auto-wrapped in Blobs.

The Blobs do not leak in M60, but do leak in M62. (I suspect this is what got me confused last night -- I used my M60 and M62 windows interchangeably, before I realized that the bug is version-dependent.)

So, there is a problem in how we keep track of deleted IndexedDB Blobs that are still referenced. At the same time, when using Blobs directly, chrome://blob-internals doesn't show a lot of entries, so M62 might also be leaking Blob handles in the IDB value unwrapping code.

Updated reproduction that doesn't require DevTools while running.

Deleted: bug756754.html 5.3 KB

bug756754.html 5.3 KB View Download

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e0433a1fcc97551dfb9fc1774411b9d2db273b75

commit e0433a1fcc97551dfb9fc1774411b9d2db273b75
Author: Daniel Murphy <dmurph@chromium.org>
Date: Tue Aug 22 23:01:42 2017

[BlobStorage] Adding DVLOG for debugging help

R: pwnall
Bug:  757142 ,  756754 
Change-Id: Id055826191299965eddf8640eb09c640ad3f58c3
Reviewed-on: https://chromium-review.googlesource.com/627019
Commit-Queue: Daniel Murphy <dmurph@chromium.org>
Reviewed-by: Victor Costan <pwnall@chromium.org>
Cr-Commit-Position: refs/heads/master@{#496487}
[modify] https://crrev.com/e0433a1fcc97551dfb9fc1774411b9d2db273b75/content/browser/blob_storage/blob_dispatcher_host.cc
[modify] https://crrev.com/e0433a1fcc97551dfb9fc1774411b9d2db273b75/storage/browser/blob/blob_storage_context.cc
[modify] https://crrev.com/e0433a1fcc97551dfb9fc1774411b9d2db273b75/storage/browser/blob/scoped_file.cc
[modify] https://crrev.com/e0433a1fcc97551dfb9fc1774411b9d2db273b75/storage/browser/blob/scoped_file.h
[modify] https://crrev.com/e0433a1fcc97551dfb9fc1774411b9d2db273b75/storage/browser/blob/shareable_file_reference.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ac35e5368b36263695abddad60257672791f184f

commit ac35e5368b36263695abddad60257672791f184f
Author: Daniel Murphy <dmurph@chromium.org>
Date: Tue Oct 03 01:28:12 2017

[IndexedDB] Fixed blob leaking and delayed deletion.

* Ensures blob cleanup is run on database close, instead of leaving
  undeleted old blobs on disk.
* Enforce a max time for the blob cleanup timer, as repeated blob
  operations made blobs never actually clean up.
* Notify the quota manager when blobs are cleaned up to update quota.

Bug:  756754 , 757142 
Change-Id: I62fd79f25aa80e08df4aa631ae40a6aebd2726e0
Reviewed-on: https://chromium-review.googlesource.com/658153
Commit-Queue: Daniel Murphy <dmurph@chromium.org>
Reviewed-by: Victor Costan <pwnall@chromium.org>
Reviewed-by: Joshua Bell <jsbell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#505908}
[modify] https://crrev.com/ac35e5368b36263695abddad60257672791f184f/content/browser/indexed_db/indexed_db_backing_store.cc
[modify] https://crrev.com/ac35e5368b36263695abddad60257672791f184f/content/browser/indexed_db/indexed_db_backing_store.h
[modify] https://crrev.com/ac35e5368b36263695abddad60257672791f184f/content/browser/indexed_db/indexed_db_backing_store_unittest.cc
[modify] https://crrev.com/ac35e5368b36263695abddad60257672791f184f/content/browser/indexed_db/indexed_db_browsertest.cc
[modify] https://crrev.com/ac35e5368b36263695abddad60257672791f184f/content/browser/indexed_db/indexed_db_context_impl.cc
[modify] https://crrev.com/ac35e5368b36263695abddad60257672791f184f/content/browser/indexed_db/indexed_db_context_impl.h
[modify] https://crrev.com/ac35e5368b36263695abddad60257672791f184f/content/browser/indexed_db/indexed_db_factory.h
[modify] https://crrev.com/ac35e5368b36263695abddad60257672791f184f/content/browser/indexed_db/indexed_db_factory_impl.cc
[modify] https://crrev.com/ac35e5368b36263695abddad60257672791f184f/content/browser/indexed_db/indexed_db_factory_impl.h
[modify] https://crrev.com/ac35e5368b36263695abddad60257672791f184f/content/browser/indexed_db/mock_indexed_db_factory.h

Since this is such an issue for Docs and is basically blocking auto-blobing, which is required for WASM, I'd like to merge this to 62. Ideally I could merge this to 61 but I think that might be a bit much and 62 is so soon.

This bug requires manual review: We are only 13 days from stable.
Please contact the milestone owner if you have questions.
Owners: amineer@(Android), cmasso@(iOS), bhthompson@(ChromeOS), abdulsyed@(Desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

As discussed over chat, this is a fairly large change impacting all platforms and we're only less than 2 weeks away from M62 Stable. Can you please explain what the full impact will be to users due to this bug? Is this affecting enterprise as well? 

Full impact documented in internal bug.

As for size of change - this is not a tiny change, I would say small. It changes when a cleanup operation is executed. I understand if was deemed too risky before it had some bake time in Canary - but I also think it should be fine to merge for tomorrow's beta release.  It's a pretty safe change, and includes testing (as well as manually testing from myself). 

I'm concerned that this issue will get worse for users that it effects, as all the old blobs can stay on the system (and continue to build up) until they run chrome with this change.

My recommendation is let's let it bake in canary a few more days before considering a merge. We can discuss the impact in internal bug. 

Since this has been baked in Canary for a few days now, does it still seem fine to merge for M62? Does it seem fine in M63 Dev and can we get this scenario tested by TE?

Approving merge to M62. Branch:3202. This has baked in Canary/Dev for a week and confirmed with dmurph@ that this is a small and safe merge. 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/18580919bc4e8a90a366459cad1f7b51771d674a

commit 18580919bc4e8a90a366459cad1f7b51771d674a
Author: Daniel Murphy <dmurph@chromium.org>
Date: Tue Oct 10 18:58:12 2017

[IndexedDB] Fixed blob leaking and delayed deletion.

* Ensures blob cleanup is run on database close, instead of leaving
  undeleted old blobs on disk.
* Enforce a max time for the blob cleanup timer, as repeated blob
  operations made blobs never actually clean up.
* Notify the quota manager when blobs are cleaned up to update quota.

TBR=dmurph@chromium.org

(cherry picked from commit ac35e5368b36263695abddad60257672791f184f)

Bug:  756754 , 757142 
Change-Id: I62fd79f25aa80e08df4aa631ae40a6aebd2726e0
Reviewed-on: https://chromium-review.googlesource.com/658153
Commit-Queue: Daniel Murphy <dmurph@chromium.org>
Reviewed-by: Victor Costan <pwnall@chromium.org>
Reviewed-by: Joshua Bell <jsbell@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#505908}
Reviewed-on: https://chromium-review.googlesource.com/710158
Reviewed-by: Daniel Murphy <dmurph@chromium.org>
Cr-Commit-Position: refs/branch-heads/3202@{#638}
Cr-Branched-From: fa6a5d87adff761bc16afc5498c3f5944c1daa68-refs/heads/master@{#499098}
[modify] https://crrev.com/18580919bc4e8a90a366459cad1f7b51771d674a/content/browser/indexed_db/indexed_db_backing_store.cc
[modify] https://crrev.com/18580919bc4e8a90a366459cad1f7b51771d674a/content/browser/indexed_db/indexed_db_backing_store.h
[modify] https://crrev.com/18580919bc4e8a90a366459cad1f7b51771d674a/content/browser/indexed_db/indexed_db_backing_store_unittest.cc
[modify] https://crrev.com/18580919bc4e8a90a366459cad1f7b51771d674a/content/browser/indexed_db/indexed_db_browsertest.cc
[modify] https://crrev.com/18580919bc4e8a90a366459cad1f7b51771d674a/content/browser/indexed_db/indexed_db_context_impl.cc
[modify] https://crrev.com/18580919bc4e8a90a366459cad1f7b51771d674a/content/browser/indexed_db/indexed_db_context_impl.h
[modify] https://crrev.com/18580919bc4e8a90a366459cad1f7b51771d674a/content/browser/indexed_db/indexed_db_factory.h
[modify] https://crrev.com/18580919bc4e8a90a366459cad1f7b51771d674a/content/browser/indexed_db/indexed_db_factory_impl.cc
[modify] https://crrev.com/18580919bc4e8a90a366459cad1f7b51771d674a/content/browser/indexed_db/indexed_db_factory_impl.h
[modify] https://crrev.com/18580919bc4e8a90a366459cad1f7b51771d674a/content/browser/indexed_db/mock_indexed_db_factory.h

It seems like we're still leaking Blobs when using large value wrapping. The problem can be reproduced by patching in https://crrev.com/c/712555 and navigating to the attached page, which must be served off http(s):// (not file://).

dmurph@: Can you please take a look?

Deleted: bug756754.html 5.3 KB

bug756754.html 5.3 KB View Download

When blobs are turned on, then this no longer leaks - so it looks like a bug in auto-wrapping.

More specifically, I see that the unwrapped blobs are NEVER deleted, no matter what, so we must have a leak in the deserialization code there somewhere.

I tried reverting Marijn's change, which added a couple more references to blobs in stuff like SerializedScriptValue, but that didn't help.

Let me know if you want me to poke more at your code - I thought you might be better at reviewing this now though.  I have DVLOGs in good spots, so try:

--vmodule=indexed_db*=1,shareable*=1,scoped_file*=1 to see when blobs are deleted, file references are deleted, etc.

This ended up being fixed by https://crrev.com/c/755822