New issue
Advanced search Search tips

Issue 735269 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Mar 2018
Cc:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 3
Type: Bug

Blocking:
issue 750751



Sign in to add a comment

Streamline leveldb MDPs

Project Member Reported by dskiba@chromium.org, Jun 20 2017

Issue description

There are couple of MDPs that report memory usage of some leveldb databases:

1. LevelDBDatabaseImpl::OnMemoryDump() reports components/leveldb databases ("leveldb/mojo)
2. LevelDB::OnMemoryDump() reports components/leveldb_proto databases (leveldb/leveldb_proto)

However, those MDPs cover only a portion of all leveldb databases. crrev.com/2855953002 and follow-up CL adds a way to report all leveldb databases.

After those CLs land we'll need to revert / refactor existing leveldb MDPs.
 

Comment 1 by dskiba@chromium.org, Jun 21 2017

After the CL lands the plan is to:

1. Revert leveldb_proto MDP.

2. Use DB pointers as IDs in the leveldatabase MDP, and transfer / use them to create proper allocation edges in mojo MDP.

ssid@, I think you were going to do #2?

Comment 2 by ssid@chromium.org, Jun 21 2017

The plan is just to change the edges. Just add 
"AddSuballocation(client_dump, "leveldatabase/<correct_id>")"
call to each of the dump providers and remove the
"AddSuballocation(client_dump, system_allocator)"
calls from the client's OnMemoryDump.

If you want me to do this change, I can do it.

Comment 3 by dskiba@chromium.org, Jun 21 2017

I just realized that while right now leveldatabase MDP just reports "approximate-memory-usage", in the future it will probably do more, for example report all live iterators and their sizes. In case of in-memory databases it can report size taken by in-memory env (NewMemEnv).

So we need to remove all size reporting from existing MDPs and treat then only as means to report additional data (which leveldatabase MDP lacks).
One more place that reports leveldb databases:

LevelDBDatabase::OnMemoryDump() from content/browser/indexed_db/leveldb/leveldb_database.cc reports "leveldb/index_db" values.

I believe these 3 (this one plus 2 others from the first message) cover all places that report "leveldb/" values.
Cc: cmumford@chromium.org
Blocking: 750751
 Issue 754479  has been merged into this issue.
Labels: LowMemory
Owner: cmumford@chromium.org
Status: Started (was: Untriaged)
Project Member

Comment 9 by bugdroid1@chromium.org, Aug 15 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ae996c1b408b38064eb44b61f5f41b79cb0c6942

commit ae996c1b408b38064eb44b61f5f41b79cb0c6942
Author: Chris Mumford <cmumford@chromium.org>
Date: Tue Aug 15 21:43:00 2017

Revert "[tracing] Memory dump provider for leveldb_proto"

Reporting to memory-infra was added when DBTracker was introduced
in #483507. Reverting tracing to eliminate double-counting.

This reverts commit e4aba36676c616489620442d8ae6aa88931e680c.

Bug:  735269 
Change-Id: Ie20015f628a393dc3e540864006c622ea8d2c12c
Reviewed-on: https://chromium-review.googlesource.com/612361
Reviewed-by: Tommy Nyquist <nyquist@chromium.org>
Commit-Queue: Chris Mumford <cmumford@chromium.org>
Cr-Commit-Position: refs/heads/master@{#494559}
[modify] https://crrev.com/ae996c1b408b38064eb44b61f5f41b79cb0c6942/components/leveldb_proto/leveldb_database.cc
[modify] https://crrev.com/ae996c1b408b38064eb44b61f5f41b79cb0c6942/components/leveldb_proto/leveldb_database.h
[modify] https://crrev.com/ae996c1b408b38064eb44b61f5f41b79cb0c6942/components/leveldb_proto/proto_database_impl_unittest.cc

Owner: ssid@chromium.org
Assigning to ssid@ as per our discussion today for the remaining work on LevelDBDatabaseImpl.

Comment 11 by ssid@chromium.org, Sep 5 2017

The plan here is to change how the existing leveldb databases are reported:

1. All the databases will be reported by LevelDB tracker under "leveldatabase" column.
2. IndexedDB and DOMStorage will be reported under "renderer_storage" column, and sub-allocated from the right one under "leveldatabase" column.
3. Value Store will be reported under "extensions" column, and sub-allocated from the right one under "leveldatabase" column.

Comment 12 Deleted

Comment 13 by lalitm@google.com, Sep 6 2017

ssid@ I can take a look at this one as my starter as you suggested. Thanks!
Project Member

Comment 14 by bugdroid1@chromium.org, Sep 6 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/48316b5c56588b2ce577ed344be238c36da327a4

commit 48316b5c56588b2ce577ed344be238c36da327a4
Author: Siddhartha <ssid@chromium.org>
Date: Wed Sep 06 21:35:37 2017

LevelDatabaseImpl does not create allocator dumps under "leveldb"

LevelDatabaseImpl just uses the dump created by LevelDB tracker to create
global dump edges.

BUG= 735269 

Change-Id: If4d87635bd69704460a20e2a8be17f3dc773e2c6
Reviewed-on: https://chromium-review.googlesource.com/644498
Commit-Queue: Siddhartha S <ssid@chromium.org>
Reviewed-by: Elliot Glaysher <erg@chromium.org>
Reviewed-by: Ken Rockot <rockot@chromium.org>
Reviewed-by: Chris Mumford <cmumford@chromium.org>
Reviewed-by: Michael Nordman <michaeln@chromium.org>
Cr-Commit-Position: refs/heads/master@{#500097}
[modify] https://crrev.com/48316b5c56588b2ce577ed344be238c36da327a4/components/leveldb/leveldb_database_impl.cc
[modify] https://crrev.com/48316b5c56588b2ce577ed344be238c36da327a4/content/browser/dom_storage/session_storage_database.cc
[modify] https://crrev.com/48316b5c56588b2ce577ed344be238c36da327a4/content/browser/indexed_db/leveldb/leveldb_database.cc
[modify] https://crrev.com/48316b5c56588b2ce577ed344be238c36da327a4/extensions/browser/value_store/leveldb_value_store.cc
[modify] https://crrev.com/48316b5c56588b2ce577ed344be238c36da327a4/third_party/leveldatabase/env_chromium.cc
[modify] https://crrev.com/48316b5c56588b2ce577ed344be238c36da327a4/third_party/leveldatabase/env_chromium.h

Project Member

Comment 15 by bugdroid1@chromium.org, Sep 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e582f03635a9ec494d1c7798e57638998e456c1b

commit e582f03635a9ec494d1c7798e57638998e456c1b
Author: Lalit Maganti <lalitm@google.com>
Date: Mon Sep 11 10:44:01 2017

Change dump keys for LevelDB utilising MDPs

Changes the keys of each as specified in  bug 735269 .

Bug:  735269 
Change-Id: If202f70ba436d1b619065550eb7d349e8aea10c7
Reviewed-on: https://chromium-review.googlesource.com/652372
Reviewed-by: Devlin <rdevlin.cronin@chromium.org>
Reviewed-by: Michael Nordman <michaeln@chromium.org>
Reviewed-by: Siddhartha S <ssid@chromium.org>
Commit-Queue: Lalit Maganti <lalitm@google.com>
Cr-Commit-Position: refs/heads/master@{#500879}
[modify] https://crrev.com/e582f03635a9ec494d1c7798e57638998e456c1b/base/trace_event/memory_infra_background_whitelist.cc
[modify] https://crrev.com/e582f03635a9ec494d1c7798e57638998e456c1b/content/browser/dom_storage/dom_storage_area.cc
[modify] https://crrev.com/e582f03635a9ec494d1c7798e57638998e456c1b/content/browser/dom_storage/dom_storage_context_impl.cc
[modify] https://crrev.com/e582f03635a9ec494d1c7798e57638998e456c1b/content/browser/dom_storage/local_storage_context_mojo.cc
[modify] https://crrev.com/e582f03635a9ec494d1c7798e57638998e456c1b/content/browser/dom_storage/session_storage_database.cc
[modify] https://crrev.com/e582f03635a9ec494d1c7798e57638998e456c1b/content/browser/indexed_db/leveldb/leveldb_database.cc
[modify] https://crrev.com/e582f03635a9ec494d1c7798e57638998e456c1b/extensions/browser/value_store/leveldb_value_store.cc

Status: Fixed (was: Started)

Sign in to add a comment