New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 709243 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Last visit 26 days ago
Closed: Apr 2017
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 3
Type: Bug



Sign in to add a comment

cc::PropertyTree's usage of unordered_map is inefficient

Project Member Reported by brettw@chromium.org, Apr 6 2017

Issue description

In memory profiling //cc/trees/property_tree.h is identified as having allocated a nontrivial amount of memory over time. This memory is not persistent and it's not a leak, but it does a lot of allocations that are quickly freed.

This object has two data structures inside it, a vector, and a int->int std::unordered_map that maps IDs to indices in the vector. In a brief memory run on desktop Linux, the vector in this class allocated cumulatively 66K (not at the same time), while the unordered_map allocated cumulatively 1279K. Given that these data structures correspond to each other, the unordered map is being very wasteful.

I histogrammed the size of these data structures in the destructor of the property tree and compared the size of the map to the bucket count of the unordered_map:

Histogram: PropertyTree.BucketCount recorded 3323 samples, mean = 8.3 (flags = 0x41)
0   ... 
8   ------------------------------------------------------------------------O (3303 = 99.4%) {0.0%}
9   ... 
60  O                                                                         (20 = 0.6%) {99.4%}
67  ... 


Histogram: PropertyTree.Size recorded 3323 samples, mean = 0.1 (flags = 0x41)
0   ------------------------------------------------------------------------O (3228 = 97.1%)
1   O                                                                         (20 = 0.6%) {97.1%}
2   O                                                                         (12 = 0.4%) {97.7%}
3   O                                                                         (21 = 0.6%) {98.1%}
4   O                                                                         (4 = 0.1%) {98.7%}
5   O                                                                         (6 = 0.2%) {98.9%}
6   O                                                                         (8 = 0.2%) {99.0%}
7   O                                                                         (0 = 0.0%) {99.3%}
8   O                                                                         (4 = 0.1%) {99.3%}
9   O                                                                         (0 = 0.0%) {99.4%}
10  O                                                                         (12 = 0.4%) {99.4%}
11  O                                                                         (0 = 0.0%) {99.8%}
12  O                                                                         (4 = 0.1%) {99.8%}
14  O                                                                         (4 = 0.1%) {99.9%}
16  ... 

An unordered_map is a std::vector of buckets that are each pointers into a std::list. Therefore, every integer insertion becomes allocating a list node on the heap (two pointers plus the integer), plus an additional vector of pointers with a certain load factor.

Given the sizes of these structures, the performance of these structures will be dominated by the allocation overhead of the list nodes.

It looks like a flat_map (sorted list of nodes) might be a better fit for this data structure.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Apr 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fb21662ee5c04e2bfa9de9e4b9a86e5f101c524e

commit fb21662ee5c04e2bfa9de9e4b9a86e5f101c524e
Author: brettw <brettw@chromium.org>
Date: Tue Apr 11 04:01:18 2017

Convert cc::PropertyTree to use a flat_map.

These maps are typically very small (see bug) and the unordered_map has a large memory overhead (both in terms of space and # of allocations).

BUG= 709243 
CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_trusty_blink_rel

Review-Url: https://codereview.chromium.org/2801603010
Cr-Commit-Position: refs/heads/master@{#463524}

[modify] https://crrev.com/fb21662ee5c04e2bfa9de9e4b9a86e5f101c524e/cc/trees/property_tree.cc
[modify] https://crrev.com/fb21662ee5c04e2bfa9de9e4b9a86e5f101c524e/cc/trees/property_tree.h

Comment 2 by brettw@chromium.org, Apr 11 2017

Status: Fixed (was: Started)

Sign in to add a comment