PS: i tried running the test may times on a same specs device iphone 5s with ios 9.2 locally and it's working every time

I think I know what is going on. That CL changed Backtrace::kMaxFrameCount from 12 to 24. Backtrace is included into AllocationContext, which is included into AllocationRegister::Allocation, which in turn included into AllocationRegister::Cell. Default AllocationRegister constructor mmaps memory for 2621440 cells (kNumBuckets * kNumCellsPerBucket), and on 64-bit system each cell is 424 bytes (StackFrame: 16, Backtrace: 392, AllocationContext: 400, Allocation: 416, Cell: 424), so AllocationRegister tries to mmap ~1 GiB and sometimes fails.

I saw the same issue on Android x32, where attempt to mmap ~500 MiB frequently fails.

+primiano@

I think we can shrink Backtrace in half by storing types for all frames separately from values - we have just three types, so we need just 2 bits for each, i.e. 48 bits for all frames, instead of 24 * 8 bytes.

Another thing I can think of is to dedup backtraces in AllocationRegister - so it will internally have a separate hash map {Backtrace -> ID} and will store ID in its Allocation structure.

FYI

As a short term solution, in order to not leave bots in a flaky state, can we just use the explicit ctor of AllocationRegister in the unittest and use it a lower number of cells?

As a longer term solution:
1) do we really use those 500 MB? I think should be pretty easy to find out (look at the watermark of AllocationRegister). if not we can just halve the size of the allocation register and move on with more important stuff.

2) If we really need those, using a bitmap for the types as you say might be a good workaround, but that would make the code more complicated. I'd avoid unless needed.

> so it will internally have a separate hash map {Backtrace -> ID} and will store ID in its Allocation structure.
Nope this will make things super complicated. SO far the allocation register has been designed to not use any dynamic memory (other than the mmap). If we start putting hashmaps everything will become hard++. Let's not go there.

I'd just half the size of the register, unless we really need it that big. Should be easy to find out adding few lines of debugging.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/260bd00f1c3636b8596172a9c16868f078edad78

commit 260bd00f1c3636b8596172a9c16868f078edad78
Author: dskiba <dskiba@google.com>
Date: Thu May 05 05:43:01 2016

[tracing] Reduce number of buckets in AllocationRegister.

Recent changes caused AllocationRegister::Cell to grow 4x, and
with 0x40000 buckets it was allocating ~1 GiB on 64-bit systems. That
caused flakiness of AllocationRegister unittests on 64-bit iOS bots.
Also 32-bit Android sometimes refuses to mmap ~512 MiB when
--enable-heap-profiling flag is used.

BUG= 608354 

Review-Url: https://codereview.chromium.org/1954443002
Cr-Commit-Position: refs/heads/master@{#391767}

[modify] https://crrev.com/260bd00f1c3636b8596172a9c16868f078edad78/base/trace_event/heap_profiler_allocation_register.h

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/db346b18cb3a494e21d2c4662641eb33e1584212

commit db346b18cb3a494e21d2c4662641eb33e1584212
Author: primiano <primiano@chromium.org>
Date: Thu Jun 02 18:55:06 2016

tracing: Reduce memory usage of heap_profiler_allocation_register_unittest.cc

This test was causing flakiness on low-memory devices as it tries to allocate
~600 MB of virtual memory. Turns out the test doesn't really need it all that.
Reducing the number of cells allocated to 54 MB.

BUG= 608354 

Review-Url: https://codereview.chromium.org/2037603002
Cr-Commit-Position: refs/heads/master@{#397484}

[modify] https://crrev.com/db346b18cb3a494e21d2c4662641eb33e1584212/base/trace_event/heap_profiler_allocation_register_unittest.cc