moving to palmer who owns PA

I can't reproduce the problem on either Linux or Windows. However, these tests are 'aggressive' and OOM-y by nature; we've had issues in the past.

What's your hardware? Perhaps my machines just have so much RAM that I can't happen upon the problem. I'm on 64-bit Z840s with 64 GiB RAM for both Linux and Windows.

Z840, running 32-bit build though.

Ahh. What are your build flags? So I can do the exact same as you.

OK, I can reproduce this now (on Linux, and probably all 32-bit platforms):

~/chromium/src $ ./out/Default/base_unittests --gtest_filter=PartitionAllocTest.*
...
1 test crashed:
    PartitionAllocTest.DumpMemoryStats (../../base/allocator/partition_allocator/partition_alloc_unittest.cc:1440)
Tests took 15 seconds.

~/chromium/src $ cat out/Default/args.gn 
target_cpu = "x86"
is_debug=true
is_component_build=true
use_goma = true

I suspect the problem might be that PA is running out of address space (not actual memory):. From https://cs.chromium.org/chromium/src/base/allocator/partition_allocator/PartitionAlloc.md:

"""PartitionAlloc guarantees that different partitions exist in different regions of the process' address space. When the caller has freed all objects contained in a page in a partition, PartitionAlloc returns the physical memory to the operating system, but continues to reserve the region of address space. PartitionAlloc will only reuse an address space region for the same partition."""

However, it might be the case that the tests leak either memory or address space. I find the function `TestSetup` somewhat suspicious:

```
  // Zero the allocator structs to clear out traces
  // from previous test.
  memset(&allocator, 0, sizeof(allocator));
  memset(&generic_allocator, 0, sizeof(generic_allocator));

```

Working on a refactor that, if it succeeds, might help.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3cc4bc0d06591e905b80959e4273666fbcc06a3f

commit 3cc4bc0d06591e905b80959e4273666fbcc06a3f
Author: palmer <palmer@chromium.org>
Date: Fri May 26 21:18:48 2017

Refactor Partition Alloc unit tests.

Use testing::Test subclasses and TEST_F(...) instead of TEST(...) with
dependencies on global state.

BUG= 722911 

Review-Url: https://codereview.chromium.org/2903393002
Cr-Commit-Position: refs/heads/master@{#475131}

[modify] https://crrev.com/3cc4bc0d06591e905b80959e4273666fbcc06a3f/base/allocator/partition_allocator/partition_alloc_unittest.cc

Note: this refactor doesn't fix the bug.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/61538ef94febd23ee6ea84f0851489961661a6db

commit 61538ef94febd23ee6ea84f0851489961661a6db
Author: Erik Chen <erikchen@chromium.org>
Date: Wed Sep 20 21:41:45 2017

Refactor PartitionAllocator to allow local instances.

Previously, PartitionAllocator and related classes were trivially constructible,
but relied on 0-initialization of internal fields. This meant that only global
instances of PartitionAllocator could be used. This makes testing
PartitionAllocator difficult.

This CL adds constructors to PartitionAllocator classes, and switches globals to
LazyInstances, but otherwise should not cause behavior change. The expected
performance impact should be unnoticeable, as PartitionAllocator already has
non-trivial initialization code. This CL should not affect performance of the
allocation path itself.

Bug: 765735,  722911 
Change-Id: I8cd456772a3d71d316161a67d966e89fb9b81a63
Reviewed-on: https://chromium-review.googlesource.com/673726
Commit-Queue: Erik Chen <erikchen@chromium.org>
Reviewed-by: Kentaro Hara <haraken@chromium.org>
Cr-Commit-Position: refs/heads/master@{#503257}
[modify] https://crrev.com/61538ef94febd23ee6ea84f0851489961661a6db/base/allocator/partition_allocator/address_space_randomization.cc
[modify] https://crrev.com/61538ef94febd23ee6ea84f0851489961661a6db/base/allocator/partition_allocator/page_allocator.cc
[modify] https://crrev.com/61538ef94febd23ee6ea84f0851489961661a6db/base/allocator/partition_allocator/partition_alloc.cc
[modify] https://crrev.com/61538ef94febd23ee6ea84f0851489961661a6db/base/allocator/partition_allocator/partition_alloc.h
[modify] https://crrev.com/61538ef94febd23ee6ea84f0851489961661a6db/base/allocator/partition_allocator/partition_alloc_unittest.cc
[modify] https://crrev.com/61538ef94febd23ee6ea84f0851489961661a6db/base/allocator/partition_allocator/spin_lock.cc
[modify] https://crrev.com/61538ef94febd23ee6ea84f0851489961661a6db/base/allocator/partition_allocator/spin_lock.h
[modify] https://crrev.com/61538ef94febd23ee6ea84f0851489961661a6db/third_party/WebKit/Source/platform/wtf/allocator/Partitions.cpp
[modify] https://crrev.com/61538ef94febd23ee6ea84f0851489961661a6db/third_party/WebKit/Source/platform/wtf/allocator/Partitions.h

 Issue 803704  has been merged into this issue.

Still a problem, repros 100% on my Win10 machine.

These days the error is :

$ out\Release\base_unittests.exe --gtest_filter=PartitionAllocTest.*

(...)
[19/24] PartitionAllocTest.DumpMemoryStats (CRASHED)
Received fatal exception 0xe0000008"
PartitionAllocTest.DumpMemoryStats (CRASHED)
(...)

This is the only test failing consistently when running all base_unittests locally (e.g. out\Release\base_unittests.exe --test-launcher-jobs=42), would be nice to address.

Any updates? Still failing in every single local run.

I also see this on every run. I'm running a 32-bit version of base_unittests. It always succeeds on the retry.

@palmer - Chris, brucedawson asked me to look at this, so I'm going to have a whack at it. 

removing this test PartitionAllocTest.RepeatedReturnNullDirect fixes the issue, so I suspect there's an issue with that test.

OK, great. Please CC me on the CL. Thanks!

The issue appears to be that the 32 bit version of MAYBE_RepeatedReturnNull allocates all the memory accessible to the process with generic_allocator. Although the test frees all the memory it allocates, generic_allocator never returns any memory to the OS, so the next test, DumpMemoryStats, can't allocate any memory because it uses the allocator, and there's no OS memory available to it.

I've been trying to figure out how to get the generic allocator to free its pages at the end of the test, w/o much luck so far. There are other, hackier, approaches, like letting the allocator object reserve some memory so that it can still run after generic_allocator has exhausted OS memory.  

#18: Is there a way to express that a test should always run in its own process, and never be batched together with other tests? If so, that might work.

I don't know if there's a way to force a test into it's own process (I'll look), but I am considering just splitting the OOM tests into their own file, which would accomplish the same thing.

#20: That SGTM. Thanks!

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7980674378877d07a2b41f0a55a235fe13056cb1

commit 7980674378877d07a2b41f0a55a235fe13056cb1
Author: David Bienvenu <davidbienvenu@chromium.org>
Date: Tue Apr 10 20:39:34 2018

Put ReturnNull tests in their own class

This should fix an OOM crash in PartitionAllocTest.DumpMemoryStats
because it will make the ReturnNull tests run in a separate process,
and not exhaust process memory for subsequent tests.

Bug:  722911 
Change-Id: I841b846ea3322328a1d10391332c190b59db6db0
Reviewed-on: https://chromium-review.googlesource.com/1005506
Reviewed-by: Tom Sepez <tsepez@chromium.org>
Reviewed-by: Kentaro Hara <haraken@chromium.org>
Commit-Queue: David Bienvenu <davidbienvenu@chromium.org>
Cr-Commit-Position: refs/heads/master@{#549635}
[modify] https://crrev.com/7980674378877d07a2b41f0a55a235fe13056cb1/base/allocator/partition_allocator/partition_alloc_unittest.cc