New issue
Advanced search Search tips

Issue 718710 link

Starred by 2 users

Issue metadata

Status: Duplicate
Merged: issue 851148
Owner:
Closed: Jul 12
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 3
Type: Bug



Sign in to add a comment

PartitionAllocTest.RepeatedReturnNullDirect times out

Project Member Reported by thestig@chromium.org, May 5 2017

Issue description

What steps will reproduce the problem?
(1) On Linux, with r469589, build base_unittests with:

is_debug = true
is_component_build = true

(2) Run base_unittests

What is the expected result?

All tests pass in a reasonable amount of time.

What happens instead?

[2561/2581] RedirectionToTaskScheduler/SequencedWorkerPoolTest.AllowsAfterShutdown/0 (6 ms)
[2562/2581] PartitionAllocTest.MultiPageAllocs (96 ms)
[2563/2581] PartitionAllocTest.GenericAlloc (16 ms)
[2564/2581] PartitionAllocTest.GenericAllocSizes (4748 ms)
[2565/2581] PartitionAllocTest.GenericAllocGetSize (7197 ms)
[2566/2581] PartitionAllocTest.Realloc (0 ms)
[2567/2581] PartitionAllocTest.PartialPageFreelists (0 ms)
[2568/2581] PartitionAllocTest.PageRefilling (1 ms)
[2569/2581] PartitionAllocTest.PartialPages (1 ms)
[2570/2581] PartitionAllocTest.MappingCollision (120 ms)
[2571/2581] PartitionAllocTest.FreeCache (0 ms)
Still waiting for the following processes to finish:
        /path/to/base_unittests --gtest_flagfile=/tmp/blah --single-process-tests --test-launcher-output=/tmp/blah/test_results.xml
(repeats 10X)
[2572/2581] PartitionAllocTest.LostFreePagesBug (0 ms)
[ RUN      ] PartitionAllocTest.RepeatedReturnNullDirect
[       OK ] PartitionAllocTest.RepeatedReturnNullDirect (143139 ms)
[ RUN      ] PartitionAllocTest.RepeatedReturnNull
[       OK ] PartitionAllocTest.RepeatedReturnNull (2176 ms)
[ RUN      ] PartitionAllocTest.DumpMemoryStats
[       OK ] PartitionAllocTest.DumpMemoryStats (2 ms)
[ RUN      ] PartitionAllocTest.Purge
[       OK ] PartitionAllocTest.Purge (0 ms)
[ RUN      ] PartitionAllocTest.PreferActiveOverEmpty
[       OK ] PartitionAllocTest.PreferActiveOverEmpty (1 ms)
[ RUN      ] PartitionAllocTest.PurgeDiscardable
[       OK ] PartitionAllocTest.PurgeDiscardable (0 ms)
[ RUN      ] PartitionAllocTest.ReallocMovesCookies
[       OK ] PartitionAllocTest.ReallocMovesCookies (0 ms)
[----------] 8 tests from PartitionAllocTest (145318 ms total)

[----------] 2 tests from XDGUtilTest
[ RUN      ] XDGUtilTest.GetDesktopEnvironmentGnome
[       OK ] XDGUtilTest.GetDesktopEnvironmentGnome (1 ms)
[ RUN      ] XDGUtilTest.GetDesktopEnvironmentMATE
[       OK ] XDGUtilTest.GetDesktopEnvironmentMATE (0 ms)
[----------] 2 tests from XDGUtilTest (1 ms total)

[----------] Global test environment tear-down
[==========] 10 tests from 2 test cases ran. (145320 ms total) <--- 120 second timeout?
[  PASSED  ] 10 tests.
[2573/2581] PartitionAllocTest.RepeatedReturnNullDirect (TIMED OUT)
[2574/2581] PartitionAllocTest.RepeatedReturnNull (2176 ms)
[2575/2581] PartitionAllocTest.DumpMemoryStats (2 ms)
[2576/2581] PartitionAllocTest.Purge (0 ms)
[2577/2581] PartitionAllocTest.PreferActiveOverEmpty (1 ms)
[2578/2581] PartitionAllocTest.PurgeDiscardable (0 ms)
[2579/2581] PartitionAllocTest.ReallocMovesCookies (0 ms)
[2580/2581] XDGUtilTest.GetDesktopEnvironmentGnome (1 ms)
[2581/2581] XDGUtilTest.GetDesktopEnvironmentMATE (0 ms)
Retrying 1 test (retry #1)
Still waiting for the following processes to finish:
        /path/to/base_unittests --gtest_flagfile=/tmp/blah --single-process-tests --test-launcher-output=/tmp/blah/test_results.xml
[2582/2582] PartitionAllocTest.RepeatedReturnNullDirect (18970 ms) <--- That's a long time for 1 test.
 
Status: Assigned (was: Untriaged)

Comment 2 by palmer@chromium.org, May 24 2017

Can I see your /tmp/blah flagfile?

Comment 3 by palmer@chromium.org, May 24 2017

Here is what I get:

```
chromium/src $ cat out/Default/args.gn 
is_debug = true
is_component_build = true
use_afl = false
is_asan = false
enable_nacl = false
use_goma = true

chromium/src $ ./out/Default/base_unittests --single-process-tests --gtest_filter=PartitionAllocTest.*
...
[       OK ] PartitionAllocTest.RepeatedReturnNullDirect (4926 ms)
...
```

Note that these tests are inherently, and necessarily?, aggressive and weird:

```
// Unit tests that check if an allocation fails in "return null" mode,
// repeating it doesn't crash, and still returns null. The tests need to
// stress memory subsystem limits to do so, hence they try to allocate
// 6 GB of memory, each with a different per-allocation block sizes.
//
// On 64-bit POSIX systems, the address space is limited to 6 GB using
// setrlimit() first.

// Test "return null" for larger, direct-mapped allocations first. As a
// direct-mapped allocation's pages are unmapped and freed on release, this
// test is performd first for these "return null" tests in order to leave
// sufficient unreserved virtual memory around for the later one(s).
```

and:

```
static void DoReturnNullTest(size_t allocSize) {
  // TODO(crbug.com/678782): Where necessary and possible, disable the
  // platform's OOM-killing behavior. OOM-killing makes this test flaky on
  // low-memory devices.
  if (!IsLargeMemoryDevice()) {
    LOG(WARNING) << "Skipping test on this device because of crbug.com/678782";
    return;
  } 

  TestSetup();

  EXPECT_TRUE(SetAddressSpaceLimit());

  // Work out the number of allocations for 6 GB of memory.
  const int numAllocations = (6 * 1024 * 1024) / (allocSize / 1024);

  void** ptrs = reinterpret_cast<void**>(PartitionAllocGeneric(
      generic_allocator.root(), numAllocations * sizeof(void*), type_name));
  int i;

  for (i = 0; i < numAllocations; ++i) {
    ptrs[i] = PartitionAllocGenericFlags(generic_allocator.root(),
                                         PartitionAllocReturnNull, allocSize,
                                         type_name);
    if (!i)
      EXPECT_TRUE(ptrs[0]);
    if (!ptrs[i]) {
      ptrs[i] = PartitionAllocGenericFlags(generic_allocator.root(),
                                           PartitionAllocReturnNull, allocSize,
                                           type_name);
      EXPECT_FALSE(ptrs[i]);
      break;
    }
  }

  // We shouldn't succeed in allocating all 6 GB of memory. If we do, then
  // we're not actually testing anything here.
  EXPECT_LT(i, numAllocations);

  // Free, reallocate and free again each block we allocated. We do this to
  // check that freeing memory also works correctly after a failed allocation.
  for (--i; i >= 0; --i) {
    PartitionFreeGeneric(generic_allocator.root(), ptrs[i]);
    ptrs[i] = PartitionAllocGenericFlags(generic_allocator.root(),
                                         PartitionAllocReturnNull, allocSize,
                                         type_name);
    EXPECT_TRUE(ptrs[i]);
    PartitionFreeGeneric(generic_allocator.root(), ptrs[i]);
  }

  PartitionFreeGeneric(generic_allocator.root(), ptrs);

  EXPECT_TRUE(ClearAddressSpaceLimit());
}
```

These tests have always been this way; what's new (as of late 2016) is that they're in base_unittests. So, I think this might be WontFix, unless the slowness is a huge problem?

Comment 4 by palmer@chromium.org, May 24 2017

Slowness and/or timing out. Are they timing out on our bots on a regular basis?
Mergedinto: 851148
Status: Duplicate (was: Assigned)
It looks like rharrison fixed this in https://chromium-review.googlesource.com/1106602, and that  issue 851148  was a duplicate of this one. Thanks rharrison! That's the bug of record now, so duping this one into that one.

Sign in to add a comment