New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 734810 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit 15 days ago
Closed: Jun 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug-Regression
Hotlist-MemoryInfra



Sign in to add a comment

base_unittests failing on chromium.memory/Linux TSan Tests

Project Member Reported by pdr@chromium.org, Jun 19 2017

Issue description

base_unittests failing on chromium.memory/Linux TSan Tests. The test is:
MemoryDumpSchedulerTest.StopAndStartOnAnotherThread

Here's a link: https://uberchromegw.corp.google.com/i/chromium.memory/builders/Linux%20TSan%20Tests/builds/7646

Builders failed on: 
- Linux TSan Tests: 
  https://build.chromium.org/p/chromium.memory/builders/Linux%20TSan%20Tests


I think this is a regression from https://chromium.googlesource.com/chromium/src/+/d4a5e98235da0cc8c4fb1ae2f67826b0b480ce4c
 

Comment 1 by pdr@chromium.org, Jun 19 2017

Components: Internals>Instrumentation>Memory

Comment 3 by dskiba@chromium.org, Jun 20 2017

Cc: primiano@chromium.org ssid@chromium.org
So the race happens inside gTest library:

testing::internal::UntypedFunctionMockerBase::UntypedInvokeWith() calls untyped_expectations_.size()

and FunctionMockerBase::AddNewExpectation() calls untyped_expectations_.push_back()

and they are not synchronized.

I think there is a bug in MemoryDumpSchedulerTest.StopAndStartOnAnotherThread test: in the second part ("MemoryDumpSchedulerTest Thread 2") it should first call EXPECT_CALL() (which calls AddNewExpectation) and then call scheduler_->Start().

What happens how is scheduler_->Start() posts two tasks which in the end invoke CallbackWrapper::OnTick mock method *before* the main thread gets to EXPECT_CALL() statements.

ssid@, primiano@ do you think my analysis is correct?

I'm still trying to understand how my CL (or any CL from the blamelist) could trigger that...

Comment 4 by dskiba@chromium.org, Jun 20 2017

Aha! The test is just flaky, https://uberchromegw.corp.google.com/i/chromium.memory/builders/Linux%20TSan%20Tests/builds/7647 succeeded.

ssid@, can you take this, and submit a fix?

Comment 5 by dskiba@chromium.org, Jun 20 2017

Yup, the test is flaky, here is similar failure from the past (Jun 16): https://uberchromegw.corp.google.com/i/chromium.memory/builders/Linux%20TSan%20Tests/builds/7558


Comment 6 by pdr@chromium.org, Jun 20 2017

Good find! This test is indeed flaky and became so recently. Here's a link to the test history:
https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=base_unittests&tests=MemoryDumpSchedulerTest.StopAndStartOnAnotherThread%0A

Comment 7 by dskiba@chromium.org, Jun 20 2017

The fix is easy, I'm going to land it instead of disabling the test.

Comment 8 by dskiba@chromium.org, Jun 20 2017

https://codereview.chromium.org/2952493002 is in the commit queue.

Comment 9 by hayato@chromium.org, Jun 20 2017

Labels: -Sheriff-Chromium
Status: Fixed (was: Available)
Re #3: the analysis in #3 makes perfect sense, so does your fix #8.
I wished there was a way to preemptively warn for these patterns that end up causing test flakiness.

Marking as fixed as the flakiness seems to have gone. Reopen if it's not the case.

Sign in to add a comment