New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 732861 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Jun 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

"MemoryDumpSchedulerTest.MultipleTriggers" is flaky

Project Member Reported by chromium...@appspot.gserviceaccount.com, Jun 13 2017

Issue description

"MemoryDumpSchedulerTest.MultipleTriggers" is flaky.

This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label.

We have detected 3 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMwsSBUZsYWtlIihNZW1vcnlEdW1wU2NoZWR1bGVyVGVzdC5NdWx0aXBsZVRyaWdnZXJzDA.

Flaky tests should be disabled within 30 minutes unless culprit CL is found and reverted. Please see more details here: https://sites.google.com/a/chromium.org/dev/developers/tree-sheriffs/sheriffing-bug-queues#triaging-auto-filed-flakiness-bugs
 
Owner: primiano@chromium.org
Hi primiano@, could you triage this bug since you are the latest one touched that test?
Cc: hjd@chromium.org
Status: Assigned (was: Untriaged)
sure, will look at that tomorrow morning. (+hjd to create peer pressure in case I forget)
(I am about to leave, feel free to TBR a change that disables it under asan/tsan if that will be problematic for the next 12 hours)
Okay, disabling it on Linux

Comment 5 by hjd@chromium.org, Jun 13 2017

hmm, something odd about TSAN and PostDelayedTask and/or TimeTicks::Now()?
Or maybe our tolerances are just too tight and we should allow actual >= (expected - 1ms) to pass?

Code link: https://cs.chromium.org/chromium/src/base/trace_event/memory_dump_scheduler.cc?l=88
Test link: https://cs.chromium.org/chromium/src/base/trace_event/memory_dump_scheduler_unittest.cc?l=91

Failures:
1)
[ RUN      ] MemoryDumpSchedulerTest.MultipleTriggers
../../base/trace_event/memory_dump_scheduler_unittest.cc:128: Failure
Expected: ((t2 - t1).InMillisecondsF()) >= (kPeriodDetailedMs), actual: 8.807 vs 9
[  FAILED  ] MemoryDumpSchedulerTest.MultipleTriggers (21 ms)
[2598/2598] MemoryDumpSchedulerTest.MultipleTriggers (21 ms)
Retrying 1 test (retry #2)
[29033:29055:0613/070957.802324:509129994:ERROR:kill_posix.cc(84)] Unable to terminate process group 32249: No such process
[ RUN      ] MemoryDumpSchedulerTest.MultipleTriggers
../../base/trace_event/memory_dump_scheduler_unittest.cc:128: Failure
Expected: ((t2 - t1).InMillisecondsF()) >= (kPeriodDetailedMs), actual: 8.939 vs 9
[  FAILED  ] MemoryDumpSchedulerTest.MultipleTriggers (21 ms)
[2599/2599] MemoryDumpSchedulerTest.MultipleTriggers (21 ms)
Retrying 1 test (retry #3)
[29033:29051:0613/070958.067652:509395315:ERROR:kill_posix.cc(84)] Unable to terminate process group 32252: No such process
[ RUN      ] MemoryDumpSchedulerTest.MultipleTriggers
../../base/trace_event/memory_dump_scheduler_unittest.cc:128: Failure
Expected: ((t2 - t1).InMillisecondsF()) >= (kPeriodDetailedMs), actual: 8.657 vs 9
[  FAILED  ] MemoryDumpSchedulerTest.MultipleTriggers (21 ms)
[2600/2600] MemoryDumpSchedulerTest.MultipleTriggers (21 ms)

2)
2584/2584] MemoryDumpManagerTest.DumpWithTracingInitializedButDisabled (7 ms)
Retrying 1 test (retry #2)
[19008:19012:0612/144849.790573:27124983094:ERROR:kill_posix.cc(84)] Unable to terminate process group 22166: No such process
[ RUN      ] MemoryDumpSchedulerTest.MultipleTriggers
../../base/trace_event/memory_dump_scheduler_unittest.cc:128: Failure
Expected: ((t2 - t1).InMillisecondsF()) >= (kPeriodDetailedMs), actual: 8.89 vs 9
[  FAILED  ] MemoryDumpSchedulerTest.MultipleTriggers (21 ms)
[2585/2585] MemoryDumpSchedulerTest.MultipleTriggers (21 ms)
Retrying 1 test (retry #3)
[19008:19014:0612/144850.056640:27125249163:ERROR:kill_posix.cc(84)] Unable to terminate process group 22169: No such process
[ RUN      ] MemoryDumpSchedulerTest.MultipleTriggers
../../base/trace_event/memory_dump_scheduler_unittest.cc:128: Failure
Expected: ((t2 - t1).InMillisecondsF()) >= (kPeriodDetailedMs), actual: 8.776 vs 9
[  FAILED  ] MemoryDumpSchedulerTest.MultipleTriggers (21 ms)
[2586/2586] MemoryDumpSchedulerTest.MultipleTriggers (21 ms)

3)
[ RUN      ] MemoryDumpSchedulerTest.MultipleTriggers
../../base/trace_event/memory_dump_scheduler_unittest.cc:128: Failure
Expected: ((t2 - t1).InMillisecondsF()) >= (kPeriodDetailedMs), actual: 8.8 vs 9
[  FAILED  ] MemoryDumpSchedulerTest.MultipleTriggers (21 ms)
[2585/2586] MemoryDumpSchedulerTest.MultipleTriggers (21 ms)
[ RUN      ] MemoryDumpManagerTest.TestBackgroundTracingSetup
[       OK ] MemoryDumpManagerTest.TestBackgroundTracingSetup (14 ms)
[2586/2586] MemoryDumpManagerTest.TestBackgroundTracingSetup (14 ms)
Retrying 1 test (retry #3)
[2010:2022:0612/131615.920744:83059658:ERROR:kill_posix.cc(84)] Unable to terminate process group 5192: No such process
[ RUN      ] MemoryDumpSchedulerTest.MultipleTriggers
../../base/trace_event/memory_dump_scheduler_unittest.cc:128: Failure
Expected: ((t2 - t1).InMillisecondsF()) >= (kPeriodDetailedMs), actual: 8.956 vs 9
[  FAILED  ] MemoryDumpSchedulerTest.MultipleTriggers (21 ms)
[2587/2587] MemoryDumpSchedulerTest.MultipleTriggers (21 ms)
Project Member

Comment 6 by bugdroid1@chromium.org, Jun 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/06e2d539179523cf3ce6041128bc5e8977cf7e85

commit 06e2d539179523cf3ce6041128bc5e8977cf7e85
Author: gogerald <gogerald@chromium.org>
Date: Tue Jun 13 18:41:59 2017

Disable MemoryDumpSchedulerTest.MultipleTriggers on Linux

TBR=primiano@chromium.org
BUG= 732861 

Review-Url: https://codereview.chromium.org/2940693002
Cr-Commit-Position: refs/heads/master@{#479078}

[modify] https://crrev.com/06e2d539179523cf3ce6041128bc5e8977cf7e85/base/trace_event/memory_dump_scheduler_unittest.cc

Labels: -Sheriff-Chromium
Dropping sheriff label since disabled.
Cc: -hjd@chromium.org primiano@chromium.org ssid@chromium.org
Owner: hjd@chromium.org
Status: Started (was: Assigned)
moving to hjd. we discussed this offline, this is because 
MemoryDumpScheduler::Tick() first Posts the next task and then runs the callback.
Which means that the time spent in the execution of the callback will be subtracted from the t2-t1 measurement. I think the more practical thing to do here is to just invert the order and guarantee that the PostTask happens always after the callback is run, so that the t2-t1 measurement becomes necessarily >= the actual time elapsed
Project Member

Comment 9 by bugdroid1@chromium.org, Jun 14 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/52d98977cc8d59dfa5fdd9b37858fe35af6737ad

commit 52d98977cc8d59dfa5fdd9b37858fe35af6737ad
Author: Hector Dearman <hjd@google.com>
Date: Wed Jun 14 15:30:52 2017

memory-infra: Re-enable MemoryDumpSchedulerTest.MultipleTriggers

De-flakes and re-enables MemoryDumpSchedulerTest.MultipleTriggers.
This test was flaky because we scheduled the next task before calling
the callback. When calling the callback takes a long time (due to TSAN)
the timing (as seen by the tests) is off. Instead we can run the
callback before scheduling the next tick avoiding this issue.

Bug:  732861 
Change-Id: I39c8c63c2c6589e7319f48bbf15e26acea9b6295
Reviewed-on: https://chromium-review.googlesource.com/535541
Reviewed-by: Primiano Tucci <primiano@chromium.org>
Commit-Queue: Hector Dearman <hjd@chromium.org>
Cr-Commit-Position: refs/heads/master@{#479395}
[modify] https://crrev.com/52d98977cc8d59dfa5fdd9b37858fe35af6737ad/base/trace_event/memory_dump_scheduler.cc
[modify] https://crrev.com/52d98977cc8d59dfa5fdd9b37858fe35af6737ad/base/trace_event/memory_dump_scheduler_unittest.cc

Comment 10 by hjd@chromium.org, Jun 14 2017

Status: Fixed (was: Started)

Sign in to add a comment