"SingleClientDirectorySyncTest.DeleteDirectoryWhenCorrupted" is flaky |
|||||||||||||||||
Issue description"SingleClientDirectorySyncTest.DeleteDirectoryWhenCorrupted" is flaky. This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label. We have detected 4 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyRQsSBUZsYWtlIjpTaW5nbGVDbGllbnREaXJlY3RvcnlTeW5jVGVzdC5EZWxldGVEaXJlY3RvcnlXaGVuQ29ycnVwdGVkDA. Flaky tests should be disabled within 30 minutes unless culprit CL is found and reverted. Please see more details here: https://sites.google.com/a/chromium.org/dev/developers/tree-sheriffs/sheriffing-bug-queues#triaging-auto-filed-flakiness-bugs
,
Jun 8 2018
Actually, reassigning to treib@, and the flakes seems to have pickup up substantially in rate right after he submitted changelist 5fff27695a81d7089b1143785c3b93c265bd0e37.
,
Jun 8 2018
Example failure: [ RUN ] SingleClientDirectorySyncTest.DeleteDirectoryWhenCorrupted [97632:775:0607/171451.758439:29036281160133:WARNING:loopback_server.cc(638)] Loopback sync persistent state file does not exist. [97632:3075:0607/171452.206429:WARNING:notification_platform_bridge_mac.mm(510)] AlertNotificationService: XPC connection invalidated. [97641:775:0607/171452.263657:WARNING:vt_video_decode_accelerator_mac.cc(193)] Failed to create VTDecompressionSession: Error Domain=NSOSStatusErrorDomain Code=-8973 "codecOpenErr" (-8973) [97641:775:0607/171452.264094:WARNING:vt_video_decode_accelerator_mac.cc(215)] Hardware decoding with VideoToolbox is not supported [97641:775:0607/171452.297855:ERROR:vt_video_encode_accelerator_mac.cc(516)] VTCompressionSessionCreate failed: -12908 2018-06-07 17:14:52.434 sync_integration_tests[97632:2747445] *** Owner supplied to -[NSTrackingArea initWithRect:options:owner:userInfo:] referenced a deallocating object. Tracking area behavior is undefined. Break on NSTrackingAreaDeallocatingOwnerError to debug. [97632:775:0607/171453.268539:WARNING:gaia_auth_fetcher.cc(902)] Could not reach Google Accounts servers: errno -11 2018-06-07 17:14:53.478 sync_integration_tests[97632:2747445] *** Owner supplied to -[NSTrackingArea initWithRect:options:owner:userInfo:] referenced a deallocating object. Tracking area behavior is undefined. Break on NSTrackingAreaDeallocatingOwnerError to debug. [97632:93959:0607/171454.480003:WARNING:get_updates_processor.cc(103)] Skipping unexpected progress marker for non-enabled type Bookmarks [97632:93959:0607/171454.480266:WARNING:sync_encryption_handler_impl.cc(970)] Nigori had empty encryption keybag. [97632:93959:0607/171454.482795:WARNING:sync_encryption_handler_impl.cc(970)] Nigori had empty encryption keybag. [97632:93959:0607/171454.490630:WARNING:sync_encryption_handler_impl.cc(346)] Ignoring new implicit passphrase. Keystore migration already performed. [97632:93959:0607/171454.494450:WARNING:get_updates_processor.cc(103)] Skipping unexpected progress marker for non-enabled type Bookmarks [97632:93959:0607/171454.593377:ERROR:connection.cc(1890)] SyncDirectory sqlite error 11, errno 0: database disk image is malformed, sql: INSERT OR REPLACE INTO metas ( metahandle, base_version, server_version, local_external_id, transaction_version, mtime, server_mtime, ctime, server_ctime, id, parent_id, server_parent_id, is_unsynced, is_unapplied_update, is_del, is_dir, server_is_dir, server_is_del, non_unique_name, server_non_unique_name, unique_server_tag, unique_client_tag, unique_bookmark_tag, specifics, server_specifics, base_server_specifics, server_unique_position, unique_position ) VALUES ( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ? ) [97632:775:0607/171454.594037:ERROR:profile_sync_service.cc(823)] Unrecoverable error detected at OnCatastrophicError@../../components/sync/syncable/directory.cc:1522 -- ProfileSyncService unusable: Catastrophic error detected, Sync DB is unrecoverable BrowserTestBase received signal: Terminated: 15. Backtrace: 0 sync_integration_tests 0x000000010d2f08fc base::debug::StackTrace::StackTrace(unsigned long) + 28 1 sync_integration_tests 0x00000001138099b5 content::(anonymous namespace)::DumpStackTraceSignalHandler(int) + 229 2 libsystem_platform.dylib 0x00007fff59c65f5a _sigtramp + 26 3 ??? 0x0000000000000000 0x0 + 0 4 sync_integration_tests 0x000000010d28e5a7 base::WaitableEvent::TimedWaitUntil(base::TimeTicks const&) + 359 5 sync_integration_tests 0x000000010d28e691 base::WaitableEvent::TimedWait(base::TimeDelta const&) + 49 6 sync_integration_tests 0x000000010983d220 (anonymous namespace)::WaitForExistingTasksOnLoop(base::MessageLoop*) + 272 7 sync_integration_tests 0x000000010983d753 (anonymous namespace)::SingleClientDirectorySyncTest_DeleteDirectoryWhenCorrupted_Test::RunTestOnMainThread() + 1059 8 sync_integration_tests 0x000000011380968f content::BrowserTestBase::ProxyRunTestOnMainThreadLoop() + 591 9 sync_integration_tests 0x000000010d63b310 ChromeBrowserMainParts::PreMainMessageLoopRunImpl() + 4736 10 sync_integration_tests 0x000000010d639f8e ChromeBrowserMainParts::PreMainMessageLoopRun() + 62 11 sync_integration_tests 0x000000010ab93803 content::BrowserMainLoop::PreMainMessageLoopRun() + 67 12 sync_integration_tests 0x000000010b115155 content::StartupTaskRunner::RunAllTasksNow() + 117 13 sync_integration_tests 0x000000010ab91f92 content::BrowserMainLoop::CreateStartupTasks() + 914 14 sync_integration_tests 0x000000010ab964c5 content::BrowserMainRunnerImpl::Initialize(content::MainFunctionParams const&, std::__1::unique_ptr<content::BrowserProcessSubThread, std::__1::default_delete<content::BrowserProcessSubThread> >) + 101 15 sync_integration_tests 0x000000010ab8fd95 content::BrowserMain(content::MainFunctionParams const&, std::__1::unique_ptr<content::BrowserProcessSubThread, std::__1::default_delete<content::BrowserProcessSubThread> >) + 229 16 sync_integration_tests 0x000000010cd33663 content::RunBrowserProcessMain(content::MainFunctionParams const&, content::ContentMainDelegate*, std::__1::unique_ptr<content::BrowserProcessSubThread, std::__1::default_delete<content::BrowserProcessSubThread> >) + 195 17 sync_integration_tests 0x000000010cd3476e content::ContentMainRunnerImpl::Run() + 686 18 sync_integration_tests 0x000000010f4747b7 service_manager::Main(service_manager::MainParams const&) + 2855 19 sync_integration_tests 0x000000010cd33574 content::ContentMain(content::ContentMainParams const&) + 68 20 sync_integration_tests 0x000000011380921c content::BrowserTestBase::SetUp() + 3212 21 sync_integration_tests 0x000000010cedf385 InProcessBrowserTest::SetUp() + 613 22 sync_integration_tests 0x000000010cece1bc SyncTest::SetUp() + 668 23 sync_integration_tests 0x000000010991c95d testing::Test::Run() + 109 24 sync_integration_tests 0x000000010991d670 testing::TestInfo::Run() + 320 25 sync_integration_tests 0x000000010991dca7 testing::TestCase::Run() + 279 26 sync_integration_tests 0x000000010992a437 testing::internal::UnitTestImpl::RunAllTests() + 871 27 sync_integration_tests 0x000000010992a0ad testing::UnitTest::Run() + 109 28 sync_integration_tests 0x000000010d1d67f6 base::TestSuite::Run() + 166 29 sync_integration_tests 0x000000010cedc545 ChromeTestSuiteRunner::RunTestSuite(int, char**) + 37 30 sync_integration_tests 0x000000011381ffd6 content::LaunchTests(content::TestLauncherDelegate*, unsigned long, int, char**) + 470 31 sync_integration_tests 0x000000010cedca5b LaunchChromeTests(unsigned long, content::TestLauncherDelegate*, int, char**) + 347 32 sync_integration_tests 0x000000010cea0d0e main + 94 33 libdyld.dylib 0x00007fff599e4115 start + 1 34 ??? 0x000000000000000a 0x0 + 10
,
Jun 8 2018
Sent https://chromium-review.googlesource.com/c/chromium/src/+/1093548 to commit queue to disable the test.
,
Jun 8 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/72845cec82dd60e1fa04f9991f0df7a85b123dd0 commit 72845cec82dd60e1fa04f9991f0df7a85b123dd0 Author: Mark Pearson <mpearson@chromium.org> Date: Fri Jun 08 20:43:38 2018 Disable Flaky Sync Test Disables SingleClientDirectorySyncTest.DeleteDirectoryWhenCorrupted due to repeated flakes on Mac. TBR=treib Bug: 850980 Change-Id: I0906849dd03a2ce0a6fa007de10036519b502958 Reviewed-on: https://chromium-review.googlesource.com/1093548 Reviewed-by: Mark Pearson <mpearson@chromium.org> Commit-Queue: Mark Pearson <mpearson@chromium.org> Cr-Commit-Position: refs/heads/master@{#565732} [modify] https://crrev.com/72845cec82dd60e1fa04f9991f0df7a85b123dd0/chrome/browser/sync/test/integration/single_client_directory_sync_test.cc
,
Jun 8 2018
Removing from sheriff queue
,
Jun 11 2018
Looks like we're timing out while waiting for the Sync thread to do its stuff here [1]. I don't see how https://crrev.com/c/1086936 would have affected that, but I can't rule it out of course. Will try to repro locally. [1] https://cs.chromium.org/chromium/src/chrome/browser/sync/test/integration/single_client_directory_sync_test.cc?rcl=4ef47d5f79c6dcbc52f8cf29ead1491f146bc827&l=152
,
Jun 12 2018
I haven't been able to repro locally on Linux. It now seems to have a lot of failures on Windows, so I'll try to repro there. https://test- results.appspot.com/dashboards/flakiness_dashboard.html#testType=sync_integration_tests&tests=SingleClientDirectorySyncTest.DeleteDirectoryWhenCorrupted https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyRQsSBUZsYWtlIjpTaW5nbGVDbGllbnREaXJlY3RvcnlTeW5jVGVzdC5EZWxldGVEaXJlY3RvcnlXaGVuQ29ycnVwdGVkDA
,
Jun 12 2018
There are at least two identical-looking failures from before my CL [1]: https://ci.chromium.org/p/chromium/builders/luci.chromium.try/mac_chromium_rel_ng/60769 https://ci.chromium.org/p/chromium/builders/luci.chromium.try/mac_chromium_rel_ng/58052 (There are more older failures, but we don't seem to have logs for those anymore, so I can't verify if they're the same.) So I'm pretty sure that my CL isn't the actual culprit, it just somehow made the existing problem more visible. [1] https://chromium-review.googlesource.com/1086936
,
Jun 12 2018
So, the problem is that the WaitForExistingTasksOnLoop(sync_loop) call [1] times out. Wild guess: This is a deadlock between the sync thread and the main (browser) thread. WaitForExistingTasksOnLoop basically posts a task to the sync thread and then blocks the main thread until that task has run. If the sync thread in turn waits for some task on the main thread to run, then Bam! deadlock. [1] https://cs.chromium.org/chromium/src/chrome/browser/sync/test/integration/single_client_directory_sync_test.cc?rcl=4ef47d5f79c6dcbc52f8cf29ead1491f146bc827&l=152
,
Jun 12 2018
,
Jun 12 2018
Pending CL with potential fix: https://chromium-review.googlesource.com/c/chromium/src/+/1097479
,
Jun 12 2018
Detected 16 new flakes for test/step "SingleClientDirectorySyncTest.DeleteDirectoryWhenCorrupted". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyRQsSBUZsYWtlIjpTaW5nbGVDbGllbnREaXJlY3RvcnlTeW5jVGVzdC5EZWxldGVEaXJlY3RvcnlXaGVuQ29ycnVwdGVkDA. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
Jun 13 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f011c8004d7905800dbd8fa284a11b5db97d06b8 commit f011c8004d7905800dbd8fa284a11b5db97d06b8 Author: Tatiana Gornak <melandory@chromium.org> Date: Wed Jun 13 10:04:05 2018 Disable Flaky Sync Test on Win Disables SingleClientDirectorySyncTest.DeleteDirectoryWhenCorrupted due to repeated flakes on Win TBR=treib Bug: 850980 Change-Id: I68b330145218982dd6be976a3c62f7713128be11 Reviewed-on: https://chromium-review.googlesource.com/1098665 Reviewed-by: Tatiana Gornak <melandory@chromium.org> Commit-Queue: Tatiana Gornak <melandory@chromium.org> Cr-Commit-Position: refs/heads/master@{#566782} [modify] https://crrev.com/f011c8004d7905800dbd8fa284a11b5db97d06b8/chrome/browser/sync/test/integration/single_client_directory_sync_test.cc
,
Jun 13 2018
,
Jun 13 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/52bbdd4e45ed7d03be5e407acc575914fc36a7c3 commit 52bbdd4e45ed7d03be5e407acc575914fc36a7c3 Author: Marc Treib <treib@chromium.org> Date: Wed Jun 13 11:45:55 2018 ProfileSyncService: Add missing NotifyObservers() on unrecoverable error ...plus fix and reenable SingleClientDirectorySyncTest.DeleteDirectoryWhenCorrupted. This test waits until it sees an unrecoverable error through the SyncServiceObserver. Since we weren't actually notifying in this case, it relied on a random different notification. Interestingly, adding the missing NotifyObservers() made the test hang consistently. My theory is that there was a possible deadlock between the sync thread and the main thread. It seems to be fixed by explicitly waiting for the main thread tasks to run before waiting on the sync thread. Bug: 850980 Change-Id: I2f636ddfa457b6d977f909e0d0a17a26aca5c166 Reviewed-on: https://chromium-review.googlesource.com/1097479 Reviewed-by: Mikel Astiz <mastiz@chromium.org> Commit-Queue: Marc Treib <treib@chromium.org> Cr-Commit-Position: refs/heads/master@{#566801} [modify] https://crrev.com/52bbdd4e45ed7d03be5e407acc575914fc36a7c3/chrome/browser/sync/test/integration/single_client_directory_sync_test.cc [modify] https://crrev.com/52bbdd4e45ed7d03be5e407acc575914fc36a7c3/components/browser_sync/profile_sync_service.cc
,
Jun 13 2018
That should fix it. I'll check the flakiness dashboard tomorrow to make sure it's not flaky anymore.
,
Jun 14 2018
The NextAction date has arrived: 2018-06-14
,
Jun 14 2018
It's still flaky, though much less so than before. Unfortunately I don't know of a way to track historic flakiness, so I don't know if it's still worse than it was before my CL from comment 2, or if it was just always a bit flaky. Let's watch for a few more days and then decide if the test needs to be disabled again.
,
Jun 14 2018
Detected 11 new flakes for test/step "SingleClientDirectorySyncTest.DeleteDirectoryWhenCorrupted". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyRQsSBUZsYWtlIjpTaW5nbGVDbGllbnREaXJlY3RvcnlTeW5jVGVzdC5EZWxldGVEaXJlY3RvcnlXaGVuQ29ycnVwdGVkDA. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
Jun 15 2018
Looks like it's now flaky mostly on ChromeOS?! I'll disable the test for CrOS.
,
Jun 15 2018
CL to disable the test on CrOS: https://chromium-review.googlesource.com/c/chromium/src/+/1101696
,
Jun 15 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/36f8b2f2a9608376c164c73a2a3955f9f6bdbae8 commit 36f8b2f2a9608376c164c73a2a3955f9f6bdbae8 Author: Marc Treib <treib@chromium.org> Date: Fri Jun 15 09:29:08 2018 Disable flaky SingleClientDirectorySyncTest.DeleteDirectoryWhenCorrupted on CrOS TBR=mastiz Bug: 850980 Change-Id: I34f162e61a6bbcab7eb702132c6a55cedfae927b Reviewed-on: https://chromium-review.googlesource.com/1101696 Reviewed-by: Marc Treib <treib@chromium.org> Commit-Queue: Marc Treib <treib@chromium.org> Cr-Commit-Position: refs/heads/master@{#567590} [modify] https://crrev.com/36f8b2f2a9608376c164c73a2a3955f9f6bdbae8/chrome/browser/sync/test/integration/single_client_directory_sync_test.cc
,
Jun 15 2018
Taking out of the sheriff queue again.
,
Jun 26 2018
This one is flaky on Linux MSan: https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Linux%20MSan%20Tests/10595
,
Jun 27 2018
https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=sync_integration_tests&tests=SingleClientDirectorySyncTest.DeleteDirectoryWhenCorrupted Looks like it got much flakier on Linux MSan around 2018-06-26 3AM GMT
,
Jun 27 2018
Can WaitForExistingTasksOnLoop do something more normal like this:
{
base::RunLoop run_loop;
// Post a task to |loop| that will, in turn, post a task back to the current
// sequenced task runner to quit the nested loop.
loop->task_runner()->PostTaskAndReply(FROM_HERE, base::DoNothing(),
run_loop.QuitClosure());
run_loop.Run();
}
This will allow message handling by the main thread while waiting for the sync loop.
,
Jun 27 2018
Example failure log: (looks slightly different than the one from #3, but maybe that's just because it's a different bot) [ RUN ] SingleClientDirectorySyncTest.DeleteDirectoryWhenCorrupted [18791:18791:0625/195133.923891:18024111921:WARNING:loopback_server.cc(638)] Loopback sync persistent state file does not exist. Xlib: extension "RANDR" missing on display ":99". [18791:18931:0625/195135.302511:ERROR:bus.cc(394)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix") [18791:18791:0625/195135.324789:WARNING:password_store_factory.cc(250)] Using basic (unencrypted) store for password storage. See https://chromium.googlesource.com/chromium/src/+/master/docs/linux_password_storage.md for more information about password storage options. (sync_integration_tests:18791): LIBDBUSMENU-GLIB-WARNING **: Unable to get session bus: Unknown or unsupported transport 'disabled' for address 'disabled:' [18791:18791:0625/195135.939952:WARNING:password_store_factory.cc(250)] Using basic (unencrypted) store for password storage. See https://chromium.googlesource.com/chromium/src/+/master/docs/linux_password_storage.md for more information about password storage options. [18791:18791:0625/195136.257925:WARNING:password_store_factory.cc(250)] Using basic (unencrypted) store for password storage. See https://chromium.googlesource.com/chromium/src/+/master/docs/linux_password_storage.md for more information about password storage options. [18791:18992:0625/195136.371746:WARNING:sync_encryption_handler_impl.cc(970)] Nigori had empty encryption keybag. [18791:18992:0625/195136.386742:WARNING:sync_encryption_handler_impl.cc(970)] Nigori had empty encryption keybag. [18791:18791:0625/195136.389977:ERROR:account_info_fetcher.cc(74)] OnOAuthError [18791:18791:0625/195136.390083:WARNING:account_fetcher_service.cc(322)] Failed to get UserInfo for gaia-id-user@gmail.com [18791:18791:0625/195136.422105:ERROR:account_tracker.cc(267)] OnOAuthError [18791:18791:0625/195136.422189:WARNING:account_tracker.cc(194)] Failed to get UserInfo for gaia-id-user@gmail.com [18791:18992:0625/195136.436357:WARNING:sync_encryption_handler_impl.cc(346)] Ignoring new implicit passphrase. Keystore migration already performed. [18791:18992:0625/195136.731658:ERROR:connection.cc(1890)] SyncDirectory sqlite error 11, errno 0: database disk image is malformed, sql: INSERT OR REPLACE INTO metas ( metahandle, base_version, server_version, local_external_id, transaction_version, mtime, server_mtime, ctime, server_ctime, id, parent_id, server_parent_id, is_unsynced, is_unapplied_update, is_del, is_dir, server_is_dir, server_is_del, non_unique_name, server_non_unique_name, unique_server_tag, unique_client_tag, unique_bookmark_tag, specifics, server_specifics, base_server_specifics, server_unique_position, unique_position ) VALUES ( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ? ) [18791:18791:0625/195136.734272:ERROR:profile_sync_service.cc(814)] Unrecoverable error detected at OnCatastrophicError@../../components/sync/syncable/directory.cc:1522 -- ProfileSyncService unusable: Catastrophic error detected, Sync DB is unrecoverable ../../chrome/browser/sync/test/integration/single_client_directory_sync_test.cc:156: Failure Value of: WaitForExistingTasksOnLoop(sync_loop) Actual: false Expected: true Stack trace: #0 0x000002da38e7 in StackTraceGetter::CurrentStackTrace(int, int) ./../../third_party/googletest/custom/gtest/internal/custom/stack_trace_getter.cc:22:27 #1 0x000002dd8cd9 in testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop(int) ./../../third_party/googletest/src/googletest/src/gtest.cc:810:35 #2 0x000002dd6e32 in testing::internal::AssertHelper::operator=(testing::Message const&) const ./../../third_party/googletest/src/googletest/src/gtest.cc:386:25 #3 0x0000024fc640 in (anonymous namespace)::SingleClientDirectorySyncTest_DeleteDirectoryWhenCorrupted_Test::RunTestOnMainThread() ./../../chrome/browser/sync/test/integration/single_client_directory_sync_test.cc:156:3 #4 0x000025fb799a in content::BrowserTestBase::ProxyRunTestOnMainThreadLoop() ./../../content/public/test/browser_test_base.cc:409:5 #5 0x00000e52a7fb in Run ./../../base/callback.h:129:12 #6 0x00000e52a7fb in ChromeBrowserMainParts::PreMainMessageLoopRunImpl() ./../../chrome/browser/chrome_browser_main.cc:1996:0 #7 0x00000e52570c in ChromeBrowserMainParts::PreMainMessageLoopRun() ./../../chrome/browser/chrome_browser_main.cc:1380:18 #8 0x00000607a09b in content::BrowserMainLoop::PreMainMessageLoopRun() ./../../content/browser/browser_main_loop.cc:966:13 #9 0x0000076fc38d in Run ./../../base/callback.h:129:12 #10 0x0000076fc38d in content::StartupTaskRunner::RunAllTasksNow() ./../../content/browser/startup_task_runner.cc:43:0 #11 0x000006071f4b in content::BrowserMainLoop::CreateStartupTasks() ./../../content/browser/browser_main_loop.cc:877:25 #12 0x000006087a96 in content::BrowserMainRunnerImpl::Initialize(content::MainFunctionParams const&, std::__1::unique_ptr<content::BrowserProcessSubThread, std::__1::default_delete<content::BrowserProcessSubThread> >) ./../../content/browser/browser_main_runner_impl.cc:148:15 #13 0x00000606654c in content::BrowserMain(content::MainFunctionParams const&, std::__1::unique_ptr<content::BrowserProcessSubThread, std::__1::default_delete<content::BrowserProcessSubThread> >) ./../../content/browser/browser_main.cc:47:20 #14 0x00000cc3a2e2 in content::RunBrowserProcessMain(content::MainFunctionParams const&, content::ContentMainDelegate*, std::__1::unique_ptr<content::BrowserProcessSubThread, std::__1::default_delete<content::BrowserProcessSubThread> >) ./../../content/app/content_main_runner_impl.cc:608:10 #15 0x00000cc3e927 in content::ContentMainRunnerImpl::Run() ./../../content/app/content_main_runner_impl.cc:955:12 #16 0x000015435416 in service_manager::Main(service_manager::MainParams const&) ./../../services/service_manager/embedder/main.cc:459:29 #17 0x00000cc34928 in content::ContentMain(content::ContentMainParams const&) ./../../content/app/content_main.cc:19:10 #18 0x000025fb5da9 in content::BrowserTestBase::SetUp() ./../../content/public/test/browser_test_base.cc:325:3 #19 0x00000cdad7e0 in InProcessBrowserTest::SetUp() ./../../chrome/test/base/in_process_browser_test.cc:248:20 BrowserTestBase received signal: Segmentation fault. Backtrace: #0 0x00000243c671 in __interceptor_backtrace /b/build/slave/linux_upload_clang/build/src/third_party/llvm/compiler-rt/lib/msan/../sanitizer_common/sanitizer_common_interceptors.inc:4024:13 #1 0x00000d77f4af in base::debug::StackTrace::StackTrace(unsigned long) ./../../base/debug/stack_trace_posix.cc:808:41 #2 0x000025fba2d2 in content::(anonymous namespace)::DumpStackTraceSignalHandler(int) ./../../content/public/test/browser_test_base.cc:88:5 #3 0x0000024640e9 in SignalHandler(int) /b/build/slave/linux_upload_clang/build/src/third_party/llvm/compiler-rt/lib/msan/msan_interceptors.cc:987:3 #4 0x7f2bb0f8bcb0 in killpg ??:? #5 0x7f2bb0f8bcb0 in ?? ??:0 #6 0x7f2bba2c9404 in __GI___pthread_mutex_lock /build/eglibc-ripdx6/eglibc-2.19/nptl/../nptl/pthread_mutex_lock.c:63:0 #7 0x00000243ca6e in __interceptor_pthread_mutex_lock /b/build/slave/linux_upload_clang/build/src/third_party/llvm/compiler-rt/lib/msan/../sanitizer_common/sanitizer_common_interceptors.inc:4073:13 #8 0x00000d7b6483 in base::internal::LockImpl::Lock() ./../../base/synchronization/lock_impl_posix.cc:102:12 #9 0x00000d7b6d23 in Acquire ./../../base/synchronization/lock.h:26:26 #10 0x00000d7b6d23 in AutoLock ./../../base/synchronization/lock.h:113:0 #11 0x00000d7b6d23 in base::WaitableEvent::Signal() ./../../base/synchronization/waitable_event_posix.cc:57:0 #12 0x00000d4327fe in Run ./../../base/callback.h:99:12 #13 0x00000d4327fe in base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*) ./../../base/debug/task_annotator.cc:101:0 #14 0x00000d4cd625 in base::MessageLoop::RunTask(base::PendingTask*) ./../../base/message_loop/message_loop.cc:319:25 #15 0x00000d4cf1e4 in DeferOrRunPendingTask ./../../base/message_loop/message_loop.cc:329:5 #16 0x00000d4cf1e4 in base::MessageLoop::DoWork() ./../../base/message_loop/message_loop.cc:373:0 #17 0x00000d4dcac4 in base::MessagePumpDefault::Run(base::MessagePump::Delegate*) ./../../base/message_loop/message_pump_default.cc:37:31 #18 0x00000d59f390 in base::RunLoop::Run() ./../../base/run_loop.cc:102:14 #19 0x00000d6968a0 in base::Thread::ThreadMain() ./../../base/threading/thread.cc:337:3 #20 0x00000d7c8908 in base::(anonymous namespace)::ThreadFunc(void*) ./../../base/threading/platform_thread_posix.cc:76:13 #21 0x7f2bba2c7184 in start_thread /build/eglibc-ripdx6/eglibc-2.19/nptl/pthread_create.c:312:0 #22 0x7f2bb105303d in clone /build/eglibc-ripdx6/eglibc-2.19/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:111:0
,
Jun 27 2018
I'm disabling it everywhere for now since it flakes across pretty much all platforms.
,
Jun 27 2018
re #27: WaitForExistingTasksOnLoop is indeed very weird. I'll try your suggestion, thanks!
,
Jun 27 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/fa2cc2f70b803acd3c9e5fd374c0ac7497b9e3b8 commit fa2cc2f70b803acd3c9e5fd374c0ac7497b9e3b8 Author: Greg Thompson <grt@chromium.org> Date: Wed Jun 27 08:43:17 2018 Disable SingleClientDirectorySyncTest.DeleteDirectoryWhenCorrupted. It flakes just about everywhere. BUG= 850980 TBR=treib@chromium.org NOTRY=true Change-Id: I2f4ac0620028240cdd24c75c6d45666ef6edcadc Reviewed-on: https://chromium-review.googlesource.com/1116545 Commit-Queue: Greg Thompson <grt@chromium.org> Reviewed-by: Greg Thompson <grt@chromium.org> Cr-Commit-Position: refs/heads/master@{#570698} [modify] https://crrev.com/fa2cc2f70b803acd3c9e5fd374c0ac7497b9e3b8/chrome/browser/sync/test/integration/single_client_directory_sync_test.cc
,
Jun 27 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/fcbc70a425e7762e105837fa0ea341c13918f43a commit fcbc70a425e7762e105837fa0ea341c13918f43a Author: Marc Treib <treib@chromium.org> Date: Wed Jun 27 09:27:09 2018 Make WaitForExistingTasksOnLoop from SingleClientDirectorySyncTest saner This is an attempt to deflake the currently disabled SingleClientDirectorySyncTest.SingleClientDirectorySyncTest. The test was failing due to running into the 45s timeout in WaitForExistingTasksOnLoop. The theory is that that's because that function is based on WaitableEvent, and thus doesn't allow processing of events on the main thread while waiting. This CL changes it to spin a nested RunLoop instead. I could never reproduce the flakes locally, so I don't know if this helps at all. As such, this CL leaves the test disabled for now; I'll send a separate one to try re-enabling. Bug: 850980 Change-Id: I16823fe01574662586e578d5f601299c58004286 Reviewed-on: https://chromium-review.googlesource.com/1116549 Reviewed-by: Greg Thompson <grt@chromium.org> Commit-Queue: Marc Treib <treib@chromium.org> Cr-Commit-Position: refs/heads/master@{#570710} [modify] https://crrev.com/fcbc70a425e7762e105837fa0ea341c13918f43a/chrome/browser/sync/test/integration/single_client_directory_sync_test.cc
,
Jun 27 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d71930ea0f4eedb9a6b6e3732cfcbf026fe4e767 commit d71930ea0f4eedb9a6b6e3732cfcbf026fe4e767 Author: Marc Treib <treib@chromium.org> Date: Wed Jun 27 12:31:52 2018 Reenable SingleClientDirectorySyncTest.DeleteDirectoryWhenCorrupted The hope is that https://crrev.com/c/1116549 has resolved the flakiness. Let's see if it worked :) Bug: 850980 Change-Id: I12c8832aba6b735179abdeaa220c4634625be326 Reviewed-on: https://chromium-review.googlesource.com/1116780 Reviewed-by: Greg Thompson <grt@chromium.org> Commit-Queue: Marc Treib <treib@chromium.org> Cr-Commit-Position: refs/heads/master@{#570740} [modify] https://crrev.com/d71930ea0f4eedb9a6b6e3732cfcbf026fe4e767/chrome/browser/sync/test/integration/single_client_directory_sync_test.cc
,
Jun 27 2018
Setting reminder for myself to check tomorrow if it's still flaky. https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=sync_integration_tests&tests=SingleClientDirectorySyncTest.DeleteDirectoryWhenCorrupted
,
Jun 28 2018
The NextAction date has arrived: 2018-06-28
,
Jun 28 2018
No new failures so far after r570740, but not many runs yet. Let's give it a few more days.
,
Jun 28 2018
Taking out of the sheriff queue though, since there's currently nothing to be done by the sheriffs.
,
Jul 3
The NextAction date has arrived: 2018-07-03
,
Jul 3
Flakiness dashboard is still looking good, let's consider this fixed until proven otherwise :) |
|||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||
Comment 1 by mpear...@chromium.org
, Jun 8 2018Status: Assigned (was: Untriaged)