Issue metadata
Sign in to add a comment
|
SyncSchedulerImplTest.ScheduleClearServerData_FailsRetriesSucceeds failing on Mac |
||||||||||||||||||||
Issue descriptionFiled by sheriff-o-matic@appspot.gserviceaccount.com on behalf of fhorschig@google.com The components_unittests test SyncSchedulerImplTest.ScheduleClearServerData_FailsRetriesSucceeds started failing on : - Mac10.12 Tests - Mac10.13 Tests - Mac ASan 64 Tests (1) Latest builds: https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac%20ASan%2064%20Tests%20%281%29/47396 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac10.13%20Tests/7598 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac10.12%20Tests/17521 All three failures occured for the first time when https://crrev.com/c/1361863 landed. Log: [ RUN ] SyncSchedulerImplTest.ScheduleClearServerData_FailsRetriesSucceeds [3879:775:1211/054135.026607:2584810315049:WARNING:syncer_proto_util.cc(342)] Error posting from syncer: Response Code (bogus on error): -1 Content-Length (bogus on error): -1 Server Status: CONNECTION_UNAVAILABLE ../../components/sync/engine_impl/sync_scheduler_impl_unittest.cc:1780: Failure Expected equality of these values: 1 success_counter.times_called() Which is: 0 Stack trace: 0 components_unittests 0x000000010b16b23b testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop(int) + 91 1 components_unittests 0x000000010b16abf9 testing::internal::AssertHelper::operator=(testing::Message const&) const + 89 2 components_unittests 0x000000010a4c5c7d syncer::SyncSchedulerImplTest_ScheduleClearServerData_FailsRetriesSucceeds_Test::TestBody() + 1741 [ FAILED ] SyncSchedulerImplTest.ScheduleClearServerData_FailsRetriesSucceeds (71 ms) [ RUN ] SyncSchedulerImplTest.ScheduleClearServerData_FailsRetriesSucceeds [4746:775:1211/054222.252934:2632037742964:WARNING:syncer_proto_util.cc(342)] Error posting from syncer: Response Code (bogus on error): -1 Content-Length (bogus on error): -1 Server Status: CONNECTION_UNAVAILABLE ../../components/sync/engine_impl/sync_scheduler_impl_unittest.cc:1780: Failure Expected equality of these values: 1 success_counter.times_called() Which is: 0 Stack trace: 0 components_unittests 0x000000010b00023b testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop(int) + 91 1 components_unittests 0x000000010afffbf9 testing::internal::AssertHelper::operator=(testing::Message const&) const + 89 2 components_unittests 0x000000010a35ac7d syncer::SyncSchedulerImplTest_ScheduleClearServerData_FailsRetriesSucceeds_Test::TestBody() + 1741 [ FAILED ] SyncSchedulerImplTest.ScheduleClearServerData_FailsRetriesSucceeds (101 ms) [ RUN ] SyncSchedulerImplTest.ScheduleClearServerData_FailsRetriesSucceeds [4756:775:1211/054224.383358:2634168216914:WARNING:syncer_proto_util.cc(342)] Error posting from syncer: Response Code (bogus on error): -1 Content-Length (bogus on error): -1 Server Status: CONNECTION_UNAVAILABLE ../../components/sync/engine_impl/sync_scheduler_impl_unittest.cc:1780: Failure Expected equality of these values: 1 success_counter.times_called() Which is: 0 Stack trace: 0 components_unittests 0x000000010645223b testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop(int) + 91 1 components_unittests 0x0000000106451bf9 testing::internal::AssertHelper::operator=(testing::Message const&) const + 89 2 components_unittests 0x00000001057acc7d syncer::SyncSchedulerImplTest_ScheduleClearServerData_FailsRetriesSucceeds_Test::TestBody() + 1741 [ FAILED ] SyncSchedulerImplTest.ScheduleClearServerData_FailsRetriesSucceeds (114 ms) [ RUN ] SyncSchedulerImplTest.ScheduleClearServerData_FailsRetriesSucceeds [4757:775:1211/054224.788026:2634572894413:WARNING:syncer_proto_util.cc(342)] Error posting from syncer: Response Code (bogus on error): -1 Content-Length (bogus on error): -1 Server Status: CONNECTION_UNAVAILABLE ../../components/sync/engine_impl/sync_scheduler_impl_unittest.cc:1780: Failure Expected equality of these values: 1 success_counter.times_called() Which is: 0 Stack trace: 0 components_unittests 0x0000000110a0d23b testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop(int) + 91 1 components_unittests 0x0000000110a0cbf9 testing::internal::AssertHelper::operator=(testing::Message const&) const + 89 2 components_unittests 0x000000010fd67c7d syncer::SyncSchedulerImplTest_ScheduleClearServerData_FailsRetriesSucceeds_Test::TestBody() + 1741 [ FAILED ] SyncSchedulerImplTest.ScheduleClearServerData_FailsRetriesSucceeds (85 ms)
,
Dec 11
Okay, the next two builds on the other Mac bots also failed: https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac10.13%20Tests/7599 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac10.12%20Tests/17522 The flakiness dashboard isn't really a help here: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=components_unittests&tests=SyncSchedulerImplTest.ScheduleClearServerData_FailsRetriesSucceeds
,
Dec 11
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b1525775f8eb45cd60230ff8329e99167011d979 commit b1525775f8eb45cd60230ff8329e99167011d979 Author: Friedrich Horschig [CET] <fhorschig@chromium.org> Date: Tue Dec 11 14:45:00 2018 Revert "[Reland #3] Use the SequenceManager in ScopedTaskEnvironment" This reverts commit d1317fc30ce4e9ef08c5197c0f67767dfe585df4. Reason for revert: SyncSchedulerImplTest is failing and this is the most likely culprit. More information in the linked bug. Bug: 913946 Original change's description: > [Reland #3] Use the SequenceManager in ScopedTaskEnvironment > > A reland of https://crrev.com/c/1324391 > > This is necessary because we want content::TestBrowserThreadBundle to > own a BrowserUIThreadScheduler, but that also owns a ScopedTaskEnvironment > and you can't have two SequenceManagers on the same thread. > > This patch allows ScopedTaskEnvironment to optionally work with an > externally owned SequenceManager solving the problem. > > This implements https://docs.google.com/document/d/1y08C6JQ9Yta3EQXzwIqqIIKHq9500WV6CWFZzZfDx7I/edit?usp=drivesdk, > > We now have the ability to mock time on the UI and IO threads. > > RE HttpServiceTest change HttpServiceTest.MultipleRequests was failing > mysteriously on fuschia on the bots only (not locally). IO_MOCK_TIME seems > to have fixed this. > > > TBR=gab@chromium.org,fdoray@chromium.org,wez@chromium.org > > Bug: 863341, 891670, 708584 > Change-Id: I95444a0a50f8b577c5fae62c12c8423c7e8f21d6 > Reviewed-on: https://chromium-review.googlesource.com/c/1361863 > Commit-Queue: Alex Clarke <alexclarke@chromium.org> > Reviewed-by: Alex Clarke <alexclarke@chromium.org> > Reviewed-by: Sami Kyöstilä <skyostil@chromium.org> > Cr-Commit-Position: refs/heads/master@{#615496} TBR=wez@chromium.org,gab@chromium.org,fdoray@chromium.org,skyostil@chromium.org,alexclarke@chromium.org Change-Id: If5ff0362758b5270eb3113d2fd6eeeb7facc37c3 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: 863341, 891670, 708584 Reviewed-on: https://chromium-review.googlesource.com/c/1371811 Reviewed-by: Friedrich Horschig [CET] <fhorschig@chromium.org> Commit-Queue: Friedrich Horschig [CET] <fhorschig@chromium.org> Cr-Commit-Position: refs/heads/master@{#615520} [modify] https://crrev.com/b1525775f8eb45cd60230ff8329e99167011d979/base/test/scoped_task_environment.cc [modify] https://crrev.com/b1525775f8eb45cd60230ff8329e99167011d979/base/test/scoped_task_environment.h [modify] https://crrev.com/b1525775f8eb45cd60230ff8329e99167011d979/base/test/scoped_task_environment_unittest.cc [modify] https://crrev.com/b1525775f8eb45cd60230ff8329e99167011d979/webrunner/net_http/http_service_unittest.cc
,
Dec 11
Could well be related. Unfortunately the Mac & iOS message pumps don't behave the same way the other ones do which is causing us no end of trouble.
,
Dec 11
Issue 913957 has been merged into this issue.
,
Dec 11
FYI: the MSAN builder hasn't shown another failure and Mac 10.13 also cycled green BEFORE the revert landed: https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac10.13%20Tests/7600 If Mac 10.12 does the same, this could really have been an occurrence of three flakes with very extraordinary timing.
,
Dec 11
Mac 10.12 failed continuously until the revert was in. So the revert seemed to have effect and the original CL might just increase the flakiness a lot. I did not notice any other test that broke by the original CL and cycled green with the revert. And neither did findit. Removing from sheriff queue.
,
Dec 13
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2ded172c0daa913d3d6e0b118d86ed38210cf06f commit 2ded172c0daa913d3d6e0b118d86ed38210cf06f Author: Alex Clarke <alexclarke@chromium.org> Date: Thu Dec 13 12:55:20 2018 [Reland #4] Use the SequenceManager in ScopedTaskEnvironment A reland of https://crrev.com/c/1324391 This is necessary because we want content::TestBrowserThreadBundle to own a BrowserUIThreadScheduler, but that also owns a ScopedTaskEnvironment and you can't have two SequenceManagers on the same thread. This patch allows ScopedTaskEnvironment to optionally work with an externally owned SequenceManager solving the problem. This implements https://docs.google.com/document/d/1y08C6JQ9Yta3EQXzwIqqIIKHq9500WV6CWFZzZfDx7I/edit?usp=drivesdk, We now have the ability to mock time on the UI and IO threads. HttpServiceTest had been changed because HttpServiceTest.MultipleRequests was failing mysteriously on fuchsia on the bots only (not locally). IO_MOCK_TIME seems to have fixed this. SyncSchedulerImplTest has been changed to use MOCK_TIME because it was flaky on Mac. This change required base::TimeTicks::Now() to be overridden. These tests should now run faster which I understand was a historical problem (see https://crbug.com/402212 ). The chromeos test changes are needed because C++ requires a default constructor to const construct an object. ScopedTaskEnvironment has a template vararg constructor which due to a defect in the C++ standard isn't recognized for this purpose. C++ will happily let you default construct a non-const object using it however. TBR=khorimoto@chromium.org, wez@chromium.org, jkrcal@chromium.org Bug: 863341, 891670, 913946, 911928 Change-Id: Ib7e58223b531cf85aded11b75bd09c47a0993391 Reviewed-on: https://chromium-review.googlesource.com/c/1371899 Reviewed-by: Alex Clarke <alexclarke@chromium.org> Reviewed-by: Jan Krcal <jkrcal@chromium.org> Reviewed-by: Gabriel Charette <gab@chromium.org> Reviewed-by: Sami Kyöstilä <skyostil@chromium.org> Commit-Queue: Alex Clarke <alexclarke@chromium.org> Cr-Commit-Position: refs/heads/master@{#616280} [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/base/test/scoped_task_environment.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/base/test/scoped_task_environment.h [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/base/test/scoped_task_environment_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/components/nearby/count_down_latch_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/components/nearby/settable_future_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/components/tether/asynchronous_shutdown_object_container_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/components/tether/connection_preserver_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/components/tether/crash_recovery_manager_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/components/tether/host_scanner_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/components/tether/wifi_hotspot_disconnector_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/multidevice_setup/account_status_change_delegate_notifier_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/multidevice_setup/multidevice_setup_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/multidevice_setup/multidevice_setup_service_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/multidevice_setup/privileged_host_device_setter_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/secure_channel/active_connection_manager_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/secure_channel/authenticated_channel_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/secure_channel/ble_advertiser_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/secure_channel/ble_listener_operation_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/secure_channel/ble_synchronizer_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/secure_channel/client_connection_parameters_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/secure_channel/connect_to_device_operation_base_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/secure_channel/connection_attempt_base_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/secure_channel/multiplexed_channel_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/secure_channel/pending_ble_initiator_connection_request_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/secure_channel/pending_ble_listener_connection_request_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/secure_channel/pending_connection_manager_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/secure_channel/pending_connection_request_base_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/secure_channel/public/cpp/client/client_channel_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/secure_channel/public/cpp/client/secure_channel_client_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/secure_channel/secure_channel_service_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/chromeos/services/secure_channel/single_client_message_proxy_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/components/sync/engine_impl/sync_scheduler_impl_unittest.cc [modify] https://crrev.com/2ded172c0daa913d3d6e0b118d86ed38210cf06f/webrunner/net_http/http_service_unittest.cc |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by fhorschig@chromium.org
, Dec 11Components: Internals>TaskScheduler
Labels: -Pri-2 OS-Mac Pri-1 Type-Bug-Regression
Owner: alexclarke@chromium.org