Two AndroidSyncSettingsTests are flaky |
||||
Issue descriptionAndroidSyncSettingsTest was recently re-enabled in issue 605567 . Since then, testIsSyncableOnSigninAndNotOnSignout and testAccountInitialization have been flaking about a third of the time each: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=browser_side_navigation_chrome_public_test_apk&tests=AndroidSyncSettingsTest testIsSyncableOnSigninAndNotOnSignout fails with: junit.framework.AssertionFailedError: expected:<2> but was:<1> at org.chromium.components.sync.AndroidSyncSettingsTest.testAccountInitialization(AndroidSyncSettingsTest.java:172) on line: ```assertEquals(2, mSyncContentResolverDelegate.mSetIsSyncableCalls);``` https://cs.chromium.org/chromium/src/components/sync/android/javatests/src/org/chromium/components/sync/AndroidSyncSettingsTest.java?l=172&rcl=9432ee01aaa7821c3ea5cbff746a969a0725a34b testAccountInitialization fails with: junit.framework.AssertionFailedError at org.chromium.components.sync.AndroidSyncSettingsTest.testIsSyncableOnSigninAndNotOnSignout(AndroidSyncSettingsTest.java:372) on line: ```assertTrue(mSyncContentResolverDelegate.getIsSyncable(mAccount, mAuthority) == 1);``` https://cs.chromium.org/chromium/src/components/sync/android/javatests/src/org/chromium/components/sync/AndroidSyncSettingsTest.java?l=372&rcl=9432ee01aaa7821c3ea5cbff746a969a0725a34b @pavely, do you have any insight into this following you work on issue 605567 , or does it seem unrelated?
,
May 5 2017
,
May 9 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f1c5a372002a6a2367c19a3d6100fc69ef5a7ccc commit f1c5a372002a6a2367c19a3d6100fc69ef5a7ccc Author: pavely <pavely@chromium.org> Date: Tue May 09 04:45:42 2017 [Sync] Speculative changes to address flaky AndroidSyncSettings tests testAccountInitialization fails at 172 with assertEquals(2, mSyncContentResolverDelegate.mSetIsSyncableCalls); testIsSyncableOnSigninAndNotOnSignout fails at 372 with assertTrue(mSyncContentResolverDelegate.getIsSyncable(mAccount, mAuthority) == 1); I didn't find codepath that would explain both failures so I'm speculatively fixing threading issues that I spotted and enabling one of disabled tests, the one easier to reason about. Considering last time it took tests couple of days to appear on flakiness console I'm going to wait about a week, if test doesn't fail I'll enable the other one. Here are the issues I'm addressing: - counters in CountingMockSyncContentResolverDelegate are a simple ints, while they are incremented and accessed from different threads. I'm changing them to AtomicIntegers. - in AndroidSyncSettings.updateSyncability callback receiving results of getGoogleAccounts is executed on UI thread, not Instrumentation thread where all the test code executes. It accesses mAccount. I wrapped this code block in synchronized. BUG= 717960 R=skym@chromium.org Review-Url: https://codereview.chromium.org/2871753003 Cr-Commit-Position: refs/heads/master@{#470222} [modify] https://crrev.com/f1c5a372002a6a2367c19a3d6100fc69ef5a7ccc/components/sync/android/java/src/org/chromium/components/sync/AndroidSyncSettings.java [modify] https://crrev.com/f1c5a372002a6a2367c19a3d6100fc69ef5a7ccc/components/sync/android/javatests/src/org/chromium/components/sync/AndroidSyncSettingsTest.java
,
May 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/78577b17b468843edb79d4111471b16e55e2aaac commit 78577b17b468843edb79d4111471b16e55e2aaac Author: pavely <pavely@chromium.org> Date: Wed May 10 20:40:50 2017 [Sync] Add diagnostic into AndroidSyncSettingsTest After my last change tests started failing on assert in setUp claiming that getIsSyncable returns -1. I'm trying to confirm if code in AndroidSyncSettings.updateSyncability executes call to setIsSyncable, thus adding a few assertTrue. In addition I noticed that AndroidSyncSettings.mContractAuthority is initialized in ctor, but getContractAuthority() still calls mApplicationContext. I changed it to always return mContractAuthority to ensure the same value is used throughout test execution. BUG= 717960 R=skym@chromium.org Review-Url: https://codereview.chromium.org/2873893004 Cr-Commit-Position: refs/heads/master@{#470697} [modify] https://crrev.com/78577b17b468843edb79d4111471b16e55e2aaac/components/sync/android/java/src/org/chromium/components/sync/AndroidSyncSettings.java [modify] https://crrev.com/78577b17b468843edb79d4111471b16e55e2aaac/components/sync/android/javatests/src/org/chromium/components/sync/AndroidSyncSettingsTest.java
,
May 15 2017
It looks as if it isn't just these two tests; quite a few of AndroidSyncSettingsTests of the tests are flaky, meaning that Marshmallow 64 bit Tester is red most of the time. See https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=chrome_public_test_apk&tests=org.chromium.components.sync.AndroidSyncSettingsTest.
,
May 15 2017
I'm trying to narrow down the cause of these tests flakiness, but unfortunately these tests always pass on my local setup. I'm making incremental changes to isolate failure and relying on bots to capture failure and give me more information for the next iteration. Is it Ok if these tests stay red for a few days?
,
May 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6bbc32fc27c0b7eb6fb416fe65ead0cefdcf6f13 commit 6bbc32fc27c0b7eb6fb416fe65ead0cefdcf6f13 Author: pavely <pavely@chromium.org> Date: Tue May 16 00:07:17 2017 [Sync] Check that user is signed out before AndroidSyncSettingsTest After the last round of failures I suspect that user is signed in before the test which causes mAccount to be set to valid reference in AndroidSyncSettings.ctor and respectively causes updateSyncability to skip updating mSyncContentResolverDelegate. R=skym@chromium.org BUG= 717960 Review-Url: https://codereview.chromium.org/2884823002 Cr-Commit-Position: refs/heads/master@{#471953} [modify] https://crrev.com/6bbc32fc27c0b7eb6fb416fe65ead0cefdcf6f13/components/sync/android/javatests/src/org/chromium/components/sync/AndroidSyncSettingsTest.java
,
May 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7b27ecffa77d8119aa736ecbcb8402256823f956 commit 7b27ecffa77d8119aa736ecbcb8402256823f956 Author: pavely <pavely@chromium.org> Date: Tue May 16 22:44:35 2017 [Sync] Set signed in account before AndroidSyncSettings to match test expectations Test expectation is that there is no signed in account at the beginning, and by the end of setUp test expects mAccount to appear signed in from AndroidSyncSettings perspective. It is done by calling updateAccount(mAccount) from setUp(). The issue happens when there is another signed in account according to ChromeSigninController.getSignedInAccountName, potentially left from some other test execution. The fix is to reset signed in account with ChromeSigninController.setSignedInAccountName before AndroidSyncSettings initialization. R=skym@chromium.org BUG= 717960 Review-Url: https://codereview.chromium.org/2884953004 Cr-Commit-Position: refs/heads/master@{#472232} [modify] https://crrev.com/7b27ecffa77d8119aa736ecbcb8402256823f956/components/sync/android/javatests/src/org/chromium/components/sync/AndroidSyncSettingsTest.java
,
May 17 2017
,
Jan 24 2018
|
||||
►
Sign in to add a comment |
||||
Comment 1 by bugdroid1@chromium.org
, May 3 2017