New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 717960 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: May 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: Bug

Blocked on:
issue 605567



Sign in to add a comment

Two AndroidSyncSettingsTests are flaky

Project Member Reported by joh...@chromium.org, May 3 2017

Issue description

AndroidSyncSettingsTest was recently re-enabled in  issue 605567 . Since then, testIsSyncableOnSigninAndNotOnSignout and testAccountInitialization have been flaking about a third of the time each:

https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=browser_side_navigation_chrome_public_test_apk&tests=AndroidSyncSettingsTest


testIsSyncableOnSigninAndNotOnSignout fails with:

junit.framework.AssertionFailedError: expected:<2> but was:<1> at org.chromium.components.sync.AndroidSyncSettingsTest.testAccountInitialization(AndroidSyncSettingsTest.java:172)

on line:

```assertEquals(2, mSyncContentResolverDelegate.mSetIsSyncableCalls);```
https://cs.chromium.org/chromium/src/components/sync/android/javatests/src/org/chromium/components/sync/AndroidSyncSettingsTest.java?l=172&rcl=9432ee01aaa7821c3ea5cbff746a969a0725a34b



testAccountInitialization fails with:

junit.framework.AssertionFailedError at org.chromium.components.sync.AndroidSyncSettingsTest.testIsSyncableOnSigninAndNotOnSignout(AndroidSyncSettingsTest.java:372)

on line:

```assertTrue(mSyncContentResolverDelegate.getIsSyncable(mAccount, mAuthority) == 1);```
https://cs.chromium.org/chromium/src/components/sync/android/javatests/src/org/chromium/components/sync/AndroidSyncSettingsTest.java?l=372&rcl=9432ee01aaa7821c3ea5cbff746a969a0725a34b


@pavely, do you have any insight into this following you work on  issue 605567 , or does it seem unrelated?
 
Project Member

Comment 1 by bugdroid1@chromium.org, May 3 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/849f7eeb1a1e7402f69f133c50bc2167d40413a8

commit 849f7eeb1a1e7402f69f133c50bc2167d40413a8
Author: johnme <johnme@chromium.org>
Date: Wed May 03 12:46:34 2017

[Sync] Disable 2 flaky tests on Android

Disables AndroidSyncSettingsTest#testAccountInitialization and
AndroidSyncSettingsTest#testIsSyncableOnSigninAndNotOnSignout
since they are flaky on Android.

BUG= 717960 
SKIPTREECHECKS=true
TBR=pavely@chromium.org
NOTRY=true

Review-Url: https://codereview.chromium.org/2857103004
Cr-Commit-Position: refs/heads/master@{#468949}

[modify] https://crrev.com/849f7eeb1a1e7402f69f133c50bc2167d40413a8/components/sync/android/javatests/src/org/chromium/components/sync/AndroidSyncSettingsTest.java

Status: Started (was: Assigned)
Project Member

Comment 3 by bugdroid1@chromium.org, May 9 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f1c5a372002a6a2367c19a3d6100fc69ef5a7ccc

commit f1c5a372002a6a2367c19a3d6100fc69ef5a7ccc
Author: pavely <pavely@chromium.org>
Date: Tue May 09 04:45:42 2017

[Sync] Speculative changes to address flaky AndroidSyncSettings tests

testAccountInitialization fails at 172 with
assertEquals(2, mSyncContentResolverDelegate.mSetIsSyncableCalls);

testIsSyncableOnSigninAndNotOnSignout fails at 372 with
assertTrue(mSyncContentResolverDelegate.getIsSyncable(mAccount, mAuthority) == 1);

I didn't find codepath that would explain both failures so I'm speculatively
fixing threading issues that I spotted and enabling one of disabled tests, the
one easier to reason about.

Considering last time it took tests couple of days to appear on flakiness
console I'm going to wait about a week, if test doesn't fail I'll enable
the other one.

Here are the issues I'm addressing:
- counters in CountingMockSyncContentResolverDelegate are a simple ints, while
  they are incremented and accessed from different threads. I'm changing them to
  AtomicIntegers.
- in AndroidSyncSettings.updateSyncability callback receiving results of
  getGoogleAccounts is executed on UI thread, not Instrumentation thread where
  all the test code executes. It accesses mAccount. I wrapped this code block in
  synchronized.

BUG= 717960 
R=skym@chromium.org

Review-Url: https://codereview.chromium.org/2871753003
Cr-Commit-Position: refs/heads/master@{#470222}

[modify] https://crrev.com/f1c5a372002a6a2367c19a3d6100fc69ef5a7ccc/components/sync/android/java/src/org/chromium/components/sync/AndroidSyncSettings.java
[modify] https://crrev.com/f1c5a372002a6a2367c19a3d6100fc69ef5a7ccc/components/sync/android/javatests/src/org/chromium/components/sync/AndroidSyncSettingsTest.java

Project Member

Comment 4 by bugdroid1@chromium.org, May 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/78577b17b468843edb79d4111471b16e55e2aaac

commit 78577b17b468843edb79d4111471b16e55e2aaac
Author: pavely <pavely@chromium.org>
Date: Wed May 10 20:40:50 2017

[Sync] Add diagnostic into AndroidSyncSettingsTest

After my last change tests started failing on assert in setUp claiming that
getIsSyncable returns -1.

I'm trying to confirm if code in AndroidSyncSettings.updateSyncability executes
call to setIsSyncable, thus adding a few assertTrue.

In addition I noticed that AndroidSyncSettings.mContractAuthority is initialized
in ctor, but getContractAuthority() still calls mApplicationContext. I changed
it to always return mContractAuthority to ensure the same value is used
throughout test execution.

BUG= 717960 
R=skym@chromium.org

Review-Url: https://codereview.chromium.org/2873893004
Cr-Commit-Position: refs/heads/master@{#470697}

[modify] https://crrev.com/78577b17b468843edb79d4111471b16e55e2aaac/components/sync/android/java/src/org/chromium/components/sync/AndroidSyncSettings.java
[modify] https://crrev.com/78577b17b468843edb79d4111471b16e55e2aaac/components/sync/android/javatests/src/org/chromium/components/sync/AndroidSyncSettingsTest.java

It looks as if it isn't just these two tests; quite a few of AndroidSyncSettingsTests of the tests are flaky, meaning that Marshmallow 64 bit Tester is red most of the time. See https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=chrome_public_test_apk&tests=org.chromium.components.sync.AndroidSyncSettingsTest.

Comment 6 by pav...@chromium.org, May 15 2017

I'm trying to narrow down the cause of these tests flakiness, but unfortunately these tests always pass on my local setup. I'm making incremental changes to isolate failure and relying on bots to capture failure and give me more information for the next iteration.

Is it Ok if these tests stay red for a few days?
Project Member

Comment 7 by bugdroid1@chromium.org, May 16 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6bbc32fc27c0b7eb6fb416fe65ead0cefdcf6f13

commit 6bbc32fc27c0b7eb6fb416fe65ead0cefdcf6f13
Author: pavely <pavely@chromium.org>
Date: Tue May 16 00:07:17 2017

[Sync] Check that user is signed out before AndroidSyncSettingsTest

After the last round of failures I suspect that user is signed in before the
test which causes mAccount to be set to valid reference in
AndroidSyncSettings.ctor and respectively causes updateSyncability to skip
updating mSyncContentResolverDelegate.

R=skym@chromium.org
BUG= 717960 

Review-Url: https://codereview.chromium.org/2884823002
Cr-Commit-Position: refs/heads/master@{#471953}

[modify] https://crrev.com/6bbc32fc27c0b7eb6fb416fe65ead0cefdcf6f13/components/sync/android/javatests/src/org/chromium/components/sync/AndroidSyncSettingsTest.java

Project Member

Comment 8 by bugdroid1@chromium.org, May 16 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7b27ecffa77d8119aa736ecbcb8402256823f956

commit 7b27ecffa77d8119aa736ecbcb8402256823f956
Author: pavely <pavely@chromium.org>
Date: Tue May 16 22:44:35 2017

[Sync] Set signed in account before AndroidSyncSettings to match test expectations

Test expectation is that there is no signed in account at the beginning, and by
the end of setUp test expects mAccount to appear signed in from
AndroidSyncSettings perspective. It is done by calling updateAccount(mAccount)
from setUp(). The issue happens when there is another signed in account
according to ChromeSigninController.getSignedInAccountName, potentially left
from some other test execution.

The fix is to reset signed in account with
ChromeSigninController.setSignedInAccountName before AndroidSyncSettings
initialization.

R=skym@chromium.org
BUG= 717960 

Review-Url: https://codereview.chromium.org/2884953004
Cr-Commit-Position: refs/heads/master@{#472232}

[modify] https://crrev.com/7b27ecffa77d8119aa736ecbcb8402256823f956/components/sync/android/javatests/src/org/chromium/components/sync/AndroidSyncSettingsTest.java

Comment 9 by pav...@chromium.org, May 17 2017

Status: Fixed (was: Started)
Components: Tests>Disabled
Labels: Test-Disabled

Sign in to add a comment