This is blocking the catapult autoroller.

I will try to repro the failure locally.

I can't reproduce it locally.
+rogerm@ and +neduguyen to help triage.

Attempting to repo as well. I agree that the NTP change is the most likely suspect.

If we can't repro and revert quickly, though, we ought to disable the test to unblock the CQ.

I will prepare a CL to disable the test.

sgtm

CL is created to disable the test: https://codereview.chromium.org/2271333002/.

I saw some flakiness on this test in my try runs (setting up find experiment config testing).

I'm don't think my experiment (enabling deduplication of autofill profiles) should be interacting with Bookmarks at all. But there was definitely some flakiness in my trybot runs (not only for this Android test).

See https://codereview.chromium.org/2268753002 and expand the trybot results.

  - linux_android_rel_ng
  - max_chromium_rel_ng
  - win_chromium_rel_ng

O_O

https://codereview.chromium.org/2276013002 exhibited no such flakes in its try runs, so I'd say we have a new suspected CL.

A revert is created in in CQ. Thanks for the investigation. I should expand the trybots and see the previous results.

The revert (https://codereview.chromium.org/2276103003/) is landed, hopefully these bots will turn green.

I'm able to reproduce this locally, but only on the second and subsequent run of the test after APK installation -- the first run always seems to pass.

I build to match chromium.linux:Android Builder via

  ./tools/mb/mb.py gen -m chromium.linux -b "Android Builder" <output directory>
  ninja -C <output directory> [options] chrome_sync_shell_test_apk

and used an N5 on KTU84P, the same as Android Tests and linux_android_rel_ng.

Thanks for sharing, I can reproduce it locally too. Yes, need to run the test twice to get the failure.

Looks like the revert was successful.

31001 and later builds are green:
https://build.chromium.org/p/chromium.linux/builders/Android%20Tests/builds/31001

Awesome!

Thanks for fixing this, hanxi!

 Issue 640728  has been merged into this issue.

The order of the folder and bookmark is the reverse of that expected for
the 2nd and subsequent runs.

Swapping the 0 and 1 indices causes the tests to pass, but I haven't yet
tracked down why this is happening in the first place.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/778c94f05935d22b8e0555616d0faaf277ad5b7a

commit 778c94f05935d22b8e0555616d0faaf277ad5b7a
Author: rogerm <rogerm@chromium.org>
Date: Tue Aug 30 21:21:38 2016

Fix BookmarksTest#testDownloadMovedBookmark to be insensitive to item order.

BUG=620414,  640669 ,  64218 

Review-Url: https://codereview.chromium.org/2289803002
Cr-Commit-Position: refs/heads/master@{#415430}

[modify] https://crrev.com/778c94f05935d22b8e0555616d0faaf277ad5b7a/chrome/android/sync_shell/javatests/src/org/chromium/chrome/browser/sync/BookmarksTest.java

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f9f54bb9c82589bb0b00c7049b1844f00148db2b

commit f9f54bb9c82589bb0b00c7049b1844f00148db2b
Author: rogerm <rogerm@chromium.org>
Date: Wed Aug 31 14:52:00 2016

[Autofill] Declare profile cleanup field trial groups for testing

BUG=620414,  640669 

Review-Url: https://codereview.chromium.org/2279543003
Cr-Commit-Position: refs/heads/master@{#415631}

[modify] https://crrev.com/f9f54bb9c82589bb0b00c7049b1844f00148db2b/testing/variations/fieldtrial_testing_config_android.json
[modify] https://crrev.com/f9f54bb9c82589bb0b00c7049b1844f00148db2b/testing/variations/fieldtrial_testing_config_chromeos.json
[modify] https://crrev.com/f9f54bb9c82589bb0b00c7049b1844f00148db2b/testing/variations/fieldtrial_testing_config_ios.json
[modify] https://crrev.com/f9f54bb9c82589bb0b00c7049b1844f00148db2b/testing/variations/fieldtrial_testing_config_linux.json
[modify] https://crrev.com/f9f54bb9c82589bb0b00c7049b1844f00148db2b/testing/variations/fieldtrial_testing_config_mac.json
[modify] https://crrev.com/f9f54bb9c82589bb0b00c7049b1844f00148db2b/testing/variations/fieldtrial_testing_config_win.json