It looks like this whole test suite is pretty flaky except for #testHelpPageSkipsFirstRun. cc'ing other first run owners.

Will you please help investigate/triage?

Flakiness dashboard: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=chrome_public_test_apk&tests=FirstRunIntegrationTest

I am in the process of rewriting runFirstRunRedirectTestForActivity() and testAbortFirstRun() as JUnit tests (in a simillar pattern as https://chrome-internal-review.googlesource.com/c/clank/internal/apps/+/724688). If I feel ambitious, I will look at the other tests too.

Robolectric is really good for checking for started activities.

These tests are blocking the CQ which blocks patches from landing. Is switching to JUnit a small change? If not, could these tests be disabled to unblock the CQ?

I think that disabling the tests to unblock the CQ makes sense.

- I am running the tests locally and it seems like the initial SharedPreferences state is incorrect
- Switching the tests to be JUnit tests is not a small change. (Likely 1-2 hours of work)

Would a quicker fix be setting or resetting the SharedPreference state before running the tests?

Note that there were some related tests disabled during the DEX limit issue a month ago that are in a similar boat -- they need SharedPreferences cleared to pass correctly: see issue 902774.

It'd be great to get those re-enabled too.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b0c7af7b9ba7dd8288e7eed6d98107d9fd188a3f

commit b0c7af7b9ba7dd8288e7eed6d98107d9fd188a3f
Author: Peter Kotwicz <pkotwicz@chromium.org>
Date: Tue Dec 04 23:03:48 2018

Temporarily disable flaky FirstRunIntegrationTests

BUG=911316

Change-Id: If2b2c01109655b3f9066776a716ac811d8afa103
Reviewed-on: https://chromium-review.googlesource.com/c/1362103
Reviewed-by: Theresa <twellington@chromium.org>
Commit-Queue: Peter Kotwicz <pkotwicz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#613742}
[modify] https://crrev.com/b0c7af7b9ba7dd8288e7eed6d98107d9fd188a3f/chrome/android/javatests/src/org/chromium/chrome/browser/firstrun/FirstRunIntegrationTest.java

CLs re-enabling some of the tests are up:
https://chromium-review.googlesource.com/c/chromium/src/+/1362812
https://chromium-review.googlesource.com/c/chromium/src/+/1362008

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1ca54644ff73878027d7903ab3bd6288561e68a8

commit 1ca54644ff73878027d7903ab3bd6288561e68a8
Author: Peter Kotwicz <pkotwicz@chromium.org>
Date: Fri Dec 07 06:57:01 2018

[Android Tests] Fix several FirstRunIntegrationTests

This CL converts several FirstRunIntegrationTests to JUnit tests and
re-enables them.

BUG=911316

Change-Id: I7aada68234fe9a5cb25ee3bef29538cec537d49a
Reviewed-on: https://chromium-review.googlesource.com/c/1362812
Commit-Queue: Peter Kotwicz <pkotwicz@chromium.org>
Reviewed-by: Yusuf Ozuysal <yusufo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#614630}
[modify] https://crrev.com/1ca54644ff73878027d7903ab3bd6288561e68a8/chrome/android/java_sources.gni
[modify] https://crrev.com/1ca54644ff73878027d7903ab3bd6288561e68a8/chrome/android/javatests/src/org/chromium/chrome/browser/firstrun/FirstRunIntegrationTest.java
[add] https://crrev.com/1ca54644ff73878027d7903ab3bd6288561e68a8/chrome/android/junit/src/org/chromium/chrome/browser/firstrun/FirstRunIntegrationUnitTest.java

Issue 907548 has been merged into this issue.

Issue 887493 has been merged into this issue.