New issue
Advanced search Search tips

Issue 911316 link

Starred by 4 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: Bug



Sign in to add a comment

org.chromium.chrome.browser.firstrun.FirstRunIntegrationTest#testDefaultSearchEngine_DontShow is very flaky

Project Member Reported by pdr@chromium.org, Dec 3

Issue description

This test is very flaky:
https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=chrome_public_test_apk&tests=testDefaultSearchEngine_DontShow

Error:
java.lang.AssertionError
	at org.junit.Assert.fail(Assert.java:86)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.junit.Assert.assertNotNull(Assert.java:712)
	at org.junit.Assert.assertNotNull(Assert.java:722)
	at org.chromium.chrome.browser.firstrun.FirstRunIntegrationTest.runSearchEnginePromptTest(FirstRunIntegrationTest.java:284)
	at org.chromium.chrome.browser.firstrun.FirstRunIntegrationTest.testDefaultSearchEngine_DontShow(FirstRunIntegrationTest.java:243)
	at java.lang.reflect.Method.invoke(Native Method)
 
Cc: pkotw...@chromium.org yus...@chromium.org
Components: UI>Browser>FirstRun
Labels: -Pri-3 sheriff-anrdoid Pri-1
It looks like this whole test suite is pretty flaky except for #testHelpPageSkipsFirstRun. cc'ing other first run owners.

Will you please help investigate/triage?

Flakiness dashboard: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=chrome_public_test_apk&tests=FirstRunIntegrationTest
Labels: OS-Android
I am in the process of rewriting runFirstRunRedirectTestForActivity() and testAbortFirstRun() as JUnit tests (in a simillar pattern as https://chrome-internal-review.googlesource.com/c/clank/internal/apps/+/724688). If I feel ambitious, I will look at the other tests too.

Robolectric is really good for checking for started activities.
These tests are blocking the CQ which blocks patches from landing. Is switching to JUnit a small change? If not, could these tests be disabled to unblock the CQ?
I think that disabling the tests to unblock the CQ makes sense.

- I am running the tests locally and it seems like the initial SharedPreferences state is incorrect
- Switching the tests to be JUnit tests is not a small change. (Likely 1-2 hours of work)
Would a quicker fix be setting or resetting the SharedPreference state before running the tests?
Note that there were some related tests disabled during the DEX limit issue a month ago that are in a similar boat -- they need SharedPreferences cleared to pass correctly: see issue 902774.

It'd be great to get those re-enabled too.
Project Member

Comment 8 by bugdroid1@chromium.org, Dec 4

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b0c7af7b9ba7dd8288e7eed6d98107d9fd188a3f

commit b0c7af7b9ba7dd8288e7eed6d98107d9fd188a3f
Author: Peter Kotwicz <pkotwicz@chromium.org>
Date: Tue Dec 04 23:03:48 2018

Temporarily disable flaky FirstRunIntegrationTests

BUG=911316

Change-Id: If2b2c01109655b3f9066776a716ac811d8afa103
Reviewed-on: https://chromium-review.googlesource.com/c/1362103
Reviewed-by: Theresa <twellington@chromium.org>
Commit-Queue: Peter Kotwicz <pkotwicz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#613742}
[modify] https://crrev.com/b0c7af7b9ba7dd8288e7eed6d98107d9fd188a3f/chrome/android/javatests/src/org/chromium/chrome/browser/firstrun/FirstRunIntegrationTest.java

Cc: bsazonov@chromium.org
Owner: pkotw...@chromium.org
CLs re-enabling some of the tests are up:
https://chromium-review.googlesource.com/c/chromium/src/+/1362812
https://chromium-review.googlesource.com/c/chromium/src/+/1362008
Project Member

Comment 10 by bugdroid1@chromium.org, Dec 7

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1ca54644ff73878027d7903ab3bd6288561e68a8

commit 1ca54644ff73878027d7903ab3bd6288561e68a8
Author: Peter Kotwicz <pkotwicz@chromium.org>
Date: Fri Dec 07 06:57:01 2018

[Android Tests] Fix several FirstRunIntegrationTests

This CL converts several FirstRunIntegrationTests to JUnit tests and
re-enables them.

BUG=911316

Change-Id: I7aada68234fe9a5cb25ee3bef29538cec537d49a
Reviewed-on: https://chromium-review.googlesource.com/c/1362812
Commit-Queue: Peter Kotwicz <pkotwicz@chromium.org>
Reviewed-by: Yusuf Ozuysal <yusufo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#614630}
[modify] https://crrev.com/1ca54644ff73878027d7903ab3bd6288561e68a8/chrome/android/java_sources.gni
[modify] https://crrev.com/1ca54644ff73878027d7903ab3bd6288561e68a8/chrome/android/javatests/src/org/chromium/chrome/browser/firstrun/FirstRunIntegrationTest.java
[add] https://crrev.com/1ca54644ff73878027d7903ab3bd6288561e68a8/chrome/android/junit/src/org/chromium/chrome/browser/firstrun/FirstRunIntegrationUnitTest.java

Issue 907548 has been merged into this issue.
Issue 887493 has been merged into this issue.
Labels: android-fe-triaged
Status: Assigned (was: Untriaged)

Sign in to add a comment