New issue
Advanced search Search tips

Issue 921089 link

Starred by 1 user
Status: Fixed
Owner:
ctzsm@chromium.org
Closed: Jan 19
Cc:
changwan@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Bug



Sign in to add a comment

content.browser.input.TextSuggestionMenuTest#testSetComposingTextRemovesAutoCorrectionSuggestionSpan is flaky

Project Member Reported by adithyas@chromium.org, Jan 11 
Stack Trace:
java.util.concurrent.TimeoutException: waitForCallback timed out!
	at org.chromium.base.test.util.CallbackHelper.waitForCallback(CallbackHelper.java:191)
	at org.chromium.base.test.util.CallbackHelper.waitForCallback(CallbackHelper.java:202)
	at org.chromium.content_public.browser.test.util.TestCallbackHelperContainer$OnEvaluateJavaScriptResultHelper.waitUntilHasValue(TestCallbackHelperContainer.java:157)
	at org.chromium.content_public.browser.test.util.JavaScriptUtils.executeJavaScriptAndWaitForResult(JavaScriptUtils.java:54)
	at org.chromium.content_public.browser.test.util.JavaScriptUtils.executeJavaScriptAndWaitForResult(JavaScriptUtils.java:30)
	at org.chromium.content.browser.input.TextSuggestionMenuTest.testSetComposingTextRemovesAutoCorrectionSuggestionSpan(TextSuggestionMenuTest.java:332)
	at java.lang.reflect.Method.invoke(Native Method)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:52)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.chromium.base.test.ScreenshotOnFailureStatement.evaluate(ScreenshotOnFailureStatement.java:41)
	at android.support.test.internal.statement.UiThreadStatement.evaluate(UiThreadStatement.java:55)
	at android.support.test.rule.ActivityTestRule$ActivityStatement.evaluate(ActivityTestRule.java:270)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.chromium.base.test.BaseJUnit4ClassRunner.runChild(BaseJUnit4ClassRunner.java:237)
	at org.chromium.base.test.BaseJUnit4ClassRunner.runChild(BaseJUnit4ClassRunner.java:48)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.chromium.base.test.CommitSharedPreferencesTestRule$1.evaluate(CommitSharedPreferencesTestRule.java:20)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.chromium.base.test.BaseJUnit4ClassRunner.run(BaseJUnit4ClassRunner.java:220)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at android.support.test.internal.runner.TestExecutor.execute(TestExecutor.java:59)
	at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:262)
	at org.chromium.base.test.BaseChromiumAndroidJUnitRunner.onStart(BaseChromiumAndroidJUnitRunner.java:128)
	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1879)

Recent failing build: https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20CFI/4232

Flakiness dashboard: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=content_shell_test_apk&tests=org.chromium.content.browser.input.TextSuggestionMenuTest%23testSetComposingTextRemovesAutoCorrectionSuggestionSpan

Comment 1 by adithyas@chromium.org, Jan 11 

Test appears to have been flaky for a while, so I'll disable it.

ctzsm@: Could https://crrev.com/c/1361727 have caused this?
Project Member

Comment 2 by bugdroid1@chromium.org, Jan 11 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/81536963852fec963bf931b629843ed4d2c3a7bc

commit 81536963852fec963bf931b629843ed4d2c3a7bc
Author: Adithya Srinivasan <adithyas@chromium.org>
Date: Fri Jan 11 18:28:01 2019

Disable testSetComposingTextRemovesAutoCorrectionSuggestionSpan

Tbr: changwan@chromium.org
No-Try: true
Bug:  921089 
Change-Id: I23cd864d189a5b5e9b52a7a169ec576e479f36fe
Reviewed-on: https://chromium-review.googlesource.com/c/1407371
Reviewed-by: Adithya Srinivasan <adithyas@chromium.org>
Commit-Queue: Adithya Srinivasan <adithyas@chromium.org>
Cr-Commit-Position: refs/heads/master@{#622065}
[modify] https://crrev.com/81536963852fec963bf931b629843ed4d2c3a7bc/content/public/android/javatests/src/org/chromium/content/browser/input/TextSuggestionMenuTest.java

Comment 3 by ctzsm@chromium.org, Jan 11

Labels: OS-Android
Thanks for catching this, actually all 3 tests added in https://crrev.com/c/1361727 are flaky and testAutoCorrectionSuggestionSpan added in an earlier test is also flaky.

I'll take a look soon.

https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=content_shell_test_apk&tests=org.chromium.content.browser.input.TextSuggestionMenuTest

Comment 4 by ctzsm@chromium.org, Jan 12 

It might be because a race between executing the validation JavaScript and the test code. It seems sometimes we executed JavaScript before renderer side code finished, since the JavaScript has runtime error, it never returns a value, so it times out.

Comment 5 by battre@chromium.org, Jan 14 

Disabling three more flaky tests to get this out of the sheriff queue:

I  206.000s run_tests_on_device(075c572921159abe)  detected failure in org.chromium.content.browser.input.TextSuggestionMenuTest#testAutoCorrectionSuggestionSpan. raw output:
I  206.000s run_tests_on_device(075c572921159abe)    INSTRUMENTATION_STATUS: numtests=1
I  206.000s run_tests_on_device(075c572921159abe)    INSTRUMENTATION_STATUS: stream=
I  206.000s run_tests_on_device(075c572921159abe)    org.chromium.content.browser.input.TextSuggestionMenuTest:
I  206.001s run_tests_on_device(075c572921159abe)    INSTRUMENTATION_STATUS: id=AndroidJUnitRunner
I  206.001s run_tests_on_device(075c572921159abe)    INSTRUMENTATION_STATUS: test=testAutoCorrectionSuggestionSpan
I  206.001s run_tests_on_device(075c572921159abe)    INSTRUMENTATION_STATUS: class=org.chromium.content.browser.input.TextSuggestionMenuTest
I  206.001s run_tests_on_device(075c572921159abe)    INSTRUMENTATION_STATUS: current=1
I  206.001s run_tests_on_device(075c572921159abe)    INSTRUMENTATION_STATUS_CODE: 1
I  206.001s run_tests_on_device(075c572921159abe)    INSTRUMENTATION_STATUS: numtests=1
I  206.001s run_tests_on_device(075c572921159abe)    INSTRUMENTATION_STATUS: stream=
I  206.002s run_tests_on_device(075c572921159abe)    Error in testAutoCorrectionSuggestionSpan(org.chromium.content.browser.input.TextSuggestionMenuTest):
I  206.002s run_tests_on_device(075c572921159abe)    java.util.concurrent.TimeoutException: waitForCallback timed out!
I  206.002s run_tests_on_device(075c572921159abe)    	at org.chromium.base.test.util.CallbackHelper.waitForCallback(CallbackHelper.java:191)
I  206.002s run_tests_on_device(075c572921159abe)    	at org.chromium.base.test.util.CallbackHelper.waitForCallback(CallbackHelper.java:202)
I  206.002s run_tests_on_device(075c572921159abe)    	at org.chromium.content_public.browser.test.util.TestCallbackHelperContainer$OnEvaluateJavaScriptResultHelper.waitUntilHasValue(TestCallbackHelperContainer.java:157)
I  206.002s run_tests_on_device(075c572921159abe)    	at org.chromium.content_public.browser.test.util.JavaScriptUtils.executeJavaScriptAndWaitForResult(JavaScriptUtils.java:54)
I  206.002s run_tests_on_device(075c572921159abe)    	at org.chromium.content_public.browser.test.util.JavaScriptUtils.executeJavaScriptAndWaitForResult(JavaScriptUtils.java:30)
I  206.002s run_tests_on_device(075c572921159abe)    	at org.chromium.content.browser.input.TextSuggestionMenuTest.testAutoCorrectionSuggestionSpan(TextSuggestionMenuTest.java:308)

I  241.454s run_tests_on_device(023d9784e1431f8f)  detected failure in org.chromium.content.browser.input.TextSuggestionMenuTest#testCommitTextRemovesAutoCorrectionSuggestionSpan. raw output:
I  241.454s run_tests_on_device(023d9784e1431f8f)    INSTRUMENTATION_STATUS: numtests=1
I  241.454s run_tests_on_device(023d9784e1431f8f)    INSTRUMENTATION_STATUS: stream=
I  241.454s run_tests_on_device(023d9784e1431f8f)    org.chromium.content.browser.input.TextSuggestionMenuTest:
I  241.455s run_tests_on_device(023d9784e1431f8f)    INSTRUMENTATION_STATUS: id=AndroidJUnitRunner
I  241.455s run_tests_on_device(023d9784e1431f8f)    INSTRUMENTATION_STATUS: test=testCommitTextRemovesAutoCorrectionSuggestionSpan
I  241.455s run_tests_on_device(023d9784e1431f8f)    INSTRUMENTATION_STATUS: class=org.chromium.content.browser.input.TextSuggestionMenuTest
I  241.455s run_tests_on_device(023d9784e1431f8f)    INSTRUMENTATION_STATUS: current=1
I  241.455s run_tests_on_device(023d9784e1431f8f)    INSTRUMENTATION_STATUS_CODE: 1
I  241.456s run_tests_on_device(023d9784e1431f8f)    INSTRUMENTATION_STATUS: numtests=1
I  241.456s run_tests_on_device(023d9784e1431f8f)    INSTRUMENTATION_STATUS: stream=
I  241.456s run_tests_on_device(023d9784e1431f8f)    Error in testCommitTextRemovesAutoCorrectionSuggestionSpan(org.chromium.content.browser.input.TextSuggestionMenuTest):
I  241.456s run_tests_on_device(023d9784e1431f8f)    org.junit.ComparisonFailure: expected:<[1]> but was:<[0]>
I  241.456s run_tests_on_device(023d9784e1431f8f)    	at org.junit.Assert.assertEquals(Assert.java:115)
I  241.456s run_tests_on_device(023d9784e1431f8f)    	at org.junit.Assert.assertEquals(Assert.java:144)
I  241.456s run_tests_on_device(023d9784e1431f8f)    	at org.chromium.content.browser.input.TextSuggestionMenuTest.testCommitTextRemovesAutoCorrectionSuggestionSpan(TextSuggestionMenuTest.java:364)
I  241.456s run_tests_on_device(023d9784e1431f8f)    	at java.lang.reflect.Method.invoke(Native Method)
I  241.456s run_tests_on_device(023d9784e1431f8f)    	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)

I  258.454s run_tests_on_device(03003b0b08e4be3b)  detected failure in org.chromium.content.browser.input.TextSuggestionMenuTest#testFinishComposingRemovesAutoCorrectionSuggestionSpan. raw output:
I  258.454s run_tests_on_device(03003b0b08e4be3b)    INSTRUMENTATION_STATUS: numtests=1
I  258.455s run_tests_on_device(03003b0b08e4be3b)    INSTRUMENTATION_STATUS: stream=
I  258.455s run_tests_on_device(03003b0b08e4be3b)    org.chromium.content.browser.input.TextSuggestionMenuTest:
I  258.455s run_tests_on_device(03003b0b08e4be3b)    INSTRUMENTATION_STATUS: id=AndroidJUnitRunner
I  258.455s run_tests_on_device(03003b0b08e4be3b)    INSTRUMENTATION_STATUS: test=testFinishComposingRemovesAutoCorrectionSuggestionSpan
I  258.455s run_tests_on_device(03003b0b08e4be3b)    INSTRUMENTATION_STATUS: class=org.chromium.content.browser.input.TextSuggestionMenuTest
I  258.455s run_tests_on_device(03003b0b08e4be3b)    INSTRUMENTATION_STATUS: current=1
I  258.455s run_tests_on_device(03003b0b08e4be3b)    INSTRUMENTATION_STATUS_CODE: 1
I  258.455s run_tests_on_device(03003b0b08e4be3b)    INSTRUMENTATION_STATUS: numtests=1
I  258.455s run_tests_on_device(03003b0b08e4be3b)    INSTRUMENTATION_STATUS: stream=
I  258.455s run_tests_on_device(03003b0b08e4be3b)    Error in testFinishComposingRemovesAutoCorrectionSuggestionSpan(org.chromium.content.browser.input.TextSuggestionMenuTest):
I  258.455s run_tests_on_device(03003b0b08e4be3b)    java.util.concurrent.TimeoutException: waitForCallback timed out!
I  258.455s run_tests_on_device(03003b0b08e4be3b)    	at org.chromium.base.test.util.CallbackHelper.waitForCallback(CallbackHelper.java:191)
I  258.455s run_tests_on_device(03003b0b08e4be3b)    	at org.chromium.base.test.util.CallbackHelper.waitForCallback(CallbackHelper.java:202)
I  258.455s run_tests_on_device(03003b0b08e4be3b)    	at org.chromium.content_public.browser.test.util.TestCallbackHelperContainer$OnEvaluateJavaScriptResultHelper.waitUntilHasValue(TestCallbackHelperContainer.java:157)
I  258.455s run_tests_on_device(03003b0b08e4be3b)    	at org.chromium.content_public.browser.test.util.JavaScriptUtils.executeJavaScriptAndWaitForResult(JavaScriptUtils.java:54)
I  258.455s run_tests_on_device(03003b0b08e4be3b)    	at org.chromium.content_public.browser.test.util.JavaScriptUtils.executeJavaScriptAndWaitForResult(JavaScriptUtils.java:30)
I  258.455s run_tests_on_device(03003b0b08e4be3b)    	at org.chromium.content.browser.input.TextSuggestionMenuTest.testFinishComposingRemovesAutoCorrectionSuggestionSpan(TextSuggestionMenuTest.java:396)

Comment 6 by battre@chromium.org, Jan 14

Labels: -Sheriff-Chromium
CL for disabling the tests is in the CQ.

Disabled test + assigned owner --> Taking out of sheriff queue.
Project Member

Comment 7 by bugdroid1@chromium.org, Jan 14 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4c61f594740e43f15cb736d6e34d724c9f275409

commit 4c61f594740e43f15cb736d6e34d724c9f275409
Author: Dominic Battre <battre@chromium.org>
Date: Mon Jan 14 10:01:38 2019

Disable flaky content.browser.input.TextSuggestionMenuTest tests

Disable the following flaky tests (details in the bug):
- org.chromium.content.browser.input.TextSuggestionMenuTest#testAutoCorrectionSuggestionSpan
- org.chromium.content.browser.input.TextSuggestionMenuTest#testCommitTextRemovesAutoCorrectionSuggestionSpan
- org.chromium.content.browser.input.TextSuggestionMenuTest#testFinishComposingRemovesAutoCorrectionSuggestionSpan

TBR=ctzsm@chromium.org

Bug:  921089 
Change-Id: I144c426beefb7ff2c3b4323285f5e3eb747c6a74
Reviewed-on: https://chromium-review.googlesource.com/c/1407832
Reviewed-by: Dominic Battré <battre@chromium.org>
Commit-Queue: Dominic Battré <battre@chromium.org>
Cr-Commit-Position: refs/heads/master@{#622406}
[modify] https://crrev.com/4c61f594740e43f15cb736d6e34d724c9f275409/content/public/android/javatests/src/org/chromium/content/browser/input/TextSuggestionMenuTest.java
Project Member

Comment 8 by bugdroid1@chromium.org, Jan 14 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/010942cfb4c7f0208b13aa8de6b8c96a35b61aab

commit 010942cfb4c7f0208b13aa8de6b8c96a35b61aab
Author: Shimi Zhang <ctzsm@chromium.org>
Date: Mon Jan 14 20:00:59 2019

[Android IME] Fix flaky test

There is a race condition in these tests, we should wait until renderer
to perform tasks so we could run the JavaScript checking code.

I ran 500 times locally with this fix, it doesn't flaky anymore.

Bug:  921089 
Change-Id: Idbd5dffcf974ec887c4c9b682a6fa18dcc902ac0
Reviewed-on: https://chromium-review.googlesource.com/c/1409357
Reviewed-by: Changwan Ryu <changwan@chromium.org>
Commit-Queue: Shimi Zhang <ctzsm@chromium.org>
Cr-Commit-Position: refs/heads/master@{#622560}
[modify] https://crrev.com/010942cfb4c7f0208b13aa8de6b8c96a35b61aab/content/public/android/javatests/src/org/chromium/content/browser/input/TextSuggestionMenuTest.java

Comment 9 by ctzsm@chromium.org, Jan 19 (4 days ago)

Status: Fixed (was: Assigned)
Except the recent infra failure on KitKat Phone Tester (dbg), I don't see any other flakiness, mark as fixed.
►

Sign in to add a comment