New issue
Advanced search Search tips

Issue 900970 link

Starred by 1 user
Status: Untriaged
Owner: ----
Cc:
chanli@chromium.org
liaoyuke@chromium.org
jbudorick@chromium.org
st...@chromium.org
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Flaky failures in org.chromium.chrome.browser.SubresourceFilterTest#resourceNotFiltered

Project Member Reported by erikc...@chromium.org, Nov 1 
In this task:
https://chromium-swarm.appspot.com/task?id=40e42d41ebdaeb10&refresh=10&show_raw=1

The test SubresourceFilterTest#resourceNotFiltered fails, and then fails on all 3 retries. This suggests that our retry mechanism could be more robust.

The CL is completely unrelated: https://chromium-review.googlesource.com/c/chromium/src/+/1292409/19
The test passes when the build is retried: https://ci.chromium.org/b/8931126389648192368

"""
I  920.507s run_tests_on_device(06c12c50003b8099)    Error in resourceNotFiltered(org.chromium.chrome.browser.SubresourceFilterTest):
I  920.507s run_tests_on_device(06c12c50003b8099)    java.lang.AssertionError: Criteria not met in allotted time.
I  920.507s run_tests_on_device(06c12c50003b8099)    	at org.junit.Assert.fail(Assert.java:88)
I  920.507s run_tests_on_device(06c12c50003b8099)    	at org.junit.Assert.assertTrue(Assert.java:41)
I  920.508s run_tests_on_device(06c12c50003b8099)    	at org.chromium.content_public.browser.test.util.CriteriaHelper.pollInstrumentationThread(CriteriaHelper.java:92)
I  920.508s run_tests_on_device(06c12c50003b8099)    	at org.chromium.content_public.browser.test.util.CriteriaHelper.pollUiThread(CriteriaHelper.java:171)
I  920.508s run_tests_on_device(06c12c50003b8099)    	at org.chromium.content_public.browser.test.util.CriteriaHelper.pollUiThread(CriteriaHelper.java:193)
I  920.508s run_tests_on_device(06c12c50003b8099)    	at org.chromium.chrome.browser.SubresourceFilterTest.createAndPublishRulesetDisallowingSuffix(SubresourceFilterTest.java:63)
I  920.508s run_tests_on_device(06c12c50003b8099)    	at org.chromium.chrome.browser.SubresourceFilterTest.setUp(SubresourceFilterTest.java:82)
I  920.508s run_tests_on_device(06c12c50003b8099)    	at java.lang.reflect.Method.invokeNative(Method.java)
I  920.508s run_tests_on_device(06c12c50003b8099)    	at java.lang.reflect.Method.invoke(Method.java:515)
I  920.508s run_tests_on_device(06c12c50003b8099)    	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
I  920.508s run_tests_on_device(06c12c50003b8099)    	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
I  920.508s run_tests_on_device(06c12c50003b8099)    	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:52)
I  920.508s run_tests_on_device(06c12c50003b8099)    	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
I  920.508s run_tests_on_device(06c12c50003b8099)    	at org.chromium.base.test.ScreenshotOnFailureStatement.evaluate(ScreenshotOnFailureStatement.java:37)
I  920.508s run_tests_on_device(06c12c50003b8099)    	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
I  920.509s run_tests_on_device(06c12c50003b8099)    	at org.chromium.chrome.test.ChromeActivityTestRule$1.evaluate(ChromeActivityTestRule.java:124)
I  920.509s run_tests_on_device(06c12c50003b8099)    	at android.support.test.internal.statement.UiThreadStatement.evaluate(UiThreadStatement.java:55)
I  920.509s run_tests_on_device(06c12c50003b8099)    	at android.support.test.rule.ActivityTestRule$ActivityStatement.evaluate(ActivityTestRule.java:270)
I  920.509s run_tests_on_device(06c12c50003b8099)    	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
I  920.509s run_tests_on_device(06c12c50003b8099)    	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
I  920.509s run_tests_on_device(06c12c50003b8099)    	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
I  920.509s run_tests_on_device(06c12c50003b8099)    	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
I  920.509s run_tests_on_device(06c12c50003b8099)    	at org.chromium.base.test.BaseJUnit4ClassRunner.runChild(BaseJUnit4ClassRunner.java:237)
I  920.509s run_tests_on_device(06c12c50003b8099)    	at org.chromium.base.test.BaseJUnit4ClassRunner.runChild(BaseJUnit4ClassRunner.java:48)
I  920.509s run_tests_on_device(06c12c50003b8099)    	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
I  920.509s run_tests_on_device(06c12c50003b8099)    	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
I  920.509s run_tests_on_device(06c12c50003b8099)    	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
I  920.509s run_tests_on_device(06c12c50003b8099)    	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
I  920.509s run_tests_on_device(06c12c50003b8099)    	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
I  920.509s run_tests_on_device(06c12c50003b8099)    	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
I  920.510s run_tests_on_device(06c12c50003b8099)    	at org.chromium.base.test.BaseJUnit4ClassRunner.run(BaseJUnit4ClassRunner.java:220)
I  920.510s run_tests_on_device(06c12c50003b8099)    	at org.junit.runners.Suite.runChild(Suite.java:128)
I  920.510s run_tests_on_device(06c12c50003b8099)    	at org.junit.runners.Suite.runChild(Suite.java:27)
I  920.510s run_tests_on_device(06c12c50003b8099)    	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
I  920.510s run_tests_on_device(06c12c50003b8099)    	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
I  920.510s run_tests_on_device(06c12c50003b8099)    	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
I  920.510s run_tests_on_device(06c12c50003b8099)    	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
I  920.510s run_tests_on_device(06c12c50003b8099)    	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
I  920.510s run_tests_on_device(06c12c50003b8099)    	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
I  920.511s run_tests_on_device(06c12c50003b8099)    	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
I  920.511s run_tests_on_device(06c12c50003b8099)    	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
I  920.511s run_tests_on_device(06c12c50003b8099)    	at android.support.test.internal.runner.TestExecutor.execute(TestExecutor.java:59)
I  920.511s run_tests_on_device(06c12c50003b8099)    	at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:262)
I  920.511s run_tests_on_device(06c12c50003b8099)    	at org.chromium.base.test.BaseChromiumAndroidJUnitRunner.onStart(BaseChromiumAndroidJUnitRunner.java:125)
I  920.511s run_tests_on_device(06c12c50003b8099)    	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701)
I  920.511s run_tests_on_device(06c12c50003b8099)    
"""

+liaoyuke -- I don't see any instances of this on the new flake detector: https://findit-for-me.appspot.com/ranked-flakes. Is this expected?

Comment 1 by liaoyuke@chromium.org, Nov 2

Cc: chanli@chromium.org
Actually, it is on the dashboard, though not on the first page because its impacted number of CLs is not the highest.

https://findit-for-me.appspot.com/ranked-flakes?test_filter=org.chromium.chrome.browser.SubresourceFilterTest%23resourceNotFiltered

There should be only one search result instead of two, most likely there is a bug in the detector, +chanli@ to confirm, but in any case, that's a different issue.

Comment 2 by chanli@chromium.org, Nov 2 

The data is impacted by the  bug 900148 , I planned to clean the data up but get distracted by other tasks. I'll see if I can have some time today to fix the data.

Comment 3 by erikc...@chromium.org, Nov 2 

Am I searching for it wrong?
Screen Shot 2018-11-02 at 12.30.36 PM.png
36.8 KB View Download

Comment 4 by chanli@chromium.org, Nov 2 

Currently we only support searching by full test name or test suite name for gtest. 

We are working on supporting more searching cases.
►

Sign in to add a comment