org.chromium.android_webview.test.AwContentsClientFullScreenTest#testOnShowCustomViewAndPlayWithHtmlControl_videoInsideDiv__multiprocess_mode flaky on android_n5x_swarming_rel |
||||||||
Issue descriptionFailure seen here: https://ci.chromium.org/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/307943 Stack trace: java.lang.AssertionError: Criteria not met in allotted time. at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.chromium.content.browser.test.util.CriteriaHelper.pollInstrumentationThread(CriteriaHelper.java:93) at org.chromium.content.browser.test.util.DOMUtils.waitForMediaPlay(DOMUtils.java:105) at org.chromium.android_webview.test.AwContentsClientFullScreenTest.doTestOnShowCustomViewAndPlayWithHtmlControl(AwContentsClientFullScreenTest.java:218) at org.chromium.android_webview.test.AwContentsClientFullScreenTest.testOnShowCustomViewAndPlayWithHtmlControl_videoInsideDiv(AwContentsClientFullScreenTest.java:210) at java.lang.reflect.Method.invoke(Native Method) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:52) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.chromium.base.test.ScreenshotOnFailureStatement.evaluate(ScreenshotOnFailureStatement.java:37) at org.chromium.android_webview.test.AwActivityTestRule$1.evaluate(AwActivityTestRule.java:72) at android.support.test.internal.statement.UiThreadStatement.evaluate(UiThreadStatement.java:55) at android.support.test.rule.ActivityTestRule$ActivityStatement.evaluate(ActivityTestRule.java:270) ... Detail link: https://00e9e64bac537b28849a97e305726b284f3e321efc1c5201d1-apidata.googleusercontent.com/download/storage/v1/b/chromium-result-details/o/html%2Fwebview_instrumentation_test_apk_android_n5x_swarming_rel_307943_2017_11_18_T01_00_17-UTC?qk=AD5uMEuUfRPGt2vgXFOGjGMaavuR0GIYVMQYf2NUwoE8DqiiQLcR0s4XN3EEG1h4t5AzT_RTgwW9Jhax9uN_2tUUMl_ysQed5_3Rjo0HJwXBMLC0KhAGn3GyFB2gyYAyZxjDnTt49ca1YPGjN_GF6IYbtuVnfWu5MhmG0EJg8kxd5IAoTJ6_m9KlWx1iSpyq-QCSgC2FH_JKxBL-eyfOO-2ZaKohlDijb9mX39sVRR7fq64HCZnlfzYcCUwaxgxDkpaLTekeIiOolzjiZAOU-ffor_nrODYPQyUCvzwbrd_MP41AoML2eMlVWRLnO9CaBKDcbtZeFT1njEWNwU1JgkmEILDmrAmBZ8V_2IGL4bnX21MbuPb75LCCFk8Y43xFkECDkUm0n2PlR_xKM4dDRs4prmw6B4IHBXmkPDyJLWXqsD0r172hhUrNJ6hhvA-IElf1_PZVbrqGYSJ0RuVTJQPn26sJrwzSpymMPOgPTQ-q6GGjZY0xqZjz7xczdyTO1gHHDMXPzZShNss4xbxG1uf8sIVe_9OD4ni2Ms5wghNgKGcRg0Sl8S3m9mCVI2WoXAKEZEFzYfQfbMBO0MAHWtsf9ECJyOKiKdLPtmCToT5YF-L63C1ZTBq9VtEMzJZHKR8e7jI76wa9HcTPYiVE9OHBYFqNZfX1ncGwaMkQWAmuRih9dEg4v_UriJcLsDywvvEik2g5n3-xoWnkG6op7xUQ9dz1mYAOfSiBZeCIbkGXf-0IkXS1teJU479DCu6P0JgJiV3CM6Vjya7gB8_kwoRphJD9B_6yy2dAJI5rb41RGzG1Y2QA5eXzcSpQPsdVe0_cY57XWFg6F2VIJyr8HDvSvFW53xiPSwL-hHKGskv9hXORKRjivtY logcat link: https://luci-logdog.appspot.com/v/?s=chromium%2Fandroid%2Fswarming%2Flogcats%2F39e56962de910211%2F%2B%2Flogcat_logcat_org.chromium.android_webview.test.AwContentsClientFullScreenTest.testOnShowCustomViewAndPlayWithHtmlControl_videoInsideDiv__multiprocess_mode_20171118T005335-UTC_01e8be2690ac8f69 I don't see any native crashes in here. Marking P1. Tests must be reliable.
,
Nov 20 2017
Given that, removing Sheriff-Chromium while leaving the bug open for investigation.
,
Nov 20 2017
Won't-fixing this given that there are no flakes recently.
,
Nov 20 2017
+jbudorick: discussion.. It looks like the flakiness dashboard is totally useless for tracking flakiness on the tryservers? https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=webview_instrumentation_test_apk&tests=org.chromium.android_webview.test.AwContentsClientFullScreenTest%23testOnShowCustomViewAndPlayWithHtmlControl_videoInsideDiv__multiprocess_mode All of the question mark/no data boxes are actually failures in the webview test suite in some other test, which is kinda unexpected. And I guess even if they are red, it doesn't mean much overall since these are tryservers.. We really need tryflakes back I guess. Looking at the test, it's clicking on a dom object which should start the video playing, but looking at the screenshot, the video never have started. I think this is a general class of flakiness issues with clicking on dom nodes with DOMUtils. Maybe someone can try turning off checks for user gesture, and just click on things with js instead. Probably can't work as a general strategy for all tests, but if this works out well, maybe should consider doing this change more widely.
,
Nov 20 2017
,
Nov 20 2017
jbudorick for realz: See #4
,
Nov 20 2017
#4: mentioned this to ted in another bug. you want the (with patch) version, i.e., https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=webview_instrumentation_test_apk%20(with%20patch)&tests=org.chromium.android_webview.test.AwContentsClientFullScreenTest%23testOnShowCustomViewAndPlayWithHtmlControl_videoInsideDiv__multiprocess_mode that said, given the limited history presented by the dashboard vs the frequency of trybot runs, it's still not terribly useful. It isn't fair to say that this has no flakes recently when kbr posted a link of it flaking on Friday, but the flakiness dashboard makes it hard to even look back that far (all the way back to ... the previous business day)
,
Nov 20 2017
(even "Use all recorded runs" only goes back to yesterday morning)
,
Dec 1 2017
This test hasn't all week failed as far as I've seen, checking regularly on the dashboard. I filed crbug.com/791019 to follow up on Bo's suggestion in comment #4, and I'm closing this in favour of that.
,
Dec 4 2017
|
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by tyoshino@chromium.org
, Nov 20 2017