NewTabPage *EnableInterestFeed unittests suddenly got very flaky |
||||
Issue descriptionIs this due to recent crrev.com/c/1294194? Some of the failures: org.chromium.chrome.browser.ntp.NewTabPageTest#testRender_SignInPromo__EnableInterestFeed org.chromium.chrome.browser.ntp.NewTabPageTest.testRender_ArticleSectionHeader__EnableInterestFeed Only seems to happen on some builders like kitkat: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=chrome_public_test_apk&tests=_EnableInterestFeed Here's an error log from org.chromium.chrome.browser.ntp.NewTabPageTest.testRender_SignInPromo__EnableInterestFeed: org.chromium.chrome.test.util.RenderTestRule$RenderTestException: RenderTest Mismatches for: sign_in_promo. See RENDER_TESTS.md for how to fix this failure. at org.chromium.chrome.test.util.RenderTestRule.finished(RenderTestRule.java:238) at org.junit.rules.TestWatcher.finishedQuietly(TestWatcher.java:117) at org.junit.rules.TestWatcher.access$400(TestWatcher.java:46) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:64) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.chromium.base.test.BaseJUnit4ClassRunner.runChild(BaseJUnit4ClassRunner.java:237) at org.chromium.base.test.BaseJUnit4ClassRunner.runChild(BaseJUnit4ClassRunner.java:48) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.chromium.base.test.BaseJUnit4ClassRunner.run(BaseJUnit4ClassRunner.java:220) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at android.support.test.internal.runner.TestExecutor.execute(TestExecutor.java:59) at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:262) at org.chromium.base.test.BaseChromiumAndroidJUnitRunner.onStart(BaseChromiumAndroidJUnitRunner.java:125) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701) Searching for native crashes in: /b/swarming/w/itlw2kx_/tmpvulR36 Unknown Android release, consider passing --packed-lib. Reading Android symbols from: /b/swarming/w/ir Searching for Chrome symbols from within: /b/swarming/w/ir/out/Release/lib.unstripped:/b/swarming/w/ir/out/Release
,
Oct 23
P1 to either disable tests, revert the change that caused the flakiness or land some fixes.
,
Oct 23
I'm working on a fix, but let's disable them first
,
Oct 23
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/98cb9830917f3f18d1672220550478bb2455c16a commit 98cb9830917f3f18d1672220550478bb2455c16a Author: Becky Zhou <huayinz@chromium.org> Date: Tue Oct 23 21:06:15 2018 Disable some flaky render tests on NewTabPageTest TBR=twellington@chromium.org Bug: 898165 Change-Id: Id519840866ae56a2ee795ee73ba505a1a3645559 Reviewed-on: https://chromium-review.googlesource.com/c/1297057 Reviewed-by: Becky Zhou <huayinz@chromium.org> Commit-Queue: Becky Zhou <huayinz@chromium.org> Cr-Commit-Position: refs/heads/master@{#602091} [modify] https://crrev.com/98cb9830917f3f18d1672220550478bb2455c16a/chrome/android/javatests/src/org/chromium/chrome/browser/ntp/NewTabPageTest.java
,
Oct 23
Actually only NewTabPageTest#testRender_ArticleSectionHeader__EnableInterestFeed and NewTabPageTest#testRender_SignInPromo__EnableInterestFeed are flaky for different reasons. 1) testRender_SignInPromo__EnableInterestFeed was already flaky before and is related to the button under focused state during render test check. https://00e9e64bac2f5f486de2a7abfe2cf06a43d32ab186d3562a56-apidata.googleusercontent.com/download/storage/v1/b/chromium-result-details/o/a563e8fbb2cd1bf402738d59f354ca701a4d93e9?qk=AD5uMEtk1A-beO2Vk4-wYKwl1Hu_aQfVGV9mxaJQPTFJgbR4b-eYDVMt7a06tZdVawWaR-ZQT2ClaV7SR1mJ4_togLT8041VJ2TdZQW9ZX51uU1wPHh5WPeIj7criGhG_Z7StwNqLReJa4AEVv5Y-ooYVF6w0V7AJnwqNne_2PQC35QauDKM4vtwhWXAEqq_jsO8Nbg1BMaMPAqPkp97FZl37EnQx5OkhEgbizanUc46qIBr5kNUGp3hllH7xb9BjMvZTpPiM7DkmWV2HWyoA8jN7SdKMZvdrlnJTpNqI0KjUEjBqimNqXx5vu1IYG7yGfT1ukHMy1gTaQaeInIuZ_6WSL-9lLP_A09CkHILe3G-HjlPxJhEo6McP_yQ1rvbsGrCl9rpdg2pq9nflv55IY5BG8qrOO7EVXmTaxyBkQipSDFuJm6fZTUOwDHGOsyd8OumijuKTxwApdIz1NjN19gOYNJMqX0gd7RYllxazXO6M0uumH-GmPG27BvB28gQ7ymcrmDZwgtx4gkAInL1EjDHlwB-R06shAsVx_2-61i879S9oViDk27kTTpF_Xpvh8KoWdrPNUELgkqAsAb2JSRxWcdqY64XuNN2PAt2I0yu__BO39uc5n9pgC6baeipAv_n-TLwR50jLdO_WtQBam6jB0KNHYfjStUZaNo-BlicZMz07U3L-pigEGszf304xjIbMLwSD_auVOHlAacdG5xStKDwJDI9SdfBUhdqcgr6gIXBVElh39B5eupk4mRyL-qWfB6Iny_mPaPsg1xqblOv7I3hf0smvyO3XldnXk5vWlZDBCMxtxoBWEtUzQV4_11urHL2b5Ow 2) But testRender_ArticleSectionHeader__EnableInterestFeed is a recent flake staring 10/22 (first flake: https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-kitkat-arm-rel/108734). I only changed the SignInPromo for Zine, but haven't touched FeedNTP for a while, also it is flaking already before my patch is landed (my patch is landed at https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-kitkat-arm-rel/108845). I wonder what recent change could affect this...
,
Oct 24
,
Oct 26
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6f70af53a445db358606969bab775dce25faf709 commit 6f70af53a445db358606969bab775dce25faf709 Author: Becky Zhou <huayinz@chromium.org> Date: Fri Oct 26 00:05:51 2018 Speculative fix for NewTabPageTest#testRender_ArticleSectionHeader Bug: 898165 Change-Id: I89612def0f3c67ad04fc937f80f12162dc4662cc Reviewed-on: https://chromium-review.googlesource.com/c/1297174 Reviewed-by: Theresa <twellington@chromium.org> Commit-Queue: Becky Zhou <huayinz@chromium.org> Cr-Commit-Position: refs/heads/master@{#602939} [modify] https://crrev.com/6f70af53a445db358606969bab775dce25faf709/chrome/android/javatests/src/org/chromium/chrome/browser/ntp/NewTabPageTest.java |
||||
►
Sign in to add a comment |
||||
Comment 1 by huayinz@chromium.org
, Oct 23Status: Assigned (was: Untriaged)