Redirect navigation EarlGrey tests are flaky |
|||
Issue descriptionThe following tests are very flaky on iOS 11 iPad NavigationTestCase.testRedirectWindow NavigationTestCase.testRedirectRefresh About 20% of runs are red, which means the test fails three in a row. In most recent runs, one of these two tests fails once, but passes on rerun. The first failure, may be here (it's hard to tell due to flake, and the previous ~6 attempts didn't run): https://build.chromium.org/p/chromium.fyi/builders/EarlGreyiOS/builds/32090 47caa4fc1ffac8572425adf8434a86bed87e This was the prior successful run (and the 4 previous runs also succeeded) https://build.chromium.org/p/chromium.fyi/builders/EarlGreyiOS/builds/32083 revision - e685bb01d1ed700492cda24ef85615a218
,
Oct 5 2017
The first CL lists the revisions I looked between, here are the hashes: e685bb01d1ed [Android FRE] Make parameters of FirstRunFlowSequencer#markFlowAsCompleted() clearer 3b0c080b0bb2 Reland "Media Controls: Load CSS directly in Blink" 5b2b5038c423 [Android] Record UMA action when GMS error dialog is accepted 8db59b970550 Fix favicon in case of same document navigation. cd652c6677a9 Jumbo for blink/platform/scheduler and its unit tests (-3.5 CPU minutes) ada41c462c0d Display image metadata for more formats than just JPEG and ICO. 57f7ae792d99 transition-end-event-multiple-03.html is flaky 9aee02a81a5c Update V8 to version 6.3.249. 23d7fb4389c9 Add a test that opens quick view, closes it, and opens it again on a folder. 33d44ee3dd44 cc: Disable edge anti-aliasing for the UI fde0d11b0a99 BlinkGC: Document rules for non newable classes a528bfe58a1a V4L2SVDA: ensure all PictureReady are sent while resolution change f6f4d6734cab Use URL formatter in the context title in notification. cc1ab7013fef S13nServiceWorker: Add a test for large scripts in ServiceWorkerScriptURLLoader 364b4e7c096e Make SnapStrictness an enum class. 22be5ece562b Don't try to show layout test results inside testing/xvfb.py 21114a5b119d Files App: Truncate 'Add new service' text Again, it's flake so it is possible it started earlier, and we got "lucky" on a few runs. Also, if nothing looks to have contributed, we can look to see if any simulators or devices were recently updates.
,
Oct 5 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/35f0d57610051b8ebadc379a0301ae1234826e2c commit 35f0d57610051b8ebadc379a0301ae1234826e2c Author: Mike Baxley <baxley@chromium.org> Date: Thu Oct 05 21:24:23 2017 Mark redirect navigation EarlGrey tests as flaky. testRedirectWindow and testRedirectRefresh are very falky on iOS 11 iPads. TBR: eugenebut Bug: 772128 Change-Id: Icacf5d4c8f0cad3693d161ad31ade70ecbd960c7 Reviewed-on: https://chromium-review.googlesource.com/703469 Reviewed-by: Mike Baxley <baxley@chromium.org> Commit-Queue: Mike Baxley <baxley@chromium.org> Cr-Commit-Position: refs/heads/master@{#506864} [modify] https://crrev.com/35f0d57610051b8ebadc379a0301ae1234826e2c/ios/chrome/browser/web/navigation_egtest.mm
,
Oct 5 2017
I don't think there was a breakage. I'm surprised that these tests ever worked given the code which does not wait for navigation completion:
[ChromeEarlGrey loadURL:indexURL];
TapWebViewElementWithId(redirectLabel);
[[EarlGrey selectElementWithMatcher:chrome_test_util::OmniboxText(
destinationURL.GetContent())]
assertWithMatcher:grey_notNil()];
,
Oct 10 2017
Eugene, so what is your plan here?
,
Oct 10 2017
The plan is to fix the tests. Application is fine, so we don't need to block beta release with this bug.
,
Oct 17 2017
Have you made a progress fixing the test Eugene?
,
Oct 17 2017
Tests are only disabled on iPad and should not block beta release.
,
Oct 18 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d070ceb3c987d77ec9abb69ddf73a42aea07539e commit d070ceb3c987d77ec9abb69ddf73a42aea07539e Author: Eugene But <eugenebut@google.com> Date: Wed Oct 18 22:58:09 2017 Fixed -[NavigationTestCase verifyBackAndForwardAfterRedirect:]. Added code which waits for page loads after navigations. Disabled tests will be enabled in a separate CL. Also improved code formatting were OmniboxText is used. Bug: 772128 Change-Id: I93880d1053e37e012240c7013d3e642b17acf38e Reviewed-on: https://chromium-review.googlesource.com/726241 Reviewed-by: Mike Baxley <baxley@chromium.org> Commit-Queue: Eugene But <eugenebut@chromium.org> Cr-Commit-Position: refs/heads/master@{#509909} [modify] https://crrev.com/d070ceb3c987d77ec9abb69ddf73a42aea07539e/ios/chrome/browser/web/navigation_egtest.mm
,
Nov 9 2017
,
Nov 9 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/145b9447374ef27ef091cac335b3043e44fdda55 commit 145b9447374ef27ef091cac335b3043e44fdda55 Author: Eugene But <eugenebut@google.com> Date: Thu Nov 09 23:43:48 2017 Enabled testRedirectWindow and testRedirectRefresh EG tests. Tests were actually flaky and the fix was landed in crrev.com/c/726241 Bug: 772128 Change-Id: I371183a089cba7b98937cb198eb4bea4238ce824 Reviewed-on: https://chromium-review.googlesource.com/761056 Reviewed-by: Yuke Liao <liaoyuke@chromium.org> Commit-Queue: Yuke Liao <liaoyuke@chromium.org> Cr-Commit-Position: refs/heads/master@{#515348} [modify] https://crrev.com/145b9447374ef27ef091cac335b3043e44fdda55/ios/chrome/browser/web/navigation_egtest.mm |
|||
►
Sign in to add a comment |
|||
Comment 1 by baxley@chromium.org
, Oct 5 2017Labels: -Pri-2 Pri-1
Owner: eugene...@chromium.org
Status: Assigned