New issue
Advanced search Search tips

Issue 772128 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Nov 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: iOS
Pri: 1
Type: Bug



Sign in to add a comment

Redirect navigation EarlGrey tests are flaky

Project Member Reported by baxley@chromium.org, Oct 5 2017

Issue description

The following tests are very flaky on iOS 11 iPad
NavigationTestCase.testRedirectWindow
NavigationTestCase.testRedirectRefresh

About 20% of runs are red, which means the test fails three in a row. In most recent runs, one of these two tests fails once, but passes on rerun.

The first failure, may be here (it's hard to tell due to flake, and the previous ~6 attempts didn't run):
https://build.chromium.org/p/chromium.fyi/builders/EarlGreyiOS/builds/32090
47caa4fc1ffac8572425adf8434a86bed87e

This was the prior successful run (and the 4 previous runs also succeeded)
https://build.chromium.org/p/chromium.fyi/builders/EarlGreyiOS/builds/32083
revision - e685bb01d1ed700492cda24ef85615a218
 
Cc: michaeldo@chromium.org
Labels: -Pri-2 Pri-1
Owner: eugene...@chromium.org
Status: Assigned
I'm not sure who the best person is to assign to, so the two of you can probably figure out the best person?
The first CL lists the revisions I looked between, here are the hashes:
e685bb01d1ed [Android FRE] Make parameters of FirstRunFlowSequencer#markFlowAsCompleted() clearer
3b0c080b0bb2 Reland "Media Controls: Load CSS directly in Blink"
5b2b5038c423 [Android] Record UMA action when GMS error dialog is accepted
8db59b970550 Fix favicon in case of same document navigation.
cd652c6677a9 Jumbo for blink/platform/scheduler and its unit tests (-3.5 CPU minutes)
ada41c462c0d Display image metadata for more formats than just JPEG and ICO.
57f7ae792d99 transition-end-event-multiple-03.html is flaky
9aee02a81a5c Update V8 to version 6.3.249.
23d7fb4389c9 Add a test that opens quick view, closes it, and opens it again on a folder.
33d44ee3dd44 cc: Disable edge anti-aliasing for the UI
fde0d11b0a99 BlinkGC: Document rules for non newable classes
a528bfe58a1a V4L2SVDA: ensure all PictureReady are sent while resolution change
f6f4d6734cab Use URL formatter in the context title in notification.
cc1ab7013fef S13nServiceWorker: Add a test for large scripts in ServiceWorkerScriptURLLoader
364b4e7c096e Make SnapStrictness an enum class.
22be5ece562b Don't try to show layout test results inside testing/xvfb.py
21114a5b119d Files App: Truncate 'Add new service' text


Again, it's flake so it is possible it started earlier, and we got "lucky" on a few runs. Also, if nothing looks to have contributed, we can look to see if any simulators or devices were recently updates.
Project Member

Comment 3 by bugdroid1@chromium.org, Oct 5 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/35f0d57610051b8ebadc379a0301ae1234826e2c

commit 35f0d57610051b8ebadc379a0301ae1234826e2c
Author: Mike Baxley <baxley@chromium.org>
Date: Thu Oct 05 21:24:23 2017

Mark redirect navigation EarlGrey tests as flaky.

testRedirectWindow and testRedirectRefresh are very falky on iOS 11
iPads.

TBR: eugenebut
Bug:  772128 
Change-Id: Icacf5d4c8f0cad3693d161ad31ade70ecbd960c7
Reviewed-on: https://chromium-review.googlesource.com/703469
Reviewed-by: Mike Baxley <baxley@chromium.org>
Commit-Queue: Mike Baxley <baxley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#506864}
[modify] https://crrev.com/35f0d57610051b8ebadc379a0301ae1234826e2c/ios/chrome/browser/web/navigation_egtest.mm

I don't think there was a breakage. I'm surprised that these tests ever worked given the code which does not wait for navigation completion:

  [ChromeEarlGrey loadURL:indexURL];
  TapWebViewElementWithId(redirectLabel);
  [[EarlGrey selectElementWithMatcher:chrome_test_util::OmniboxText(
                                          destinationURL.GetContent())]
      assertWithMatcher:grey_notNil()];



Comment 5 by cma...@chromium.org, Oct 10 2017

Eugene, so what is your plan here?
The plan is to fix the tests. Application is fine, so we don't need to block beta release with this bug.

Comment 7 by cma...@chromium.org, Oct 17 2017

Have you made a progress fixing the test Eugene?
Labels: -ReleaseBlock-Beta
Tests are only disabled on iPad and should not block beta release.
Project Member

Comment 9 by bugdroid1@chromium.org, Oct 18 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d070ceb3c987d77ec9abb69ddf73a42aea07539e

commit d070ceb3c987d77ec9abb69ddf73a42aea07539e
Author: Eugene But <eugenebut@google.com>
Date: Wed Oct 18 22:58:09 2017

Fixed -[NavigationTestCase verifyBackAndForwardAfterRedirect:].

Added code which waits for page loads after navigations.
Disabled tests will be enabled in a separate CL.

Also improved code formatting were OmniboxText is used.

Bug:  772128 
Change-Id: I93880d1053e37e012240c7013d3e642b17acf38e
Reviewed-on: https://chromium-review.googlesource.com/726241
Reviewed-by: Mike Baxley <baxley@chromium.org>
Commit-Queue: Eugene But <eugenebut@chromium.org>
Cr-Commit-Position: refs/heads/master@{#509909}
[modify] https://crrev.com/d070ceb3c987d77ec9abb69ddf73a42aea07539e/ios/chrome/browser/web/navigation_egtest.mm

Status: Fixed (was: Assigned)
Project Member

Comment 11 by bugdroid1@chromium.org, Nov 9 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/145b9447374ef27ef091cac335b3043e44fdda55

commit 145b9447374ef27ef091cac335b3043e44fdda55
Author: Eugene But <eugenebut@google.com>
Date: Thu Nov 09 23:43:48 2017

Enabled testRedirectWindow and testRedirectRefresh EG tests.

Tests were actually flaky and the fix was landed in crrev.com/c/726241

Bug:  772128 
Change-Id: I371183a089cba7b98937cb198eb4bea4238ce824
Reviewed-on: https://chromium-review.googlesource.com/761056
Reviewed-by: Yuke Liao <liaoyuke@chromium.org>
Commit-Queue: Yuke Liao <liaoyuke@chromium.org>
Cr-Commit-Position: refs/heads/master@{#515348}
[modify] https://crrev.com/145b9447374ef27ef091cac335b3043e44fdda55/ios/chrome/browser/web/navigation_egtest.mm

Sign in to add a comment