New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 620353 link

Starred by 2 users

Issue metadata

Status: Verified
Owner:
Closed: Jun 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

Frequent intermittent failures in SAMLPolicyTest

Project Member Reported by steve...@chromium.org, Jun 15 2016

Issue description

Since the introduction of this CL:

Clear the login webview when SAML flow is canceled
https://codereview.chromium.org/1988113004

We have seen frequent but intermittent failures (24/28 failed) in SAMLPolicyTest:
https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%28dbg%29%281%29?numbuilds=100

We need to either revert the change or fix the test quickly.

 
Cc: jdufault@chromium.org
Labels: -Pri-0 Pri-1
Owner: emaxx@chromium.org
Reverted in [1]; assigning to emaxx@ to fix the broken test and lowering to p1 (it's not immediately obvious to me why the test broke).

1: https://codereview.chromium.org/2068953003/

Comment 2 Deleted

Comment 3 by emaxx@chromium.org, Jun 15 2016

(deleted the previous comment due to typos)

Hmm, isn't the builder URL still reporting failures despite reverting of my CL?
https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%28dbg%29%281%29?numbuilds=100

I tried running this SAMLPolicyTest.SAMLInterstitialNext test locally, and it works for me regardless of my CL when the network connection is present, and fails for me regardless of my CL when the network connection is missing (tested with "unshare" tool).
Not sure, however, whether my environment is following the builder environment completely.
It may be a timing issue. The test lab builders tend to be heavily loaded.
I would suggest closely reading through the test and making sure there is
no place where an assumption about timing is being made. A frequent cause
of test flakiness is functions that rely on the order of operations of
asynchronous calls.
Re #3: I don't think the revert has been picked up by the bots yet.

I recommend using stress[1] to simulate load; it has frequently helped me reproduce test flakes.

1: http://linux.die.net/man/1/stress

Comment 6 by emaxx@chromium.org, Jun 17 2016

Status: Started (was: Assigned)
Project Member

Comment 7 by bugdroid1@chromium.org, Jun 20 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/93dc10b6eaaf3be6c4634a2e91e671da21338424

commit 93dc10b6eaaf3be6c4634a2e91e671da21338424
Author: emaxx <emaxx@chromium.org>
Date: Mon Jun 20 17:09:45 2016

Reland of Clear the login webview when SAML flow is canceled (original CL crrev.com/1988113004).

The original CL was later reverted (CL crrev.com/2068953003) due to
frequent failures of the SAMLPolicyTest.SAMLInterstitialNext test.

This follow-up CL should resolve the tests flakiness.

The problem with the original CL was that it could introduce raising the
"ready" event too early, so that the test code could start working when
the webview was still pointing to the blank page.

BUG= 613245 , 620353 
CQ_INCLUDE_TRYBOTS=tryserver.chromium.linux:closure_compilation

Review-Url: https://codereview.chromium.org/2082533002
Cr-Commit-Position: refs/heads/master@{#400703}

[modify] https://crrev.com/93dc10b6eaaf3be6c4634a2e91e671da21338424/chrome/browser/resources/chromeos/login/screen_gaia_signin.js
[modify] https://crrev.com/93dc10b6eaaf3be6c4634a2e91e671da21338424/chrome/browser/resources/gaia_auth_host/authenticator.js

Comment 8 by emaxx@chromium.org, Jun 21 2016

Status: Fixed (was: Started)
Labels: VerifyIn-53
Project Member

Comment 10 by bugdroid1@chromium.org, Jul 8 2016

Labels: merge-merged-2743
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4aa965d679259cc8545572fd4a34688671920711

commit 4aa965d679259cc8545572fd4a34688671920711
Author: Maksim Ivanov <emaxx@chromium.org>
Date: Fri Jul 08 01:14:01 2016

Reland of Clear the login webview when SAML flow is canceled (original CL crrev.com/1988113004).

The original CL was later reverted (CL crrev.com/2068953003) due to
frequent failures of the SAMLPolicyTest.SAMLInterstitialNext test.

This follow-up CL should resolve the tests flakiness.

The problem with the original CL was that it could introduce raising the
"ready" event too early, so that the test code could start working when
the webview was still pointing to the blank page.

BUG= 613245 , 620353 
CQ_INCLUDE_TRYBOTS=tryserver.chromium.linux:closure_compilation

Review-Url: https://codereview.chromium.org/2082533002
Cr-Commit-Position: refs/heads/master@{#400703}
(cherry picked from commit 93dc10b6eaaf3be6c4634a2e91e671da21338424)

Review URL: https://codereview.chromium.org/2134543003 .

Cr-Commit-Position: refs/branch-heads/2743@{#596}
Cr-Branched-From: 2b3ae3b8090361f8af5a611712fc1a5ab2de53cb-refs/heads/master@{#394939}

[modify] https://crrev.com/4aa965d679259cc8545572fd4a34688671920711/chrome/browser/resources/chromeos/login/screen_gaia_signin.js
[modify] https://crrev.com/4aa965d679259cc8545572fd4a34688671920711/chrome/browser/resources/gaia_auth_host/authenticator.js

Labels: VerifyIn-54
Labels: VerifyIn-55
Cc: monachow@chromium.org
Components: Enterprise

Comment 14 by dchan@google.com, Nov 19 2016

Labels: VerifyIn-56

Comment 15 by dchan@google.com, Jan 21 2017

Labels: VerifyIn-57

Comment 16 by dchan@google.com, Mar 4 2017

Labels: VerifyIn-58

Comment 17 by dchan@google.com, Apr 17 2017

Labels: VerifyIn-59

Comment 18 by dchan@google.com, May 30 2017

Labels: VerifyIn-60
Status: Verified (was: Fixed)

Sign in to add a comment