Frequent intermittent failures in SAMLPolicyTest |
||||||||||||||
Issue descriptionSince the introduction of this CL: Clear the login webview when SAML flow is canceled https://codereview.chromium.org/1988113004 We have seen frequent but intermittent failures (24/28 failed) in SAMLPolicyTest: https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%28dbg%29%281%29?numbuilds=100 We need to either revert the change or fix the test quickly.
,
Jun 15 2016
(deleted the previous comment due to typos) Hmm, isn't the builder URL still reporting failures despite reverting of my CL? https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%28dbg%29%281%29?numbuilds=100 I tried running this SAMLPolicyTest.SAMLInterstitialNext test locally, and it works for me regardless of my CL when the network connection is present, and fails for me regardless of my CL when the network connection is missing (tested with "unshare" tool). Not sure, however, whether my environment is following the builder environment completely.
,
Jun 15 2016
It may be a timing issue. The test lab builders tend to be heavily loaded. I would suggest closely reading through the test and making sure there is no place where an assumption about timing is being made. A frequent cause of test flakiness is functions that rely on the order of operations of asynchronous calls.
,
Jun 15 2016
Re #3: I don't think the revert has been picked up by the bots yet. I recommend using stress[1] to simulate load; it has frequently helped me reproduce test flakes. 1: http://linux.die.net/man/1/stress
,
Jun 17 2016
,
Jun 20 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/93dc10b6eaaf3be6c4634a2e91e671da21338424 commit 93dc10b6eaaf3be6c4634a2e91e671da21338424 Author: emaxx <emaxx@chromium.org> Date: Mon Jun 20 17:09:45 2016 Reland of Clear the login webview when SAML flow is canceled (original CL crrev.com/1988113004). The original CL was later reverted (CL crrev.com/2068953003) due to frequent failures of the SAMLPolicyTest.SAMLInterstitialNext test. This follow-up CL should resolve the tests flakiness. The problem with the original CL was that it could introduce raising the "ready" event too early, so that the test code could start working when the webview was still pointing to the blank page. BUG= 613245 , 620353 CQ_INCLUDE_TRYBOTS=tryserver.chromium.linux:closure_compilation Review-Url: https://codereview.chromium.org/2082533002 Cr-Commit-Position: refs/heads/master@{#400703} [modify] https://crrev.com/93dc10b6eaaf3be6c4634a2e91e671da21338424/chrome/browser/resources/chromeos/login/screen_gaia_signin.js [modify] https://crrev.com/93dc10b6eaaf3be6c4634a2e91e671da21338424/chrome/browser/resources/gaia_auth_host/authenticator.js
,
Jun 21 2016
,
Jul 1 2016
,
Jul 8 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4aa965d679259cc8545572fd4a34688671920711 commit 4aa965d679259cc8545572fd4a34688671920711 Author: Maksim Ivanov <emaxx@chromium.org> Date: Fri Jul 08 01:14:01 2016 Reland of Clear the login webview when SAML flow is canceled (original CL crrev.com/1988113004). The original CL was later reverted (CL crrev.com/2068953003) due to frequent failures of the SAMLPolicyTest.SAMLInterstitialNext test. This follow-up CL should resolve the tests flakiness. The problem with the original CL was that it could introduce raising the "ready" event too early, so that the test code could start working when the webview was still pointing to the blank page. BUG= 613245 , 620353 CQ_INCLUDE_TRYBOTS=tryserver.chromium.linux:closure_compilation Review-Url: https://codereview.chromium.org/2082533002 Cr-Commit-Position: refs/heads/master@{#400703} (cherry picked from commit 93dc10b6eaaf3be6c4634a2e91e671da21338424) Review URL: https://codereview.chromium.org/2134543003 . Cr-Commit-Position: refs/branch-heads/2743@{#596} Cr-Branched-From: 2b3ae3b8090361f8af5a611712fc1a5ab2de53cb-refs/heads/master@{#394939} [modify] https://crrev.com/4aa965d679259cc8545572fd4a34688671920711/chrome/browser/resources/chromeos/login/screen_gaia_signin.js [modify] https://crrev.com/4aa965d679259cc8545572fd4a34688671920711/chrome/browser/resources/gaia_auth_host/authenticator.js
,
Aug 29 2016
,
Oct 7 2016
,
Nov 16 2016
,
Nov 19 2016
,
Jan 21 2017
,
Mar 4 2017
,
Apr 17 2017
,
May 30 2017
,
Jul 6 2017
|
||||||||||||||
►
Sign in to add a comment |
||||||||||||||
Comment 1 by jdufault@chromium.org
, Jun 15 2016Labels: -Pri-0 Pri-1
Owner: emaxx@chromium.org