Several All/PasswordManagerBrowserTestWithConditionalPopupViews* tests are very flaky |
||
Issue descriptionSee https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=interactive_ui_tests&tests=All%2FPasswordManagerBrowserTestWithConditionalPopupViews Example of flaky tests: - All/PasswordManagerBrowserTestWithConditionalPopupViews.AutofillBaseTagWithNoActionTest/0 - All/PasswordManagerBrowserTestWithConditionalPopupViews.AutofillBaseTagWithNoActionTest/1 - All/PasswordManagerBrowserTestWithConditionalPopupViews.AutofillLoginSignupForm/0 - All/PasswordManagerBrowserTestWithConditionalPopupViews.AutofillLoginSignupForm/1 - All/PasswordManagerBrowserTestWithConditionalPopupViews.AutofillPasswordFormWithAutocompleteOff/0 - All/PasswordManagerBrowserTestWithConditionalPopupViews.AutofillPasswordFormWithAutocompleteOff/1 - All/PasswordManagerBrowserTestWithConditionalPopupViews.AutofillPasswordFormWithoutUsernameField/0 - All/PasswordManagerBrowserTestWithConditionalPopupViews.AutofillPasswordFormWithoutUsernameField/1 - All/PasswordManagerBrowserTestWithConditionalPopupViews.AutofillPasswordNoFormElement/0 - All/PasswordManagerBrowserTestWithConditionalPopupViews.AutofillPasswordNoFormElement/1 - All/PasswordManagerBrowserTestWithConditionalPopupViews.PromptForFetchWithoutOnSubmit/0 - All/PasswordManagerBrowserTestWithConditionalPopupViews.PromptForFetchWithoutOnSubmit/1 Tests that are a bit flaky - All/PasswordManagerBrowserTestWithConditionalPopupViews.PromptForFetchWithNewPasswordsWithoutOnSubmit/0 - All/PasswordManagerBrowserTestWithConditionalPopupViews.PromptForFetchWithNewPasswordsWithoutOnSubmit/1 - All/PasswordManagerBrowserTestWithConditionalPopupViews.PromptForXHRWithNewPasswordsWithoutOnSubmit/0 - All/PasswordManagerBrowserTestWithConditionalPopupViews.PromptForXHRWithNewPasswordsWithoutOnSubmit/1 - All/PasswordManagerBrowserTestWithConditionalPopupViews.PromptForXHRWithoutOnSubmit/0 - All/PasswordManagerBrowserTestWithConditionalPopupViews.PromptForXHRWithoutOnSubmit/1 Tests that are NOT flaky - All/PasswordManagerBrowserTestWithConditionalPopupViews.ManualFallbackForSaving/0 - All/PasswordManagerBrowserTestWithConditionalPopupViews.ManualFallbackForSaving/1 - All/PasswordManagerBrowserTestWithConditionalPopupViews.ManualFallbackForSaving_GoToManagedState/0 - All/PasswordManagerBrowserTestWithConditionalPopupViews.ManualFallbackForSaving_GoToManagedState/1 - All/PasswordManagerBrowserTestWithConditionalPopupViews.ManualFallbackForSaving_HideAfterTimeout/0 - All/PasswordManagerBrowserTestWithConditionalPopupViews.ManualFallbackForSaving_HideAfterTimeout/1 - All/PasswordManagerBrowserTestWithConditionalPopupViews.ManualFallbackForSaving_HideIcon/0 - All/PasswordManagerBrowserTestWithConditionalPopupViews.UsernameChanged/0 - All/PasswordManagerBrowserTestWithConditionalPopupViews.UsernameChanged/1 Sample output of one of the Autofill tests: [ RUN ] All/PasswordManagerBrowserTestWithConditionalPopupViews.AutofillBaseTagWithNoActionTest/1 [61634:4099:0604/002013.615439:WARNING:notification_platform_bridge_mac.mm(510)] AlertNotificationService: XPC connection invalidated. [61634:775:0604/002014.096427:INFO:CONSOLE(0)] "[DOM] Input elements should have autocomplete attributes (suggested: "current-password"): (More info: https://goo.gl/9p2vKq) %o", source: http://127.0.0.1:63908/password/password_xhr_submit.html (0) [61634:775:0604/002014.096526:INFO:CONSOLE(0)] "[DOM] Input elements should have autocomplete attributes (suggested: "username"): (More info: https://goo.gl/9p2vKq) %o", source: http://127.0.0.1:63908/password/password_xhr_submit.html (0) [61634:83203:0604/002014.137391:WARNING:embedded_test_server.cc(229)] Request not handled. Returning 404: /favicon.ico ../../chrome/browser/password_manager/password_manager_test_base.cc:640: Failure Expected equality of these values: expected_value Which is: "123" return_value Which is: "" element_id = password_field Stack trace: 0 interactive_ui_tests 0x00000001037187fb testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop(int) + 91 1 interactive_ui_tests 0x00000001037181b9 testing::internal::AssertHelper::operator=(testing::Message const&) const + 89 2 interactive_ui_tests 0x0000000109fd10e9 PasswordManagerBrowserTestBase::CheckElementValue(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 617 3 interactive_ui_tests 0x0000000109fd0e6b PasswordManagerBrowserTestBase::CheckElementValue(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 59 4 interactive_ui_tests 0x00000001035831b5 PasswordManagerInteractiveTestBase::VerifyPasswordIsSavedAndFilled(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 629 5 interactive_ui_tests 0x0000000103587a78 password_manager::PasswordManagerBrowserTestWithConditionalPopupViews_AutofillBaseTagWithNoActionTest_Test::RunTestOnMainThread() + 328 Assigning to vasilii for prep-work in https://chromium-review.googlesource.com/c/chromium/src/+/1071668
,
Jul 17
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c355e2e1229ccd4aeba81a5f6793b808a6dcaaa4 commit c355e2e1229ccd4aeba81a5f6793b808a6dcaaa4 Author: Alexey Khoroshilov <sense@yandex-team.ru> Date: Tue Jul 17 08:57:14 2018 Fix an element value check in PasswordManager interactive tests. Bug: 849582 Change-Id: I6e75d9a62b4b58569c2c29b913c0a6a0cab0bf83 Reviewed-on: https://chromium-review.googlesource.com/1139057 Reviewed-by: Vasilii Sukhanov <vasilii@chromium.org> Commit-Queue: Alexey Khoroshilov <sense@yandex-team.ru> Cr-Commit-Position: refs/heads/master@{#575584} [modify] https://crrev.com/c355e2e1229ccd4aeba81a5f6793b808a6dcaaa4/chrome/browser/password_manager/password_manager_interactive_test_base.cc [modify] https://crrev.com/c355e2e1229ccd4aeba81a5f6793b808a6dcaaa4/chrome/browser/password_manager/password_manager_interactive_uitest.cc
,
Jul 18
It looks like the tests are still flaky, example: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=interactive_ui_tests&tests=All/PasswordManagerBrowserTestWithConditionalPopupViews.AutofillPasswordFormWithoutUsernameField/0 http://build.chromium.org/p/chromium.memory/builders/Linux%20ChromiumOS%20MSan%20Tests/builds/7968
,
Jul 18
I started a revert: https://crrev.com/c/1142084
,
Jul 18
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/13417d256f140dbdcea9edc24480ac853eedbfec commit 13417d256f140dbdcea9edc24480ac853eedbfec Author: Christian Dullweber <dullweber@chromium.org> Date: Wed Jul 18 16:26:54 2018 Revert "Fix an element value check in PasswordManager interactive tests." This reverts commit c355e2e1229ccd4aeba81a5f6793b808a6dcaaa4. Reason for revert: The tests are still flaky http://build.chromium.org/p/chromium.memory/builders/Linux%20ChromiumOS%20MSan%20Tests/builds/7968 Original change's description: > Fix an element value check in PasswordManager interactive tests. > > Bug: 849582 > Change-Id: I6e75d9a62b4b58569c2c29b913c0a6a0cab0bf83 > Reviewed-on: https://chromium-review.googlesource.com/1139057 > Reviewed-by: Vasilii Sukhanov <vasilii@chromium.org> > Commit-Queue: Alexey Khoroshilov <sense@yandex-team.ru> > Cr-Commit-Position: refs/heads/master@{#575584} TBR=vasilii@chromium.org,dvadym@chromium.org,sense@yandex-team.ru # Not skipping CQ checks because original CL landed > 1 day ago. Bug: 849582 Change-Id: I4d363b0dfefa99256d50cae5fd51102204f7610a Reviewed-on: https://chromium-review.googlesource.com/1142084 Commit-Queue: Christian Dullweber <dullweber@chromium.org> Reviewed-by: Christian Dullweber <dullweber@chromium.org> Cr-Commit-Position: refs/heads/master@{#576085} [modify] https://crrev.com/13417d256f140dbdcea9edc24480ac853eedbfec/chrome/browser/password_manager/password_manager_interactive_test_base.cc [modify] https://crrev.com/13417d256f140dbdcea9edc24480ac853eedbfec/chrome/browser/password_manager/password_manager_interactive_uitest.cc
,
Jul 26
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b9a893cd50f940feedb3ebe8bbd29f0da95e1653 commit b9a893cd50f940feedb3ebe8bbd29f0da95e1653 Author: Alexey Khoroshilov <sense@yandex-team.ru> Date: Thu Jul 26 04:46:02 2018 Improve an HTML input fill logic to fix test flakiness. Bug: 849582 Change-Id: Idb681d282fd6754033779fb36c879cb72b0abf88 Reviewed-on: https://chromium-review.googlesource.com/1142765 Commit-Queue: Alexey Khoroshilov <sense@yandex-team.ru> Reviewed-by: Vasilii Sukhanov <vasilii@chromium.org> Cr-Commit-Position: refs/heads/master@{#578206} [modify] https://crrev.com/b9a893cd50f940feedb3ebe8bbd29f0da95e1653/chrome/browser/password_manager/password_manager_interactive_test_base.cc [modify] https://crrev.com/b9a893cd50f940feedb3ebe8bbd29f0da95e1653/chrome/browser/password_manager/password_manager_interactive_test_base.h [modify] https://crrev.com/b9a893cd50f940feedb3ebe8bbd29f0da95e1653/chrome/browser/password_manager/password_manager_interactive_uitest.cc
,
Jul 26
result on Mac [ RUN ] All/PasswordManagerBrowserTestWithConditionalPopupViews.PromptForFetchWithoutOnSubmit/0 [61917:4099:0725/231614.156693:WARNING:notification_platform_bridge_mac.mm(510)] AlertNotificationService: XPC connection invalidated. [61917:775:0725/231614.728982:INFO:CONSOLE(0)] "[DOM] Input elements should have autocomplete attributes (suggested: "current-password"): (More info: https://goo.gl/9p2vKq) %o", source: http://127.0.0.1:56629/password/password_fetch_submit.html (0) [61917:775:0725/231614.729383:INFO:CONSOLE(0)] "[DOM] Input elements should have autocomplete attributes (suggested: "username"): (More info: https://goo.gl/9p2vKq) %o", source: http://127.0.0.1:56629/password/password_fetch_submit.html (0) [61917:90115:0725/231614.770293:WARNING:embedded_test_server.cc(238)] Request not handled. Returning 404: /favicon.ico BrowserTestBase received signal: Terminated: 15. Backtrace: 0 interactive_ui_tests 0x000000010dbdf8fc base::debug::StackTrace::StackTrace(unsigned long) + 28 1 interactive_ui_tests 0x000000010d5e8125 content::(anonymous namespace)::DumpStackTraceSignalHandler(int) + 229 2 libsystem_platform.dylib 0x00007fff726f7f5a _sigtramp + 26 3 ??? 0x00007ffee4db8bb8 0x0 + 140732738014136 4 CoreFoundation 0x00007fff4a533045 __CFRunLoopServiceMachPort + 341 5 CoreFoundation 0x00007fff4a532397 __CFRunLoopRun + 1783 6 CoreFoundation 0x00007fff4a531a07 CFRunLoopRunSpecific + 487 7 HIToolbox 0x00007fff4980dd96 RunCurrentEventLoopInMode + 286 8 HIToolbox 0x00007fff4980db06 ReceiveNextEventCommon + 613 9 HIToolbox 0x00007fff4980d884 _BlockUntilNextEventMatchingListInModeWithFilter + 64 10 AppKit 0x00007fff47abfa73 _DPSNextEvent + 2085 11 AppKit 0x00007fff48255e34 -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 3044 12 interactive_ui_tests 0x000000010d0a2bc0 __71-[BrowserCrApplication nextEventMatchingMask:untilDate:inMode:dequeue:]_block_invoke + 64 13 interactive_ui_tests 0x000000010db4127a base::mac::CallWithEHFrame(void () block_pointer) + 10 14 interactive_ui_tests 0x000000010d0a2af4 -[BrowserCrApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 164 15 AppKit 0x00007fff47ab4885 -[NSApplication run] + 764 16 interactive_ui_tests 0x000000010db51b6c base::MessagePumpNSApplication::DoRun(base::MessagePump::Delegate*) + 364 17 interactive_ui_tests 0x000000010db506ae base::MessagePumpCFRunLoopBase::Run(base::MessagePump::Delegate*) + 110 18 interactive_ui_tests 0x000000010db73dc5 base::RunLoop::Run() + 53 19 interactive_ui_tests 0x000000010d5ea211 content::DOMMessageQueue::WaitForMessage(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) + 97 20 interactive_ui_tests 0x000000010d5e99e1 content::(anonymous namespace)::ExecuteScriptHelper(content::RenderFrameHost*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool, std::__1::unique_ptr<base::Value, std::__1::default_delete<base::Value> >*) + 369 21 interactive_ui_tests 0x000000010d5e9c46 content::ExecuteScriptWithoutUserGestureAndExtractInt(content::ToRenderFrameHost const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int*) + 38 22 interactive_ui_tests 0x0000000111c28b7a PasswordManagerBrowserTestBase::WaitForElementValue(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 362 23 interactive_ui_tests 0x0000000111c289fb PasswordManagerBrowserTestBase::WaitForElementValue(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 59 24 interactive_ui_tests 0x000000010af1ead7 PasswordManagerInteractiveTestBase::FillElementWithValue(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, base::Optional<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > const&) + 1927 25 interactive_ui_tests 0x000000010af22335 password_manager::PasswordManagerBrowserTestWithConditionalPopupViews_PromptForFetchWithoutOnSubmit_Test::RunTestOnMainThread() + 261 26 interactive_ui_tests 0x000000010d5e79eb content::BrowserTestBase::ProxyRunTestOnMainThreadLoop() + 491 27 interactive_ui_tests 0x000000010d0a8b07 ChromeBrowserMainParts::PreMainMessageLoopRunImpl() + 4375 28 interactive_ui_tests 0x000000010d0a78ee ChromeBrowserMainParts::PreMainMessageLoopRun() + 62 29 interactive_ui_tests 0x000000010bc2e163 content::BrowserMainLoop::PreMainMessageLoopRun() + 67 30 interactive_ui_tests 0x000000010c01ec27 content::StartupTaskRunner::RunAllTasksNow() + 39 31 interactive_ui_tests 0x000000010bc2c908 content::BrowserMainLoop::CreateStartupTasks() + 712 32 interactive_ui_tests 0x000000010bc304c0 content::BrowserMainRunnerImpl::Initialize(content::MainFunctionParams const&) + 96 33 interactive_ui_tests 0x000000010bc2aa44 content::BrowserMain(content::MainFunctionParams const&) + 180 34 interactive_ui_tests 0x000000010d026268 content::ContentMainRunnerImpl::Run(bool) + 824 35 interactive_ui_tests 0x000000010ec94301 service_manager::Main(service_manager::MainParams const&) + 2849 36 interactive_ui_tests 0x000000010d0252b4 content::ContentMain(content::ContentMainParams const&) + 68 37 interactive_ui_tests 0x000000010d5e7619 content::BrowserTestBase::SetUp() + 2697 38 interactive_ui_tests 0x000000010d052bf3 InProcessBrowserTest::SetUp() + 723 39 interactive_ui_tests 0x000000010af23cb1 password_manager::PasswordManagerBrowserTestWithConditionalPopupViews::SetUp() + 113 40 interactive_ui_tests 0x000000010b0c2f9d testing::Test::Run() + 109 41 interactive_ui_tests 0x000000010b0c3c30 testing::TestInfo::Run() + 320 42 interactive_ui_tests 0x000000010b0c41a7 testing::TestCase::Run() + 279 43 interactive_ui_tests 0x000000010b0cf7d7 testing::internal::UnitTestImpl::RunAllTests() + 871 44 interactive_ui_tests 0x000000010b0cf44d testing::UnitTest::Run() + 109 45 interactive_ui_tests 0x000000010db1a366 base::TestSuite::Run() + 166 46 interactive_ui_tests 0x000000010afdc173 InteractiveUITestSuiteRunner::RunTestSuite(int, char**) + 51 47 interactive_ui_tests 0x000000010d5ee968 content::LaunchTests(content::TestLauncherDelegate*, unsigned long, int, char**) + 552 48 interactive_ui_tests 0x000000010d04b13d LaunchChromeTests(unsigned long, content::TestLauncherDelegate*, int, char**) + 333 49 interactive_ui_tests 0x000000010afdc0dc main + 92 50 libdyld.dylib 0x00007fff723e9015 start + 1
,
Jul 26
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/89b69b4f8865bead99045fc90d70db07521b8f18 commit 89b69b4f8865bead99045fc90d70db07521b8f18 Author: Alexey Khoroshilov <sense@yandex-team.ru> Date: Thu Jul 26 11:22:56 2018 Revert "Improve an HTML input fill logic to fix test flakiness." This reverts commit b9a893cd50f940feedb3ebe8bbd29f0da95e1653. Reason for revert: Flaky on MacOS. Original change's description: > Improve an HTML input fill logic to fix test flakiness. > > Bug: 849582 > Change-Id: Idb681d282fd6754033779fb36c879cb72b0abf88 > Reviewed-on: https://chromium-review.googlesource.com/1142765 > Commit-Queue: Alexey Khoroshilov <sense@yandex-team.ru> > Reviewed-by: Vasilii Sukhanov <vasilii@chromium.org> > Cr-Commit-Position: refs/heads/master@{#578206} TBR=sky@chromium.org,vasilii@chromium.org,sense@yandex-team.ru Change-Id: I8d1e71e16dbae6d6293a6f6c2ed63102de8713aa No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: 849582 Reviewed-on: https://chromium-review.googlesource.com/1151150 Commit-Queue: Vasilii Sukhanov <vasilii@chromium.org> Reviewed-by: Vasilii Sukhanov <vasilii@chromium.org> Cr-Commit-Position: refs/heads/master@{#578261} [modify] https://crrev.com/89b69b4f8865bead99045fc90d70db07521b8f18/chrome/browser/password_manager/password_manager_interactive_test_base.cc [modify] https://crrev.com/89b69b4f8865bead99045fc90d70db07521b8f18/chrome/browser/password_manager/password_manager_interactive_test_base.h [modify] https://crrev.com/89b69b4f8865bead99045fc90d70db07521b8f18/chrome/browser/password_manager/password_manager_interactive_uitest.cc
,
Sep 25
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/26fb34f4fb8dab02e0e8c951f1d181e29a702c9e commit 26fb34f4fb8dab02e0e8c951f1d181e29a702c9e Author: Colin Blundell <blundell@chromium.org> Date: Tue Sep 25 09:51:15 2018 [Password autofill] Disable flaky test See https://crbug.com/887745 for FindIt's (and my) analysis of the flake. TBR=vasilii@chromium.org Bug: 887745 , 849582 Change-Id: I8c6e520679d7462eb759526d14a375ed05f5503f Reviewed-on: https://chromium-review.googlesource.com/1242945 Reviewed-by: Colin Blundell <blundell@chromium.org> Commit-Queue: Colin Blundell <blundell@chromium.org> Cr-Commit-Position: refs/heads/master@{#593881} [modify] https://crrev.com/26fb34f4fb8dab02e0e8c951f1d181e29a702c9e/chrome/browser/password_manager/password_manager_interactive_uitest.cc
,
Oct 8
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/0e92f93c558ba3ffe013e9effd38e6814a51528c commit 0e92f93c558ba3ffe013e9effd38e6814a51528c Author: Giovanni Ortuño Urquidi <ortuno@chromium.org> Date: Mon Oct 08 03:09:08 2018 sheriff: Disable flaky password manager test TBR=vasilii@chromium.org Bug: 893008, 849582 Change-Id: Id88b5b61e0b25528ac94951ee5aa3ac13b021eff Reviewed-on: https://chromium-review.googlesource.com/c/1267756 Reviewed-by: Giovanni Ortuño Urquidi <ortuno@chromium.org> Commit-Queue: Giovanni Ortuño Urquidi <ortuno@chromium.org> Cr-Commit-Position: refs/heads/master@{#597475} [modify] https://crrev.com/0e92f93c558ba3ffe013e9effd38e6814a51528c/chrome/browser/password_manager/password_manager_interactive_uitest.cc
,
Oct 8
|
||
►
Sign in to add a comment |
||
Comment 1 by bugdroid1@chromium.org
, Jun 5 2018