KioskTests are flaky on ChromiumOS bots. |
|||||||||||||||||
Issue descriptionBuild is broken: browser_tests on Ubuntu-12.04 Revision range: chromium 394285 : 394304 Failing builders: Linux ChromiumOS Tests (dbg)(1): https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20(dbg)(1) Sample log: [ RUN ] KioskUpdateTest.PRE_LaunchOfflineEnabledAppHasUpdate HTTPS server started on 127.0.0.1:34173... sending server_data: {"host": "127.0.0.1", "port": 34173} (36 bytes) [17416:17416:0517/205701:WARNING:disk_mount_manager.cc(732)] DiskMountManager was already initialized [17416:17416:0517/205702:WARNING:audio_manager.cc(297)] Multiple instances of AudioManager detected [17416:17416:0517/205702:WARNING:audio_manager.cc(271)] Multiple instances of AudioManager detected Xlib: extension "RANDR" missing on display ":9". [17416:17416:0517/205702:WARNING:child_account_service.cc(302)] User instance wasn't found while setting child account flag. [17416:17416:0517/205702:WARNING:message_queue.cc(38)] Leaking 2 ports in unreceived messages [17416:17416:0517/205702:INFO:power_button_observer.cc(37)] Creating PowerButtonObserver 0x1f659bf899e0 [17416:17416:0517/205702:INFO:lock_state_controller.cc(231)] LockStateController::OnLoginStateChanged login_status_: 0, status: 0 [17416:17416:0517/205702:INFO:kiosk_browsertest.cc(222)] kiosk locked = 1 [17416:17416:0517/205702:WARNING:auto_enrollment_controller.cc(127)] Auto-enrollment disabled: command line. [17416:17416:0517/205702:WARNING:auto_enrollment_controller.cc(259)] New auto-enrollment state: 5 [17416:17416:0517/205702:WARNING:auto_enrollment_controller.cc(127)] Auto-enrollment disabled: command line. [17416:17416:0517/205702:WARNING:auto_enrollment_controller.cc(259)] New auto-enrollment state: 5 [17416:17481:0517/205702:WARNING:freezer_cgroup_process_manager.cc(59)] Cgroup freezer does not exist or is not writable. Unable to freeze renderer processes. [17491:17491:0517/205703:ERROR:service_registry_impl.cc(95)] mojom::ResourceUsageReporter not found [17416:17416:0517/205704:WARNING:CONSOLE(1)] "This file is deprecated. Please use `iron-flex-layout/iron-flex-layout-classes.html`, and one of the specific dom-modules instead", source: chrome://resources/polymer/v1_0/iron-flex-layout/classes/iron-shadow-flex-layout-extracted.js (1) [17416:17416:0517/205705:WARNING:CONSOLE(1)] "This file is deprecated. Please use `iron-flex-layout/iron-flex-layout-classes.html`, and one of the specific dom-modules instead", source: chrome://resources/polymer/v1_0/iron-flex-layout/classes/iron-flex-layout-extracted.js (1) [17416:17416:0517/205709:WARNING:CONSOLE(0)] "/deep/ combinator is deprecated. See https://www.chromestatus.com/features/6750456638341120 for more details.", source: (0) [17416:17416:0517/205710:INFO:CONSOLE(7263)] "Loading asset bundle oauth-enrollment", source: chrome://oobe/oobe.js (7263) [17416:17416:0517/205710:INFO:signin_screen_handler.cc(321)] GetUserLRUInputMethod(''): no input method for this user [17416:17416:0517/205711:INFO:signin_screen_handler.cc(1232)] Login WebUI >> active: 1, source: gaia-signin [17416:17510:0517/205711:ERROR:fake_gaia.cc(315)] Unhandled request /update_check.xml [1:1:0517/205711:WARNING:message_queue.cc(38)] Leaking 1 ports in unreceived messages [17416:17416:0517/205711:INFO:CONSOLE(7247)] "Finished loading asset bundle oauth-enrollment", source: chrome://oobe/oobe.js (7247) [17416:17416:0517/205712:INFO:signin_screen_handler.cc(321)] GetUserLRUInputMethod(''): no input method for this user [17416:17416:0517/205712:INFO:signin_screen_handler.cc(1232)] Login WebUI >> active: 0, source: gaia-signin [17416:17416:0517/205712:INFO:signin_screen_handler.cc(1232)] Login WebUI >> active: 1, source: gaia-signin [17416:17510:0517/205712:WARNING:fake_gaia.cc(312)] Serving request /embedded/setup/chromeos localhost - - [17/May/2016 20:57:12] "GET /embedded/setup/chromeos?chrometype=chromedevice&client_id=dummytoken&client_version=52.0.2740.0&platform_version=0.0.0.0&endpoint_gen=1.0&hl=en-US HTTP/1.1" 200 - [17416:17416:0517/205712:ERROR:CONSOLE(80)] "Uncaught TypeError: Cannot read property 'getBoundingClientRect' of null", source: chrome://resources/js/cr/ui/position_util.js (80) [17416:17416:0517/205713:WARNING:gaia_screen_handler.cc(420)] Ignoring Gaia webview error: ERR_ABORTED [17416:17416:0517/205713:WARNING:CONSOLE(237)] "<webview>: The load has aborted with error -3: ERR_ABORTED.", source: extensions::webViewEvents (237) [17416:17510:0517/205713:WARNING:fake_gaia.cc(312)] Serving request /embedded/setup/chromeos localhost - - [17/May/2016 20:57:13] "GET /embedded/setup/chromeos?chrometype=chromedevice&client_id=dummytoken&client_version=52.0.2740.0&platform_version=0.0.0.0&endpoint_gen=1.0&hl=en-US HTTP/1.1" 200 - [17416:17510:0517/205714:ERROR:fake_gaia.cc(315)] Unhandled request /favicon.ico [17416:17510:0517/205714:WARNING:embedded_test_server.cc(190)] Request not handled. Returning 404: /favicon.ico localhost - - [17/May/2016 20:57:14] "GET /favicon.ico HTTP/1.1" 404 - [17416:17416:0517/205714:WARNING:CONSOLE(2274)] "Unrecognized message from GAIA: clearOldAttempts", source: chrome://oobe/gaia_auth_host.js (2274) [17416:17416:0517/205714:WARNING:CONSOLE(2274)] "Unrecognized message from GAIA: clearOldAttempts", source: chrome://oobe/gaia_auth_host.js (2274) [17416:17416:0517/205714:WARNING:CONSOLE(2274)] "Unrecognized message from GAIA: clearOldAttempts", source: chrome://oobe/gaia_auth_host.js (2274) [17416:17416:0517/205714:WARNING:CONSOLE(2274)] "Unrecognized message from GAIA: clearOldAttempts", source: chrome://oobe/gaia_auth_host.js (2274) BrowserTestBase signal handler received SIGTERM. Backtrace: #0 0x7fb01c43254e base::debug::StackTrace::StackTrace() #1 0x0000026ce3ca content::(anonymous namespace)::DumpStackTraceSignalHandler() #2 0x7fb004f68150 <unknown> #3 0x7fb0050199d3 __poll #4 0x7fb005aebff6 <unknown> #5 0x7fb005aec124 g_main_context_iteration #6 0x7fb01c3f7b9f base::MessagePumpGlib::Run() #7 0x7fb01c4a57fa base::MessageLoop::RunHandler() #8 0x7fb01c52be14 base::RunLoop::Run() #9 0x000005b96a16 content::RunThisRunLoop() #10 0x000005b97337 content::MessageLoopRunner::Run() #11 0x000005b9784a content::WindowedNotificationObserver::Wait() #12 0x00000154de13 chromeos::KioskTest::WaitForAppLaunchWithOptions() #13 0x00000154bc2e chromeos::KioskTest::WaitForAppLaunchSuccess() #14 0x00000154cef5 chromeos::KioskUpdateTest::PreCacheAndLaunchApp() #15 0x000001542957 chromeos::KioskUpdateTest_PRE_LaunchOfflineEnabledAppHasUpdate_Test::RunTestOnMainThread() #16 0x000002665918 InProcessBrowserTest::RunTestOnMainThreadLoop() #17 0x0000026ce1a7 content::BrowserTestBase::ProxyRunTestOnMainThreadLoop() #18 0x0000009ee24d _ZN4base8internal15RunnableAdapterIM24DevToolsBeforeUnloadTestFvvEE3RunIRK13scoped_refptrI53DevToolsBeforeUnloadTest_TestDockedDevToolsClose_TestEJEEEvOT_DpOT0_ #19 0x0000026cf229 _ZN4base8internal12InvokeHelperILb0EvNS0_15RunnableAdapterIMN7content15BrowserTestBaseEFvvEEEE8MakeItSoIJRK13scoped_refptrIS4_EEEEvS7_DpOT_ #20 0x0000026cf1f0 _ZN4base8internal7InvokerINS_13IndexSequenceIJLm0EEEENS0_9BindStateINS0_15RunnableAdapterIMN7content15BrowserTestBaseEFvvEEEFvPS7_EJSB_EEENS0_12InvokeHelperILb0EvSA_EEFvvEE3RunEPNS0_13BindStateBaseE #21 0x0000007fb56e base::Callback<>::Run() #22 0x0000062b7cf0 ChromeBrowserMainParts::PreMainMessageLoopRunImpl() #23 0x0000062b69c7 ChromeBrowserMainParts::PreMainMessageLoopRun() #24 0x000002963395 chromeos::ChromeBrowserMainPartsChromeos::PreMainMessageLoopRun() #25 0x7fb016432124 content::BrowserMainLoop::PreMainMessageLoopRun() #26 0x7fb0162f43b0 _ZN4base8internal15RunnableAdapterIMNS_18CancelableCallbackIFvvEEEKFvvEE3RunIPS4_JEEEvOT_DpOT0_ #27 0x7fb01643a699 _ZN4base8internal12InvokeHelperILb0EiNS0_15RunnableAdapterIMN7content15BrowserMainLoopEFivEEEE8MakeItSoIJPS4_EEEiS7_DpOT_ #28 0x7fb01643a65d _ZN4base8internal7InvokerINS_13IndexSequenceIJLm0EEEENS0_9BindStateINS0_15RunnableAdapterIMN7content15BrowserMainLoopEFivEEEFiPS7_EJNS0_17UnretainedWrapperIS7_EEEEENS0_12InvokeHelperILb0EiSA_EEFivEE3RunEPNS0_13BindStateBaseE #29 0x7fb0162a095e base::Callback<>::Run() #30 0x7fb016e0d4fb content::StartupTaskRunner::RunAllTasksNow() #31 0x7fb016430335 content::BrowserMainLoop::CreateStartupTasks() #32 0x7fb01643c810 content::BrowserMainRunnerImpl::Initialize() #33 0x7fb01642c5dd content::BrowserMain() #34 0x7fb01623a026 content::RunNamedProcessTypeMain() #35 0x7fb01623cdc5 content::ContentMainRunnerImpl::Run() #36 0x7fb016239172 content::ContentMain() #37 0x0000026cdf16 content::BrowserTestBase::SetUp() #38 0x000002664478 InProcessBrowserTest::SetUp() #39 0x0000013c366d ExtensionBrowserTest::SetUp() #40 0x0000015c8dc1 chromeos::OobeBaseTest::SetUp() #41 0x00000155040d chromeos::KioskTest::SetUp() #42 0x0000015514c6 chromeos::KioskUpdateTest::SetUp() #43 0x000000811efa _ZN7testing8internal12InvokeHelperIvNSt3tr15tupleIJEEEE12InvokeMethodIN10extensions33BluetoothPrivateApiTest_Pair_TestEMNS7_23BluetoothPrivateApiTestEFvvEEEvPT_T0_RKS4_ #44 0x0000028839fe testing::internal::HandleExceptionsInMethodIfSupported<>() #45 0x000002878c73 testing::Test::Run() #46 0x0000028793e8 testing::TestInfo::Run() #47 0x00000287998a testing::TestCase::Run() #48 0x00000287ecdc testing::internal::UnitTestImpl::RunAllTests() #49 0x0000028947ba testing::internal::HandleSehExceptionsInMethodIfSupported<>() #50 0x00000288502e testing::internal::HandleExceptionsInMethodIfSupported<>() #51 0x00000287e97f testing::UnitTest::Run() #52 0x000002674c41 RUN_ALL_TESTS() #53 0x000002673d28 base::TestSuite::Run() #54 0x0000025c80c2 ChromeTestSuiteRunner::RunTestSuite() #55 0x0000025c81a9 ChromeTestLauncherDelegate::RunTestSuite() #56 0x000005b8ccd5 content::LaunchTests() #57 0x0000025c84c9 LaunchChromeTests() #58 0x0000013ce5f5 main #59 0x7fb004f5376d __libc_start_main #60 0x000000765205 <unknown>
,
May 18 2016
Also flaky on the try bot: https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_chromeos_dbg_ng
,
May 18 2016
,
May 18 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/599274cdbfd7b12951429a747c2547d987cc27df commit 599274cdbfd7b12951429a747c2547d987cc27df Author: amistry <amistry@chromium.org> Date: Wed May 18 21:38:37 2016 Wait for OobeWebUI to finish loading before running tests. BUG= 612688 Review-Url: https://codereview.chromium.org/1987053003 Cr-Commit-Position: refs/heads/master@{#394557} [modify] https://crrev.com/599274cdbfd7b12951429a747c2547d987cc27df/chrome/browser/chromeos/login/test/oobe_base_test.cc
,
May 19 2016
Hm. This didn't fix what I thought it would fix. The JS error which I think the the root cause of this flake is still there. I'll keep digging: https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%28dbg%29%281%29/builds/14440 [18903:18903:0518/220141:ERROR:CONSOLE(80)] "Uncaught TypeError: Cannot read property 'getBoundingClientRect' of null", source: chrome://resources/js/cr/ui/position_util.js (80)
,
May 23 2016
We need to either fix or disable these, they are causing continuous failures on: https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%28dbg%29%281%29
,
May 23 2016
+Jenny and Xiyuan in case they have some ideas.
,
May 23 2016
I spent a bit of time debugging this, but now I'm stuck. What's happening is that an exception (see comment 5) is being thrown in the call path of launch an app for testing. Because of this, the app is never launched and the test times out. I managed to get a full stack trace of the JS exception:
[7518:7518:0520/163942:INFO:CONSOLE(83)] "TypeError: Cannot read property 'getBoundingClientRect' of null
at positionPopupAroundRect (chrome://resources/js/cr/ui/position_util.js:81:44)
at positionPopupAroundElement (chrome://resources/js/cr/ui/position_util.js:225:5)
at HTMLButtonElement.positionMenu_ (chrome://resources/js/cr/ui/menu_button.js:261:7)
at HTMLButtonElement.showMenu (chrome://resources/js/cr/ui/menu_button.js:209:12)
at HTMLButtonElement.showMenu (chrome://oobe/oobe.js:1334:39)
at HTMLButtonElement.findAndRunAppForTesting (chrome://oobe/oobe.js:1348:12)
at Function.AppsMenuButton.runAppForTesting (chrome://oobe/oobe.js:1447:27)
at <anonymous>:1:22", source: chrome://resources/js/cr/ui/position_util.js (83)
Code:
availRect = popupElement.offsetParent.getBoundingClientRect();
And this is where my abilities end. I don't know enough HTML/CSS/JS to understand why the element's offsetParent attribute is null.
,
May 24 2016
Jenny, are you familiar with this test?
,
May 24 2016
According to [1], offsetParent of an element is null when the element has display:none. Looks like there might be a race condition of when we try to launch the app and when the apps menu is made visible. [1] https://developer.mozilla.org/en-US/docs/Web/API/HTMLElement/offsetParent
,
May 24 2016
Recently these tests are more than just flaky, they have been failing reliably: https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%28dbg%29%281%29
,
May 24 2016
The failed tests are changing. So it is still some kind of flaky. I have a pending CL: https://codereview.chromium.org/2001403003/ that changes how the test app is launched. It should make the js error go away.
,
May 24 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8e37ebd0270bba886594552f9b79c10b50d3c0dc commit 8e37ebd0270bba886594552f9b79c10b50d3c0dc Author: xiyuan <xiyuan@chromium.org> Date: Tue May 24 21:48:02 2016 kiosk: Fix AppsMenu.findAndRunAppForTesting js error AppsMenu.findAndRunAppForTesting depends on whether "Apps" menu is visible or not and is flaky on bots. The method should launch the app regardless of whether the menu itself is visible or not. Change the impl to use its internal data to launch the app. BUG= 612688 CQ_INCLUDE_TRYBOTS=tryserver.chromium.linux:closure_compilation Review-Url: https://codereview.chromium.org/2001403003 Cr-Commit-Position: refs/heads/master@{#395706} [modify] https://crrev.com/8e37ebd0270bba886594552f9b79c10b50d3c0dc/chrome/browser/resources/chromeos/login/apps_menu.js
,
Jun 1 2016
Pri-0 bugs are critical regressions or serious emergencies, and this bug has not been updated in three days. Could you please provide an update, or adjust the priority to a more appropriate level if applicable? If a fix is in active development, please set the status to Started. Thanks for your time! To disable nags, add the Disable-Nags label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Jun 1 2016
Lower priority as the kiosk tests are much less flaky now. Keep it open because there are still some failures in the last 200 runs: KioskAppManagerTest.UpdateApp: 2 times https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%28dbg%29%281%29/builds/14776 https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%28dbg%29%281%29/builds/14639 #23 0x00000146f4a0 chromeos::(anonymous namespace)::AppDataLoadWaiter::Wait() #24 0x000001479a5d chromeos::KioskAppManagerTest::RunAddNewAppTest() KioskUpdateTest.PRE_PreserveLocalData: 1 time https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%28dbg%29%281%29/builds/14763 #18 0x000005bd38fa content::WindowedNotificationObserver::Wait() #19 0x000001568fea chromeos::KioskTest::PrepareAppLaunch() KioskAppManagerTest.GoodApp: 1 time https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%28dbg%29%281%29/builds/14657 #23 0x00000146f430 chromeos::(anonymous namespace)::AppDataLoadWaiter::Wait() #24 0x0000014715d1 chromeos::KioskAppManagerTest_GoodApp_Test::RunTestOnMainThread()
,
Jun 28 2016
These seem to be getting flakier. Just today, we've seen failures in: KioskTest.PRE_LaunchAppNetworkDown KioskTest.LaunchAppWithNetworkConfigAccelerator KioskTest.PRE_LaunchAppNetworkDown KioskTest.LaunchAppUserCancel KioskTest.LaunchAppUserCancelKioskTest.LaunchAppWithNetworkConfigAccelerator KioskTest.LaunchAppWithNetworkConfigAccelerator KioskTest.LaunchAppUserCancel https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%281%29?numbuilds=200
,
Jun 28 2016
https://codereview.chromium.org/2108843002/ might help
,
Jul 4 2016
The original report seems to have been an OOBE / js issue. I'm not sure if it's at all related to comment #16, which was filed as Issue 624201 . And I don't know if xiyuan@'s CL fixed this, but let's say "yes", since all Kiosk tests now seem stable.
,
Aug 29 2016
,
Oct 7 2016
,
Nov 19 2016
,
Jan 21 2017
,
Mar 4 2017
,
Apr 17 2017
,
May 30 2017
,
Aug 1 2017
,
Oct 14 2017
|
|||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||
Comment 1 by amistry@chromium.org
, May 18 2016