desktopui_MashLogin is failing consistently |
|||
Issue descriptiondesktopui_MashLogin is failing consistently Likely triggered by this CL enabling SingleProcessMash by default: https://chromium-review.googlesource.com/c/chromium/src/+/1393467 Here are the failure links I've been notified of so far: http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/273128538-chromeos-test/chromeos4-row9-rack10-host7/debug/ http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/273129246-chromeos-test/chromeos6-row2-rack11-host18/debug/ http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/273129366-chromeos-test/chromeos6-row4-rack17-host18/debug/ http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/273129969-chromeos-test/chromeos4-row2-rack3-host14/debug/ http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/273138800-chromeos-test/chromeos6-row2-rack23-host1/debug/ http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/273163470-chromeos-test/chromeos6-row2-rack11-host17/debug/ http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/273180423-chromeos-test/chromeos4-row9-rack10-host7/debug/ http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/273187647-chromeos-test/chromeos4-row2-rack3-host14/debug/ http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/273187925-chromeos-test/chromeos6-row4-rack12-host6/debug/ http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/273276028-chromeos-test/chromeos6-row2-rack11-host12/debug/ http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/273280882-chromeos-test/chromeos6-row4-rack23-host13/debug/ http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/273294659-chromeos-test/chromeos4-row2-rack3-host12/debug/ http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/273309858-chromeos-test/chromeos6-row4-rack12-host6/debug/ http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/273297096-chromeos-test/chromeos4-row9-rack10-host9/debug/ http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/273336498-chromeos-test/chromeos6-row2-rack11-host7/debug/ Here's the first email: ===================================================================================================================================== This report is automatically generated to track the following Failure: Test: desktopui_MashLogin. Suite: chrome-informational. Chrome Version: 73.0.3660.0. Build: veyron_minnie-tot-chrome-pfq-informational/R73-11513.0.0-b3303178. Reason: Unhandled BrowserConnectionGoneException: Timed out while waiting 240s for _GetDevToolsClient.. build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/veyron_minnie-tot-chrome-pfq-informational/R73-11513.0.0-b3303178. results log: http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/273128538-chromeos-test/chromeos4-row9-rack10-host7/debug/. status log: http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/273128538-chromeos-test/chromeos4-row9-rack10-host7/status.log. job link: http://cautotest-prod/afe/#tab_id=view_job&object_id=273128538. You may want to check the test history: https://stainless.corp.google.com/search?test=^desktopui\_MashLogin$&first_date=2018-12-06&last_date=2019-01-03&row=model&col=build&view=matrix You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=desktopui_MashLogin ===================================================================================================================================== Here's the top info and stack from dmp.txt: ===================================================================================================================================== Operating system: Linux 0.0.0 Linux 3.14.0 #1 SMP PREEMPT Wed Jan 2 15:31:43 PST 2019 armv7l CPU: arm ARMv1 ARM part(0x4100c0d0) features: swp,half,thumb,fastmult,vfpv2,edsp,thumbee,neon,vfpv3,tls,vfpv4,idiva,idivt 4 CPUs GPU: UNKNOWN Crash reason: SIGSEGV Crash address: 0x0 Process uptime: not available Thread 0 (crashed) 0 chrome!ChromeBrowserMainExtraPartsMetrics::PostBrowserStart() [chrome_browser_main_extra_parts_metrics.cc : 579 + 0x0] r0 = 0x00000000 r1 = 0x71fe7700 r2 = 0x00000008 r3 = 0x00000003 r4 = 0x080095c4 r5 = 0x00492489 r6 = 0x00000001 r7 = 0xbea685b0 r8 = 0xb1368ac0 r9 = 0x09b19340 r10 = 0x09ad4d20 r12 = 0x03d78399 fp = 0x07b63698 sp = 0xbea68578 lr = 0x039b7759 pc = 0x03a42bc8 Found by: given as instruction pointer in context 1 chrome!ChromeBrowserMainParts::PostBrowserStart() [chrome_browser_main.cc : 1255 + 0x25] r4 = 0x00000005 r5 = 0x00492489 r6 = 0x00000001 r7 = 0xbea68638 r8 = 0x00000000 r9 = 0xb1368ac0 r10 = 0x09ad4d20 fp = 0x07b63698 sp = 0xbea685b8 pc = 0x039b7759 Found by: call frame info 2 chrome!chromeos::ChromeBrowserMainPartsChromeos::PostBrowserStart() [chrome_browser_main_chromeos.cc : 1023 + 0x5] r4 = 0x09ad4d20 r5 = 0x0b0fc320 r6 = 0x07eb5260 r7 = 0xbea68690 r8 = 0xb1368ac0 r9 = 0x07a46020 r10 = 0x09ad4d20 fp = 0x07b63698 sp = 0xbea68640 pc = 0x02e31141 Found by: call frame info 3 chrome!ChromeBrowserMainParts::PreMainMessageLoopRun() [chrome_browser_main.cc : 1788 + 0x17] r4 = 0x09aefe00 r5 = 0x07a46028 r6 = 0x07aa4370 r7 = 0xbea68830 r8 = 0x0b196de0 r9 = 0x07a46020 r10 = 0x09ad4d20 fp = 0x07b63698 sp = 0xbea68698 pc = 0x039b6a07 Found by: call frame info 4 chrome!chromeos::ChromeBrowserMainPartsChromeos::PreMainMessageLoopRun() [chrome_browser_main_chromeos.cc : 660 + 0x5] r4 = 0x09b9c070 r5 = 0x09ccd7e0 r6 = 0x07eb4b48 r7 = 0xbea68880 r8 = 0x09ad4d20 r9 = 0xb1368ac0 r10 = 0x09c2a800 fp = 0x09ab4420 sp = 0xbea68838 pc = 0x02e2fead Found by: call frame info 5 chrome!content::BrowserMainLoop::PreMainMessageLoopRun() [browser_main_loop.cc : 984 + 0x13] r4 = 0x09a62b00 r5 = 0x00000000 r6 = 0xb1368ac0 r7 = 0xbea688e0 r8 = 0x09be7500 r9 = 0x00000000 r10 = 0x00000001 fp = 0xb1368ac0 sp = 0xbea68888 pc = 0x02569e3d Found by: call frame info 6 chrome!content::StartupTaskRunner::RunAllTasksNow() [callback.h : 129 + 0x1] r4 = 0x09c1c6e0 r5 = 0x00000000 r6 = 0x01bdfa7d r7 = 0xbea688f8 r8 = 0x09be7500 r9 = 0x00000000 r10 = 0x00000001 fp = 0xb1368ac0 sp = 0xbea688e8 pc = 0x028a6a07 Found by: call frame info 7 chrome!content::BrowserMainLoop::CreateStartupTasks() [browser_main_loop.cc : 918 + 0x3] r4 = 0x03ca86f9 r5 = 0x01bcbc91 r6 = 0x01bdfa7d r7 = 0xbea68980 r8 = 0x09a62b00 r9 = 0x00000000 r10 = 0x00000001 fp = 0xb1368ac0 sp = 0xbea68900 pc = 0x02567a1f Found by: call frame info 8 chrome!content::BrowserMainRunnerImpl::Initialize(content::MainFunctionParams const&) [browser_main_runner_impl.cc : 144 + 0x3] r4 = 0x800001cf r5 = 0x09b19370 r6 = 0x07a46028 r7 = 0xbea68a10 r8 = 0xb1368ac0 r9 = 0xbea68ab4 r10 = 0x00000000 fp = 0x00000000 sp = 0xbea68988 pc = 0x0256c8cb Found by: call frame info 9 chrome!content::BrowserMain(content::MainFunctionParams const&) [browser_main.cc : 43 + 0x3] r4 = 0x07eb6c00 r5 = 0x07eb6c00 r6 = 0x09b19360 r7 = 0xbea68a90 r8 = 0xbea68ab4 r9 = 0xb1368ac0 r10 = 0x07c3bed8 fp = 0x07eb6e00 sp = 0xbea68a18 pc = 0x02564f79 Found by: call frame info 10 chrome!content::ContentMainRunnerImpl::Run(bool) [content_main_runner_impl.cc : 545 + 0x3] r4 = 0xffffffff r5 = 0x00000000 r6 = 0x00000000 r7 = 0xbea68bd8 r8 = 0x00000000 r9 = 0xb1368ac0 r10 = 0x07e9c0f8 fp = 0xbea68b00 sp = 0xbea68a98 pc = 0x039aca1d Found by: call frame info 11 chrome!service_manager::Main(service_manager::MainParams const&) [main.cc : 461 + 0x15] r4 = 0x09a5e720 r5 = 0x07aa1738 r6 = 0xffffffff r7 = 0xbea68d98 r8 = 0x00000003 r9 = 0xbea68dac r10 = 0xb1368ac0 fp = 0x00000000 sp = 0xbea68be0 pc = 0x039b15af Found by: call frame info 12 chrome!content::ContentMain(content::ContentMainParams const&) [content_main.cc : 19 + 0x3] r4 = 0xbea68de8 r5 = 0xbea68df0 r6 = 0xb1368ac0 r7 = 0xbea68de0 r8 = 0xbea68dac r9 = 0x00000000 r10 = 0x07e39194 fp = 0x00000000 sp = 0xbea68da0 pc = 0x039ab2f9 Found by: call frame info 13 chrome!ChromeMain [chrome_main.cc : 102 + 0x3] r4 = 0xbea68fb4 r5 = 0x00000038 r6 = 0x09a5e720 r7 = 0xbea68e58 r8 = 0xb1368ac0 r9 = 0x00000000 r10 = 0x07e39194 fp = 0x00000000 sp = 0xbea68de8 pc = 0x01bca385 Found by: call frame info 14 libc-2.27.so!__libc_start_main [libc-start.c : 308 + 0xb] r4 = 0xbea68e78 r5 = 0x00000000 r6 = 0x00000000 r7 = 0x00000000 r8 = 0x00000000 r9 = 0x00000000 r10 = 0x07e39194 fp = 0x00000000 sp = 0xbea68e60 pc = 0xb07d70a3 Found by: call frame info 15 chrome!_start + 0x32 r4 = 0x00000000 r5 = 0x00000000 r6 = 0x01a47001 r7 = 0x00000000 r8 = 0x00000000 r9 = 0x00000000 r10 = 0x07e39194 fp = 0x00000000 sp = 0xbea68fa8 pc = 0x01a47035 Found by: call frame info 16 chrome!__libc_csu_init [elf-init.c : 87 + 0xf] sp = 0xbea68fac pc = 0x01bb690d Found by: stack scanning 17 0xbea69228 r3 = 0xb134bb2d r4 = 0xbea68fb4 r5 = 0xbea69181 r6 = 0xbea6919b r7 = 0xbea691dd r8 = 0xbea691fe r9 = 0xbea6921d sp = 0xbea68fcc pc = 0xbea6922a Found by: call frame info =====================================================================================================================================
,
Jan 3
I am not encountering this running locally with --enable-features=Mash --disable-features=SingleProcessMash (note: the second flag is currently necessary, but I assume the builder is setting that? Otherwise I don't think it would get that far). I suspect this is related to the builder's environment, or how it is running chrome?
,
Jan 3
Seems the test is not adding "--disable-features=SingleProcessMash". Have we turned SPM on by default already? https://cs.corp.google.com/chromeos_public/src/third_party/autotest/files/client/site_tests/desktopui_MashLogin/desktopui_MashLogin.py?rcl=208aaf3cf1422d0f9eec201154f19a53b55c7e38&l=55
,
Jan 3
Yes, James sent out a PSA. So that is definitely a problem then. I think we should have features::IsSingleProcessMash() return false if IsMultiProcessMash() is true. We will need to audit the code for direct access to the flag also.
,
Jan 3
I'll look into this.
,
Jan 3
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/77a4ad3625d7a800de237f7e334fea9a76170f41 commit 77a4ad3625d7a800de237f7e334fea9a76170f41 Author: James Cook <jamescook@chromium.org> Date: Thu Jan 03 18:17:25 2019 autotest: Disable SingleProcessMash in desktopui_MashLogin The SingleProcessMash and Mash flags are incompatible, so only pass the Mash one. BUG= chromium:918859 TEST=this is the test Change-Id: Ie6df8987e5b546f773cecbee29c6013f84b8d5fd Reviewed-on: https://chromium-review.googlesource.com/c/1394746 Tested-by: James Cook <jamescook@chromium.org> Reviewed-by: Steven Bennetts <stevenjb@chromium.org> [modify] https://crrev.com/77a4ad3625d7a800de237f7e334fea9a76170f41/client/site_tests/desktopui_MashLogin/desktopui_MashLogin.py
,
Jan 3
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/cddd568aa23d9153e2e5100e148be9bf8b6e137a commit cddd568aa23d9153e2e5100e148be9bf8b6e137a Author: James Cook <jamescook@chromium.org> Date: Thu Jan 03 22:27:08 2019 chromeos: Disable SingleProcessMash when multi-process Mash is enabled The two feature flags are incompatible, so make multi-process mash take precedence. Fix some direct references to features::kMash for consistency. TBR=sky@chromium.org Bug: 918859 Test: chrome --login-manager --enable-features=Mash logs in Change-Id: I18ac68e5e7bd01962369549b2b966592b9db0105 Reviewed-on: https://chromium-review.googlesource.com/c/1394796 Commit-Queue: James Cook <jamescook@chromium.org> Reviewed-by: Steven Bennetts <stevenjb@chromium.org> Cr-Commit-Position: refs/heads/master@{#619786} [modify] https://crrev.com/cddd568aa23d9153e2e5100e148be9bf8b6e137a/ash/ash_service.cc [modify] https://crrev.com/cddd568aa23d9153e2e5100e148be9bf8b6e137a/ash/host/ash_window_tree_host_platform.cc [modify] https://crrev.com/cddd568aa23d9153e2e5100e148be9bf8b6e137a/ash/system/flag_warning/flag_warning_tray_unittest.cc [modify] https://crrev.com/cddd568aa23d9153e2e5100e148be9bf8b6e137a/chrome/browser/ui/ash/chrome_browser_main_extra_parts_ash.cc [modify] https://crrev.com/cddd568aa23d9153e2e5100e148be9bf8b6e137a/ui/base/ui_base_features.cc [modify] https://crrev.com/cddd568aa23d9153e2e5100e148be9bf8b6e137a/ui/base/ui_base_features.h
,
Jan 3
I manually verified that after the CL in #7 I can manually log into a device with --enable-features=Mash set. I suspect the bot failures will clear soon.
,
Jan 7
I haven't seen any recent bot failures, marking Fixed. |
|||
►
Sign in to add a comment |
|||
Comment 1 by msw@chromium.org
, Jan 3