Call stack from the crash...

#0  0x0000592504258cb1 in policy::ServerBackedStateKeysBroker::RequestStateKeys(base::Callback<void (std::vector<std::string, std::allocator<std::string> > const&), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) ()
#1  0x0000592504156037 in chromeos::AutoEnrollmentController::OnOwnershipStatusCheckDone(chromeos::DeviceSettingsService::OwnershipStatus) ()
#2  0x0000592502c1180d in base::internal::Invoker<base::internal::BindState<base::Callback<void (int), (base::internal::CopyMode)1, (base::internal::RepeatMode)1>, int>, void ()>::Run(base::internal::BindStateBase*) ()
#3  0x0000592504ad1503 in base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*) ()
#4  0x0000592504a5548f in base::MessageLoop::RunTask(base::PendingTask*) ()
#5  0x0000592504a5593e in base::MessageLoop::DeferOrRunPendingTask(base::PendingTask) ()
#6  0x0000592504a57c40 in base::MessageLoop::DoWork() [clone .part.164] ()
#7  0x0000592504a581c9 in base::MessagePumpLibevent::Run(base::MessagePump::Delegate*) ()
#8  0x0000592504a7bbcd in base::RunLoop::Run() ()
#9  0x0000592504655b43 in ChromeBrowserMainParts::MainMessageLoopRun(int*) ()
#10 0x0000592502f5cf50 in content::BrowserMainLoop::RunMainMessageLoopParts() ()
#11 0x0000592502f615a5 in content::BrowserMainRunnerImpl::Run() ()
#12 0x0000592502f59459 in content::BrowserMain(content::MainFunctionParams const&) ()
#13 0x00005925045e4ec6 in content::ContentMainRunnerImpl::Run() ()
#14 0x0000592505bf0bba in service_manager::Main(service_manager::MainParams const&) ()
#15 0x00005925045e3a74 in content::ContentMain(content::ContentMainParams const&) ()
#16 0x00005925029a1cdb in ChromeMain ()
#17 0x00007f8ea83eb816 in ?? ()
#18 0x00007fff3b018bb0 in ?? ()
#19 0x00007fff3b018c58 in ?? ()
#20 0x0000001e3b018bb0 in ?? ()
#21 0x000059250299ecd0 in ?? ()
#22 0x0000000000000000 in ?? ()

Zen, do you have repro steps, or do you remember what you did in the run-up to this crash?

OK here's what I did as best I can remember - I wasn't expecting a failure so I wasn't keeping track of everything. 

I haven't repeated so still a little unclear what is relevant and what is not. Since until we are OK with wiping the device I can't really try it again!

But my setup I have ethernet via USB connected to google (ie plugged in wall). And I have my private network with the domain controller which I connect with WiFi.

I first boot up and press key combo to enrollment screen. (At some point here I did see a yellow warning icon saying something about not being able to connect to network). 

I unplugged and replugged by usb ethernet to force the ethernet to connect (not sure if this is relevant either).

I did the first enrollment step (littlecoderz account). That went with no errors to the domain join screen. (I don't remember if I unplugged the ethernet at this point).

The domain join worked fine. I got to the desktop (I think) and then switched to the terminal crtl-alt-right. When I attempted to switch back the device rebooted itself and then was stuck at the white chrome screen.

Roman and I rebooted a couple of times and he ssh'd in and enabled dumps.

From the white chrome screen I switch to terminal to poke around. When I tried to switch back to the UI (ctrl-alt-left) the device rebooted.

For more detail. I did manage to image again without hitting this. I think the difference is that I updated chrome (to my own build) before enrollment.

However after that - when I had to deploy a new version of chrome it went back to the bad state.

I'm iterating on my deployment steps each time I get stuck and hopefully can narrow down what at least the trigger is.

Clearly the customer is not updating Chrome (except maybe auto-update?)  so it doesn't seem to be the same cause even if it is the same symptom. 

OK - so this is a flow that I've done twice that gets me to a working state...

1) boot with usb image in and ethernet out
2) at welcome screen switch to terminal and chromeos-install
3) shutdown -h now
4) remove USB image
5) turn on device - ctrl-d for dev mode
6) it boots to the welcome screen
7) insert ethernet and wait until connected
8) deploy_chrome from my workstation
9) chrome restarts and goes back to welcome screen (no white splash screen between)
10) Click "Let's go" to start the flow
11) Click "next" (noting ethernet is connected)
12) Accept TOS
13) It goes to checking for updates (but doesn't give me the escape option??)
14) Lands on sign in screen
15) ctrl-alt-e
16) Enroll with my littlecoderz account
17) At domain join screen
18) Unplug ethernet
19) Join my private network with DC
20) Enter my domain account and passport
21) Get the green check mark and success message
22) Click finished
23) At domain login screen
23) Login with domain account (note still only connected to WiFi private network and ethernet disconnected)
24) Login succeeds
25) Select image
26) Successfully logged in

CL with workaround: https://codereview.chromium.org/2762293002/

So it seems to not be related to updating exactly but simply restarting chrome after we are enrolled.

From my working state I was trying to add a flag so I edited /etc/chrome_dev.conf then did sudo restart ui and got the same behavior. A hang coming back with ctrl-alt-left, then a reboot and stuck at splash screen.

I'm going to try a build with the workaround.

Thanks!  It repros here as well.

And now I can also confirm that the workaround is effective.

Can confirm that the workaround at least solves the issue when I restart chrome.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ec080c375b6006502f1b37145761fd3a1da29469

commit ec080c375b6006502f1b37145761fd3a1da29469
Author: tnagel <tnagel@chromium.org>
Date: Wed Mar 22 14:01:07 2017

Avoid segfault for missing StateKeysBroker

This should work around the problem until the root cause has been fixed.
(Also remove somewhat irritating curly braces from switch cases.)

BUG= 703658 

Review-Url: https://codereview.chromium.org/2762293002
Cr-Commit-Position: refs/heads/master@{#458739}

[modify] https://crrev.com/ec080c375b6006502f1b37145761fd3a1da29469/chrome/browser/chromeos/login/enrollment/auto_enrollment_controller.cc

Pri-0 bugs are critical regressions or serious emergencies, and this bug has not been updated in three days. Could you please provide an update, or adjust the priority to a more appropriate level if applicable?

If a fix is in active development, please set the status to Started.

Thanks for your time! To disable nags, add the Disable-Nags label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9157ba4857d139cbdc161668e2ff6f3bce9ffbcc

commit 9157ba4857d139cbdc161668e2ff6f3bce9ffbcc
Author: tnagel <tnagel@chromium.org>
Date: Tue Apr 11 16:56:15 2017

Add stub implementations for state keys and VPD

Stubbing out state keys and VPD (containing ActivateDate) is required
to test FRE on Linux.

BUG= 703658 

Review-Url: https://codereview.chromium.org/2774673004
Cr-Commit-Position: refs/heads/master@{#463656}

[modify] https://crrev.com/9157ba4857d139cbdc161668e2ff6f3bce9ffbcc/chromeos/chromeos_paths.cc
[modify] https://crrev.com/9157ba4857d139cbdc161668e2ff6f3bce9ffbcc/chromeos/chromeos_paths.h
[modify] https://crrev.com/9157ba4857d139cbdc161668e2ff6f3bce9ffbcc/chromeos/dbus/session_manager_client.cc
[modify] https://crrev.com/9157ba4857d139cbdc161668e2ff6f3bce9ffbcc/chromeos/dbus/session_manager_client.h
[modify] https://crrev.com/9157ba4857d139cbdc161668e2ff6f3bce9ffbcc/chromeos/system/statistics_provider.cc

Downgrading to Pri-1 since a workaround is already in place.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d0605bc9ca32f07f098363e0b1dc7d5a9bc075e8

commit d0605bc9ca32f07f098363e0b1dc7d5a9bc075e8
Author: tnagel <tnagel@chromium.org>
Date: Thu Apr 13 04:41:12 2017

Allow manual testing of FRE on Linux builds

Small refactoring of WizardController::OnNetworkConnected() to merge the
flows of official and unofficial builds as much as possible to properly
trigger FRE on a Linux build.  This is done by calling OnEulaAccepted()
on unofficial builds.

BUG= 703658 

Review-Url: https://codereview.chromium.org/2816763002
Cr-Commit-Position: refs/heads/master@{#464296}

[modify] https://crrev.com/d0605bc9ca32f07f098363e0b1dc7d5a9bc075e8/chrome/browser/chromeos/login/wizard_controller.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/723dbf022b3038d9241fbf7767b46b61c0085e39

commit 723dbf022b3038d9241fbf7767b46b61c0085e39
Author: tnagel <tnagel@chromium.org>
Date: Thu Apr 20 17:10:20 2017

Don't start FRE from the login screen

Starting FRE from the login screen goes back to the auto-enrollment
days and is not needed anymore (and causes crashes).  Removing that
code path has the benefit of allowing to move the
AutoEnrollmentController from LoginDisplayHost to WizardController.

Also:
- Fold AutoEnrollmentController::Cancel() into ...::Timeout() which had
  become its only caller.
- Add VLOG() to gain insight into the workings of AutoEnrollmentClient.

BUG= 703658 

Review-Url: https://codereview.chromium.org/2771803003
Cr-Commit-Position: refs/heads/master@{#466031}

[modify] https://crrev.com/723dbf022b3038d9241fbf7767b46b61c0085e39/chrome/browser/chromeos/arc/arc_session_manager_unittest.cc
[modify] https://crrev.com/723dbf022b3038d9241fbf7767b46b61c0085e39/chrome/browser/chromeos/login/enrollment/auto_enrollment_check_screen.cc
[modify] https://crrev.com/723dbf022b3038d9241fbf7767b46b61c0085e39/chrome/browser/chromeos/login/enrollment/auto_enrollment_controller.cc
[modify] https://crrev.com/723dbf022b3038d9241fbf7767b46b61c0085e39/chrome/browser/chromeos/login/enrollment/auto_enrollment_controller.h
[modify] https://crrev.com/723dbf022b3038d9241fbf7767b46b61c0085e39/chrome/browser/chromeos/login/existing_user_controller.cc
[modify] https://crrev.com/723dbf022b3038d9241fbf7767b46b61c0085e39/chrome/browser/chromeos/login/existing_user_controller_browsertest.cc
[modify] https://crrev.com/723dbf022b3038d9241fbf7767b46b61c0085e39/chrome/browser/chromeos/login/ui/login_display_host.h
[modify] https://crrev.com/723dbf022b3038d9241fbf7767b46b61c0085e39/chrome/browser/chromeos/login/ui/login_display_host_impl.cc
[modify] https://crrev.com/723dbf022b3038d9241fbf7767b46b61c0085e39/chrome/browser/chromeos/login/ui/login_display_host_impl.h
[modify] https://crrev.com/723dbf022b3038d9241fbf7767b46b61c0085e39/chrome/browser/chromeos/login/ui/mock_login_display_host.h
[modify] https://crrev.com/723dbf022b3038d9241fbf7767b46b61c0085e39/chrome/browser/chromeos/login/wizard_controller.cc
[modify] https://crrev.com/723dbf022b3038d9241fbf7767b46b61c0085e39/chrome/browser/chromeos/login/wizard_controller.h
[modify] https://crrev.com/723dbf022b3038d9241fbf7767b46b61c0085e39/chrome/browser/chromeos/login/wizard_controller_browsertest.cc
[modify] https://crrev.com/723dbf022b3038d9241fbf7767b46b61c0085e39/chrome/browser/chromeos/policy/auto_enrollment_client.cc
[modify] https://crrev.com/723dbf022b3038d9241fbf7767b46b61c0085e39/chrome/browser/chromeos/policy/auto_enrollment_client.h
[modify] https://crrev.com/723dbf022b3038d9241fbf7767b46b61c0085e39/chrome/browser/chromeos/policy/server_backed_state_keys_broker.h

bulk Verify of older or not-user-facing Chromad bugs