+Bartosz as he's the guru of ephemeral and because we were talking about this offline.

Note 1: This is version 60.0.3112.114, so it doesn't have the pre sign-in policy fetch code ( bug 747907  was merged back to M61 only),
Note 2: Ephemeral users is actually a device policy for ChromeOS (DeviceEphemeralUsersEnabled).

What seems to be happening here:
Preconditions: Ephemeral mode is off, user home is ecryptfs.
- User signs in. This is offline, because they have an existing user home on the machine.
  [909:909:0929/091637.041455:VERBOSE1:login_performer.cc(301)] Offline auth started.
- cryptohomed reports that migration is necessary (CRYPTOHOME_ERROR_MOUNT_OLD_ENCRYPTION)
  [909:909:0929/091637.915873:ERROR:..] .. Login: homedir_methods.cc:414 HomedirMethods MountEx error (CryptohomeErrorCode): 29
- We show the encryption migration screen:
  [909:909:0929/091637.919222:VERBOSE1:wizard_controller.cc(638)] Showing encryption migration screen.
- While the screen is displayed, a device policy update comes in
  [909:909:0929/091639.040065:INFO:remote_commands_invalidator.cc(94)] RemoteCommandsInvalidator received invalidation.
  [909:909:0929/091639.040650:INFO:remote_commands_invalidator.cc(110)] Invalidations acknowledged.
  Around this time we also have:
  2017-09-29T09:16:39.383037+02:00 INFO session_manager[877]: [INFO:policy_store.cc(71)] Persisted policy to disk.
  Suggesting that the new device policy was actually stored to disk.
  Presumably this turns on ephemeral mode in device policy.
- The user decides to skip migration, going back into offline auth (this time with force dircrypto=false)
  [909:909:0929/091640.893274:VERBOSE1:login_performer.cc(301)] Offline auth started.
- Now that ephemeral mode is on, cryptohomed deletes existing user homes as its first action [1].
  Also note that cryptohomed actually reloads device policy on each mount attempt so it should adhere to the new policy [2].
  The user home does not exist (anymore), so cryptohomed reports CRYPTOHOME_ERROR_ACCOUNT_NOT_FOUND:
  [909:909:0929/091643.064977:ERROR:..] .. Login: homedir_methods.cc:414 HomedirMethods MountEx error (CryptohomeErrorCode): 1
- This gets resolved to FAILED_MOUNT (2) in CryptohomeAuthenticator::ResolveCryptohomeFailureState.
  [909:909:0929/091643.065105:VERBOSE1:cryptohome_authenticator.cc(752)] Resolved state to: 2
  Note that if we did the auth online, this would be resolved to NO_MOUNT instead, allowing re-creation of the user home.

So, this is a race condition where we authenticated offline against an existing user home, which then got deleted because the ephemeral users device policy was turned on.
Maybe we already had this race condition (if it's possible for a policy update to get persisted after the user entered their password and before chromium calls into cryptohomed to perform the auth).
In any case, the window of opportunity for a policy change to get applied is larger when the device is idling on the encryption migration screen.

What can we do about it:
- Leave as is -- the user will get an error message, their user home will be wiped, and they can sign-in again.
- Write code to re-auth online when this state is detected (quite complicated to implement).
- Restart chrome when this state is detected (effectively forcing re-signin online, because when ephemeral users are on, chrome os login only shows the user-adding gaia view)

I would make this dependent on how the switch from ephemeral=off to ephemeral=on behaves when idling on the sign-in screen with pre-existing user accounts.
On my first test, it seemed to be broken (the user pods remained there, but I couldn't sign in anymore - "Your password could not be verified" or similar.
Code in ChromeUserManagerImpl::RetrieveTrustedDevicePolicies suggests that it should switch to the "no user pods, only adding users" mode automatically by deleting all regular users and calling NotifyUserListChanged[3], I'll test if my observation was a transient failure or if this is consistently misbehaving as well.

[1] At the beginning of Mount::MountCryptohomeInner, RemoveNonOwnerCryptohomes is called if ephemeral users are enabled by device policy. This should remove all existing user homes.
[2] HomeDirs::AreEphemeralUsersEnabled calls HomeDirs::LoadDevicePolicy which calls PolicyProvider::Reload.
[3] https://cs.chromium.org/chromium/src/chrome/browser/chromeos/login/users/chrome_user_manager_impl.cc?rcl=b999cd773e8246b2f9fda4d145051fb508a2a198&l=727

So, it turns out that the "turning ephemeral mode on while the device is on sign-in screen" area needs fixing.

We need to:
- make ChromeUserManagerImpl::RetrieveTrustedDevicePolicies get called when the policy changes (currently, we only seem to observe  kAccountsPrefDeviceLocalAccounts[1]. When I change both at the same time in testing, I see that user pods get removed)
- when all user pods get removed, make the sign-in screen show the gaia sign-in screen (instead, it currently simply shows no user pods and a "Add Person" button).
- ensure that we handle the race condition:
  - chromium offline auth started
  - auth failed with CRYPTOHOME_ERROR_ACCOUNT_NOT_FOUND
  - additional condition: ephemeral users mode is now turned on
  by e.g. restarting chrome.

I will repurpose this bug to track all of these.

[1] https://cs.chromium.org/chromium/src/chrome/browser/chromeos/login/users/chrome_user_manager_impl.cc?rcl=41ef48894974833d868817bf87c265e33ba20805&l=230