New issue
Advanced search Search tips

Issue 753397 link

Starred by 3 users

Issue metadata

Status: Verified
Owner:
Closed: Oct 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug-Regression



Sign in to add a comment

New device locked, preventing enterprise enrollment.

Reported by test.t...@sausd.us, Aug 8 2017

Issue description

UserAgent: Mozilla/5.0 (X11; CrOS x86_64 9460.73.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.134 Safari/537.36
Platform: 9460.73.1 (Official Build) stable-channel relm

Steps to reproduce the problem:
1. Update device to current version
2. enroll device
3. fail

What is the expected behavior?
enrollment

What went wrong?
Brand new devices failing enrollment with the error "This device is locked in a mode that prevent enterprise enrollment. If you want to enroll the device you need to go through device recovery first."

After power washing the devices they return to enterprise enrollment mode with an indication that they are managed by the domain i want, but the login credentials must be reentered.  They may even fail a second time and require another power wash before successfully enrolling.  I'm trying to enroll 100+ machines at a time, and having to powerwash half of them each round is a real annoyance.   

Did this work before? Yes 

Chrome version: 59.0.3071.134  Channel: stable
OS Version: 9460.73.1
Flash Version: 26.0.0.102
 
Labels: Enterprise-Triaged
Owner: igorcov@chromium.org
Status: Assigned (was: Unconfirmed)
Over to Igor for triage/investigation.
Thank you for reporting this. Could you please clarify the exact flow you go through? If I understood correctly:

1. New device, never used before, go through enrollment and fail with the error above.
2. Pwerwash the device.
3. After boot the device asks to be re-enrolled to your domain.

Basically this would mean the device succeeded to enroll the first time on server side, but failed to complete on the device.

Also, could you please attach the logs from a device that went through troubling flow. Those can be taken after sign in, going in browser to chrome://net-internals/#chromeos -> Store Debug Logs.

Note: The code that throws that error: https://cs.chromium.org/chromium/src/chrome/browser/chromeos/settings/install_attributes.cc?type=cs&sq=package:chromium&l=227

Sounds like might be ssd corruption problem.

Comment 3 by test.t...@sausd.us, Aug 10 2017

We are updating all the machines before we enroll them, but besides that your description of the problem is correct.  Attached requested log. Thanks for looking into this.
debug-logs_20170810-082504.tgz
232 KB Download
net-internals-log.json
4.3 MB View Download
Labels: -Pri-2 Pri-1
Thank you for logs. No signs of data corruption, and according to the logs the
device was not into forced re-enrollment mode yet. Were the logs taken before
recovery?

There was an internet disconnected error during enrollment, and I tried to
reproduce this, but no luck. This needs further investigation, and a first step
will be to add more logs in the enrollment flow.

Comment 5 by test.t...@sausd.us, Aug 11 2017

I misunderstood. Adding logs taken after recovery. 



 
debug-logs_20170811-061459.tgz
135 KB Download
Status: Started (was: Assigned)
The relevant log for the error:
[906:906:0810/082408.492659:ERROR:install_attributes.cc(226)] Trying to re-lock with wrong mode.

It means:
device_locked_ && (device_mode != registration_mode_)

device_locked_ is set in Init after the system discovers that install attributes file is not present. This is expected.

registration_mode_ should have the value policy::DEVICE_MODE_ENTERPRISE
device_mode is one of:
DEVICE_MODE_ENTERPRISE, DEVICE_MODE_LEGACY_RETAIL_MODE or DEVICE_MODE_ENTERPRISE_AD.

Probably makes sense to put logs when the device becomes locked, when device_mode is set and when registration_mode is received from server.
One possibility is that device_locked_ becomes true here:
https://cs.chromium.org/chromium/src/chrome/browser/chromeos/settings/install_attributes.cc?l=167

Which means the flow went like this:
EnrollmentHandlerChromeOS::StartLockDevice()
InstallAttributes::LockDevice
cryptohome_client_->InstallAttributesIsReady
InstallAttributes::LockDeviceIfAttributesIsReady
InstallAttributes::ReadImmutableAttributes
cryptohome_client_->InstallAttributesIsReady
InstallAttributes::ReadAttributesIfReady

Here:
device_locked_ = true;
DecodeInstallAttributes(attr_map);

InstallAttributes::OnReadImmutableAttributes
EnrollmentHandlerChromeOS::HandleLockDeviceResult

But install_attributes.cc(226) is InstallAttributes::LockDevice

So the function LockDevice has been called the second time?
I've managed to reproduce it by having install_attributes file on a clean device. In that case the device_locked_ becomes true in the Init function because the file exists, while on the enrollment it discovers that according to file contents the device is consumer owned.

So, device_mode == DEVICE_MODE_ENTERPRISE while registration_mode_ == DEVICE_MODE_CONSUMER.

I suspect this happens on the device too.

According to logs the device was used for guest mode before, but I'm not sure yet how it could end up with the install_attributes file on it. Also need to investigate why the enrollment is allowed to start before powerwash - probably some other check failing.
The install attributes have been indeed created on the device on 2017.08.10 at 07:29:50

2017-08-10T07:29:50.814166-07:00 INFO cryptohomed[1223]: InstallAttributes have been finalized.

The enrollment attempt has been done in the same day at 08:18
[906:906:0810/081821.991571:VERBOSE1:wizard_controller.cc(1522)] Showing enrollment screen. Forcing interactive enrollment: 0.

That explains why enrollment failed.

Seems like the attributes have been saved as a result of starting user session:
2017-08-10T07:29:38.935699-07:00 INFO session_manager[954]: [INFO:session_manager_impl.cc(388)] Starting user session

Still not clear why the enrollment was allowed and why the attributes have been saved when, as it appears, only guest sign in happened.

test.test@sausd.us did anybody sign in on the device with some user account at 07:29? Or was it used only with guest mode browsing?
Seems like the install attributes is written because the flow hits this line:
https://cs.corp.google.com/chromeos_public/src/platform2/cryptohome/service.cc?type=cs&q=AttestationInitializeTpmComplete+file:%5Esrc/platform2/+package:%5Echromeos_public$&l=837

From the code:
  // Do attestation work after AddEvent because it may take long.
  AttestationInitializeTpmComplete();

  // If we mounted before the TPM finished initialization, we must
  // finalize the install attributes now too, otherwise it takes a
  // full re-login cycle to finalize.
  gboolean mounted = FALSE;
  bool is_mounted = (IsMounted(&mounted, NULL) && mounted);
  if (is_mounted && took_ownership && install_attrs_->is_first_install()) {
    install_attrs_->Finalize();
  }

It matches the logs from messages:
2017-08-10T07:29:50.483409-07:00 INFO cryptohomed[1223]: Attestation: Prepared successfully (8203ms).
2017-08-10T07:29:50.805204-07:00 INFO cryptohomed[1223]: Encrypted partition finalized.
2017-08-10T07:29:50.814166-07:00 INFO cryptohomed[1223]: InstallAttributes have been finalized.

That seems to be the restart after the system update happened. Need to understand why
(is_mounted && took_ownership && install_attrs_->is_first_install())
happens.

Comment 12 by test.t...@sausd.us, Aug 22 2017

No sign in before enrollment.  Just guest mode for updates.
I've managed to reproduce this. So, you update the OS manually from the guest mode before enrollment. Now it should be much easier for me to investigate.

For now you can avoid this problem by enrolling the device without updating. After enrollment the device should auto-update after first user sign in and the new version would hit after that restart.
The device was updated from Chrome OS version 56 to 59. After initial guest sign, the version 56 finalized the install attributes. That was the reason the updated version 59 was unable to enroll the device.

This seems to be a bug in version 56 that the install attributes was saved. Will need to investigate the code from that version.
Cc: vkasatkin@google.com
Labels: Hotlist-Enterprise
Labels: M-60 M-59
We have a customer reporting similar issue. Devices were disabled a few months ago, it was recovered but after re-enabled in Admin Console, but stuck on the locked screen with the return information. 
The device shows up as Provisioned on the Admin Console.
 
Wipe or Chrome recovery do not resolve the issue. The device goes to the lock screen after accepting the Terms of Service. The device sits on the "Determining device configuration" for a bit and goes into Disabled mode displaying the return information.
vkasatkin@, your description is a different scenario. IIUC, those devices cannot escape the disabled mode. Please submit a separate issue for that.
Project Member

Comment 18 by bugdroid1@chromium.org, Oct 12 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4a9ac22b42581cd3e08b3e0e1a6e831a5cefe8e5

commit 4a9ac22b42581cd3e08b3e0e1a6e831a5cefe8e5
Author: Igor <igorcov@chromium.org>
Date: Thu Oct 12 09:03:13 2017

Logs added for better picture of install attributes in enrollment

These logs should help investigate why the enrollment fails in some
cases.

Bug:  chromium:753397 
Test: None
Change-Id: I1f7409d6769b272d27fd1158aa5463c43a6394d7
Reviewed-on: https://chromium-review.googlesource.com/623247
Commit-Queue: Igor <igorcov@chromium.org>
Reviewed-by: Achuith Bhandarkar <achuith@chromium.org>
Cr-Commit-Position: refs/heads/master@{#508287}
[modify] https://crrev.com/4a9ac22b42581cd3e08b3e0e1a6e831a5cefe8e5/chrome/browser/chromeos/login/enrollment/enrollment_screen.cc
[modify] https://crrev.com/4a9ac22b42581cd3e08b3e0e1a6e831a5cefe8e5/chrome/browser/chromeos/settings/install_attributes.cc

Project Member

Comment 19 by bugdroid1@chromium.org, Oct 23 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/054bcf73ab35e3ef16f90de37cf81de2ce887687

commit 054bcf73ab35e3ef16f90de37cf81de2ce887687
Author: Igor <igorcov@chromium.org>
Date: Mon Oct 23 11:28:28 2017

cryptohome: Avoid finalizing install attributes for guest

When there's a guest session, the function IsMount returns true
because guest account is mounted. This change checks additionaly
for the mounted account to be non-guest when finalizing install
attributes.

BUG= chromium:753397 
TEST=None

Change-Id: I3d55670a20014c30b0fdb7f81705f7a57a4e6160
Reviewed-on: https://chromium-review.googlesource.com/712256
Commit-Ready: Igor <igorcov@chromium.org>
Tested-by: Igor <igorcov@chromium.org>
Reviewed-by: Gwendal Grignou <gwendal@chromium.org>
Reviewed-by: Andrey Pronin <apronin@chromium.org>

[modify] https://crrev.com/054bcf73ab35e3ef16f90de37cf81de2ce887687/cryptohome/service.cc

Status: Fixed (was: Started)
Tentatively marking as fixed. This issue shouldn't happen when the initial version on the device will be 64+.

Comment 21 by dchan@chromium.org, Jan 22 2018

Status: Archived (was: Fixed)

Comment 22 by dchan@chromium.org, Jan 23 2018

Status: Fixed (was: Archived)
Cc: ibezmenov@chromium.org
Status: Verified (was: Fixed)
Marked as Verified, this issue has not been observed by Enterprise test team during M-64/M-65 Full Release testing.

Also I've been trying to repro this updating device from the guest mode before enrollment (c#13). No issues with enrollment.

Sign in to add a comment