New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 892525 link

Starred by 8 users

Issue metadata

Status: Closed
Owner:
Closed: Dec 5
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

Chromebook losing enrollment on version 69.0.3497.95.

Project Member Reported by ryutas@chromium.org, Oct 5

Issue description

ChromeOS version: 69.0.3497.95
ChromeOS device model: Lenovo 500e Chromebook
Case#: 17022490
Number of affected devices: 1
Serial Number: P202DEXA

Description: Chromebook losing enrollment on version 69.0.3497.95.

Steps to reproduce: There is not specific repro info due to a student is using.

Current Behavior / Reproduction: The student has reported the device shows the Welcome screen, with force-enrollment rule applied.

Expected Behavior: the deice shouldn’t be losing enrollment.

Drive link to logs: 
https://drive.google.com/open?id=1_6s8NCoBuneEEaKNsvpZj0u3rcsfft6q

eventlog
258 | 2018-10-03 07:55:23 | Kernel Event | Clean Shutdown
259 | 2018-10-03 07:55:23 | ACPI Enter | S5
260 | 2018-10-03 07:55:58 | System boot | 94
261 | 2018-10-03 07:55:58 | Memory Cache Update | Variable | Success
262 | 2018-10-03 07:57:12 | Kernel Event | Clean Shutdown
263 | 2018-10-03 07:57:13 | ACPI Enter | S5
264 | 2018-10-03 07:57:51 | System boot | 95
265 | 2018-10-03 07:57:51 | Memory Cache Update | Variable | Success

The user reported the issue on October 3, 2018, but there is no wipe record on the same day.

from the message log,
I also noticed that below Panic message
2018-10-03T14:55:14.359246+00:00 INFO kernel: [    0.000000] Initializing cgroup subsys cpuset
2018-10-03T14:55:14.359327+00:00 INFO kernel: [    0.000000] Initializing cgroup subsys cpu

2018-10-03T14:55:14.359332+00:00 INFO kernel: [    0.000000] Command line: cros_secure console= loglevel=7 init=/sbin/init cros_secure oops=panic panic=-1 root=/dev/dm-0 

It also looks like eventlog timing and message log message timing doe not match.

similar case: crbug.com/878595
 
Labels: Enterprise-Triaged
Owner: igorcov@chromium.org
Status: Unconfirmed (was: Untriaged)
Igor, can you please take a look.
There's a wipe record from Oct 3, it went through clobber (from clobber.log):
2018/10/03 14:54:33 UTC Leave developer mode

So, no logs from before the enrollment was lost. Will check if I can find why the device decided that it left the developer mode.
Cc: apronin@chromium.org
One thing to check, would be if the device doesn't think mistakenly that the TPM is not owned.
Andrey, is it possible that here:
https://cs.corp.google.com/chromeos_public/src/platform2/init/chromeos_startup?q=chromeos_sta&sq=package:&g=0&l=59
the function would return false for some reason?
It might be worth logging a separate text for that case, to be able to distinguish in the future.
Re #3: it's possible that that file doesn't exist or is empty if there are some communication issues with the TPM. Or if TPM in some special state that returns an error in response to all requests to it. In that case would be treated as 'not owned'.
It probably makes sense to reverse the logic and check for != 0 instead of == 1 to avoid doing "chromeos-boot-alert leave_dev" after transient errors. It should be noted, though, that known special error states would cause recovery screen, so we won't get to chromeos_startup. There are also no known communication errors after a couple were fixed for TPM 2.0. 

As a side note:
Do we know the history of that device before it entered into this state. By looking at eventlog.txt and clobber.log (note: clobber.log is UTC, eventlog is local time, which I assume is UTC-7), it seems it started with triggering recovery, followed by two OS (or just cr50) updates?

<< boot after recovery key combo pressed >>
31 | 2018-09-24 07:23:15 | System boot | 43

33 | 2018-09-24 07:23:15 | Chrome OS Recovery Mode | Recovery Button Pressed | 0x02
34 | 2018-09-24 07:23:15 | EC Event | Keyboard Recovery

<< shutdown 5 minutes later >>
35 | 2018-09-24 07:29:30 | Kernel Event | Clean Shutdown
36 | 2018-09-24 07:29:31 | System boot | 44
37 | 2018-09-24 07:29:31 | System Reset

<<cr50 Update Reset indicates that cr50 f/w was updated, which typically comes with Chrome OS update>>
38 | 2018-09-24 07:29:31 | cr50 Update Reset
39 | 2018-09-24 07:29:31 | ACPI Enter | S5
40 | 2018-09-24 07:29:34 | System boot | 45
41 | 2018-09-24 07:29:34 | Memory Cache Update | Variable | Success

<< reboot ~1h30m later, followed by more reboots >>
42 | 2018-09-24 08:47:50 | Kernel Event | Clean Shutdown
43 | 2018-09-24 08:47:50 | ACPI Enter | S5
44 | 2018-09-24 08:48:01 | System boot | 46

48 | 2018-09-24 08:49:13 | Kernel Event | Clean Shutdown
49 | 2018-09-24 08:49:13 | ACPI Enter | S5
50 | 2018-09-24 08:49:32 | System boot | 47

54 | 2018-09-24 09:44:50 | Kernel Event | Clean Shutdown
56 | 2018-09-24 09:44:54 | System boot | 48

<< no clean shutdown here, cr50 update again >>
57 | 2018-09-24 09:44:54 | Memory Cache Update | Variable | Success
58 | 2018-09-24 09:44:54 | Wake Source | Power Button | 0
59 | 2018-09-24 09:44:54 | ACPI Wake | S5
60 | 2018-09-24 10:51:22 | System boot | 49
61 | 2018-09-24 10:51:22 | Memory Cache Update | Variable | Success
62 | 2018-09-24 10:51:22 | cr50 Update Reset
63 | 2018-09-24 10:51:22 | ACPI Enter | S5
64 | 2018-09-24 10:51:25 | System boot | 50
65 | 2018-09-24 10:51:25 | Memory Cache Update | Variable | Success

<< again, no clean shutdown, boot & clobber after it >>
66 | 2018-09-25 10:34:12 | System boot | 51
2018/09/25 17:34:41 UTC Leave developer mode
2018/09/25 17:34:43 UTC (preserve log): /sbin/clobber-state fast keepimg
2018/09/25 17:34:49 UTC (restore log): /sbin/clobber-state
2018/09/28 16:34:20 UTC Leave developer mode
2018/09/28 16:34:22 UTC (preserve log): /sbin/clobber-state fast keepimg
2018/09/28 16:34:28 UTC (restore log): /sbin/clobber-state
2018/10/03 14:54:33 UTC Leave developer mode
2018/10/03 14:54:35 UTC (preserve log): /sbin/clobber-state fast keepimg
2018/10/03 14:54:41 UTC (restore log): /sbin/clobber-state

Components: OS>Systems>Security
Update from the customer,
-Re-imaged the device P202DEXA and found it had lost enrollment. 
-He also found something he have not seen before, when he left the device on the pick a SSID screen, after less than an hour it went into a demo mode. 


Re #6:

> Re-imaged the device [...]

1) What does "re-imaging" mean in this case? Installed a new image from a usb stick in the recovery mode? Or rebooted to perform a pending auto-update?
2) Do I get it right that the customer re-imaged the device on ~Sep 24 and immediately after that saw the issue described in this bug? Or this update in comment #6 describes what happened _after_ seeing this issue and capturing the logs?
We have seen the same issue occur across our ~300 Dell Chromebooks.  Chromebooks show welcome screen after startup.  After the user connects to WiFi, they go through forced-enrollment.   We're happy to contribute to the investigation or testing possible solutions as we tend to have a few lose enrollment every day.
We're seeing a similar issue on a Dell 3380 in our organization. Wiped and updated to 70.x today. 
Could not reproduce the issue on 
Google Chrome: 69.0.3497.95 Platform: 10895.56.1 santa 
Google Chrome: 70.0.3538.107 Platform: 11021.79.0 santa

With FRE enabled, enrollment is still shown after recovery(M69) and update(M69-M70)
 
Labels: Needs-Feedback
Please provide steps if the issue is reproducible. Thanks.!
Labels: -Needs-Feedback
Status: Closed (was: Unconfirmed)
Issue is not reproducible on 
Google Chrome(69.0.34978.120,10895.78.0) asuka
Google Chrome(70.0.3538.110,11021.81.0) asuka

With FRE enabled, enrollment is still shown after recovery(M69) and update(M69-M70)

Closing the issue for now.

Sign in to add a comment