New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 701292 link

Starred by 12 users

Esc-Refresh-Power does not clean all data

Project Member Reported by omrilio@chromium.org, Mar 14 2017

Issue description

--- split from  issue 673859  ---

Users are reporting that after power wash with force enrollment (Esc + Refresh + Power) without switching to dev mode does not actually clear up all user space. 
The only way to clear up the space is to actually do a recovery via a USB or disable force re-enrollment and move the device into dev mode.

We should fix power wash to actually clean up the data being reported as being in use.

To help investigate the bug, we have the following feedback report list (39 items):
https://docs.google.com/spreadsheets/d/1L1WmY8gZ3HOhpzw-fychMsgXW4wyLdwDLVzKijrXOMo/edit#gid=0

uekawa@ can you help triaging?






List of reports for convenience:
report 1: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52820138741
report 2: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52820140170
report 3: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52819687984
report 4: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52819704921
report 5: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52819524731
report 6: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52819474153
report 7: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52818773751
report 8: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52818653855
report 9: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52818034957
report 10: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52817938193
report 11: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52817877685
report 12: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52817844357
report 13: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52662591157
report 14: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52662525175
report 15: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52662274803
report 16: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52662244314
report 17: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52661952362
report 18: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52661916687
report 19: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52661638358
report 20: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52661619981
report 21: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52661312824
report 22: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52661283455
report 23: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52660978062
report 24: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52660960605
report 25: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52660808796
report 26: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52660785120
report 27: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52660171733
report 28: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52660128279
report 29: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52660002683
report 30: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52659998969
report 31: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52548923783
report 32: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=52547911383
report 33: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=51812337284
report 34: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=51811920459
report 35: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=51811870542
report 36: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=51811847997
report 37: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=51811806012
report 38: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=51811789000
report 39: https://feedback.corp.google.com/product/208/neutron?&lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=673859&lReport=51804004895
 
Labels: -Pri-3 Pri-1

Comment 2 by cyrusm@chromium.org, Mar 14 2017

Cc: cyrusm@chromium.org

Comment 3 by uekawa@chromium.org, May 26 2017

Feedback links in #1 are mostly from candy ? 
I saw one from daisy.

https://cs.corp.google.com/chromeos_public/src/platform2/login_manager/session_manager_impl.cc?q=StartDeviceWipe&l=1090
fast safe keepimg reason=

should eventually call
src/platform2/init/clobber-state



From look at the code, it should write 0 to start of ${STATE_DEV} and call mkfs to recreate the file system, so it's unlikely that stateful file system does not get deleted unless the ${STATE_DEV} variable content is wrong.



Comment 4 by uekawa@chromium.org, May 26 2017

Grabbed a daisy device and tried clobbering stateful; saw that in /mnt/stateful_partition/unencrypted/clobber-state.log things look sane and stateful partition is formatted.

It would be useful to see contents of chrome://system "clobber-state.log"


I think there might be a misunderstanding here. Esc-Refresh-Power forces a reboot, but it does *NOT* trigger a power wash. It is perfectly expected for data on the device to remain present.

My understanding is that this bug was created in response to comment #26 here: https://bugs.chromium.org/p/chromium/issues/detail?id=673859#c26

If there's no other evidence of powerwash leaving around state and we don't have reproduction steps either, I think we should just close this.
When a device is attempted to switch to Developer mode, but Force Re-enrollment is enabled, should it trigger a powerwash?  If not, how can users (or administrators) reliably "powerwash" managed devices since powerwash through the typical methods is disabled on managed devices?
Surely this cannot be correct. This key combination does wipe settings including wifi and enrollment, which have to be reentered upon next boot. Also it appears that policies must be redownloaded, all user profiles are removed and apps must be redownloaded upon first login. If this is merely a forced reboot, why does the device not remain in it's previous state with all settings and profiles intact?

Comment 8 by uekawa@chromium.org, May 29 2017

Summary: Esc-Refresh-Power does not clean all data (was: powerwashing does not clean all data)
retitle to Esc-Refresh-Power. I presume that's switching to dev mode and back.
Clarification: Ctl+Alt+Refresh triggers an immediate reboot into "Chrome OS is missing or damaged screen". You can enter developer mode from there, but you don't have to. I.e. you can just as well turn the device off and back on by pressing the power button twice.

If you do enter developer mode (e.g. by pressing Ctrl-D etc.) the device will boot in developer mode, but when Forced Re-Enrollment is on (or more specifically, dev-mode blocking) the device will detect this and stop the boot process early on. It'll show a notification screen and trigger a switch back to normal mode. After another reboot, the device is back in normal mode and should wipe it's internal storage. As part of this mode transition, the TPM will have been reset (twice), and all encrypted data on internal storage that may still be present is inaccessible. 

Technically, the above is *NOT* a powerwash, but it does rely on some of the same underlying functionality that the powerwash feature also uses.

Technicalities aside, I think we'll need some clarification on reproduction steps that lead to the situation described in the original report e.g. the device to not "clear up all user space". If somebody can provide steps, that'll hopefully help make progress here.
So far from what mnissler's saying, it's working as intended.

Re comment #10: It's not unthinkable we don't clear the stateful FS in some specific edge case, in which case old encrypted cryptohome data may stick around. As mentioned before, we'd need repro steps to identify the issue and figure out a fix though.

Comment 12 by uekawa@google.com, Jul 12 2017

Blocking: 740503
Cc: fukino@chromium.org hashimoto@chromium.org uekawa@chromium.org kinaba@chromium.org naveenv@chromium.org gwendal@chromium.org
 Issue 740503  has been merged into this issue.
Cc: atwilson@chromium.org tnagel@chromium.org
+tnagel, +atwilson

FYI, this is the "device is back in OOBE but cryptohomes were not cleared" that I ran into last week.
Components: Privacy
Cc: msramek@chromium.org
Cc: jayhlee@google.com
Hi guys - this is a P1 bug that's been active since March.  Any updates here?
Owner: zalcorn@chromium.org
Any update?
Zach, want to take this over since it's OOBE/AUTH?

Comment 20 by uekawa@google.com, Oct 23 2017

summary from what I know about
 - it's possible to have an enterprise policy that disallows powerwash (powerwash would preserve some important files, but admins have the liberty of disallowing that).
 - being unable to powerwash means home directories don't get cleaned
 - somethings don't ever get cleaned up that way.
 
sounds like an enterprise policy issue to me.

As you can see most recently in #9, this is not about powerwash. Triggering recovery and exiting it drops you into OOBE but does not clear user data (under certain circumstances).
Powerwashing is also disabled when s device is Enterprise Enrolled.
Cc: zalcorn@chromium.org
Owner: dskaram@chromium.org
David, mind taking a look at this? It's not clear to me what the desired behavior is as powerwash is disabled, but users still want home directory data wiped?

Comment 24 by uekawa@google.com, Oct 30 2017

re #20 wasn't  probably concise in the terminology, 
It's not "powerwash" but at least "clobber-stateful" that runs when you turn on/off dev mode does not run.

I suspect that the user expects dev on/off will recreate the stateful partition in some useful manner, we don't if powerwash is disabled and keep using the (potentially) corrupted file system with junk remaining.

Ping - status?
Status: WontFix (was: Assigned)
Status is that we don't have a repro of any bug.

Here's our understanding:

1) Enter recovery by Esc-Refresh-Power, then exit recovery - data on disk should be left unchanged. This is what is reported as the "bug" but it's not a bug - this is WAI.

2) Enter dev mode (Esc-Refresh-Power, then Ctrl-D, then Enter) then exit Dev mode - this will clobber stateful (not powerwash, but a similar thing). As far as we can tell, this works.

So I'm closing this bug, because as far as I can tell both #1 and #2 are working correctly. If anyone has a case where #2 does not wipe the device, please reopen the bug with repro steps.
I think #1 was not the issue, the problem is with #2 in a scenario where forced re-enrollment is enabled. After Ctrl-D, the system cannot actually enter Dev mode as a result of forced re-enrollment. On screen it says "Your system will reboot and local data will be cleared" but this does not always clear the data. It does most of the time, not always. Many of us had been sending device logs after this occurs.

We are therefore stuck with Chromebooks that have nearly full internal storage which can only be remedied by reinstalling the OS via USB or disabling forced re-enrollment so we can get into dev mode where the device is fully wiped, exit dev mode and enable forced re-enrollment... a lengthy process. Whatever happens here is not the same with and without forced re-enrollment. All this has already been explained earlier in greater detail.

If the device storage is not really supposed to be cleared, why display exactly that on the screen when we press Ctrl-D?
From the other bug:

"I'm frequently observing that the Esc + Refresh + Power method (with forced re-enrollment enabled so there is no switch to dev mode) is actually not freeing up much space. All profiles are removed but out of ~10GB total usable space its showing 8GB still in use.

In these cases if I perform a full recovery via USB or disable forced re-enrollment and get the device into dev mode, a more thorough wipe is performed and most of the space is freed up as it should be. This is a time consuming, temporary solution."

This is weird to me - if profiles are deleted, that implies some kind of cleanup is happening, but why would it not free up all space?
Status: Assigned (was: WontFix)
Re-opening while we figure out why profiles would disappear but disk space would not be freed.
Cc: mnissler@chromium.org
+mnissler: Dev mode transition does not completely clear the stateful partition. Any thoughts?
Our school board is experiencing exactly this issue where profiles is not being deleted with the Esc + Refresh + Power, Ctrl +D method while Forced Enrollment is enabled.  Here the exact procedure I use to duplicate this issue EVERY time in my case:

1. I pull a chromebook from a classroom that has been in use for some time by a multitude of users.  
2. I log into the chromebook, and check the local storage.  See attachment 1.jpg for a sample of what the storage usage looks like.  Note that "Other users" is occupying 7.1GB.  (This is an Acer C740 with a 16GB SSD)
3. I press ESC+REFRESH+Power
4. The device reboots to the "Chrome OS is missing or damaged" screen.
5. I press CTRL+D
6. The screen switches to "To turn OS verification OFF, press ENTER..."
7. I press ENTER
8. The device reboots to a screen "OS verification is OFF.  Press SPACE to re-enable"
9. I press SPACE
10. The screen switches to "OS verification OFF.  Press ENTER to confirm you wish to turn OS verification on.  Your system will reboot and local data will be cleared."  
-Since the screen says that local data will be cleared, that I what I expect to happen at this point, however it does not.
11. I press ENTER.
12. The screen switches to "OS verification is ON.  Your system will reboot and local data will be cleared." for a couple seconds and the system reboots.
-Again it says that local data will be cleared but it isn't.
13. The system comes to OOB and I click Let's Go, connect to WIFI, and accept terms.
14. The system comes to Enterprise Enrollment automatically, at which point I enroll the system.
15. The system comes to the standard login screen, and I log in.
16. Looking at Storage Management in settings, I find that the drive is still full, however "Other users" is apparently taking up 0B of data.  See attachment 2.jpg

In order to properly clear the storage I must perform a USB recovery, which results in a properly cleared local storage (see Attachment 3.jpg for the final results)

Cc: -gwendal@chromium.org igorcov@chromium.org rspangler@chromium.org
So at no point do you actually see the progress bar with local data being cleared? Randall/Igor, any ideas here?
Nope, I never get a progress bar.

I assume you're referring to the one that is normally seen when you enter Developer mode?  In any case, there is no visible progress bar at any time.

There are no long wait periods at a black screen or anything, the entire process from steps 3-13 can be accomplished in 30 seconds.
Thanks, that's interesting to hear.
This is where Google loses credibility with school districts and other users of managed devices. The progress bar for clearing data upon turning OS verification back ON has NOT shown up for well over a year! It did some time back in early 2016 as I recall, but I haven't seen it since on many a managed device. I assumes, like many other admins, that it was intentional, but also assumed that data was being wiped silently (but obviously it wasn't, as this bug report indicates).
Re#32:

The firmware clears the TPM owner very early in any boot following a dev switch transition (off->on or on->off).

Clearing the TPM owner works, as indicated by profiles going away and the return to OOBE.

That does not wipe stateful in the firmware, but it should trigger the OS to do so.

The script which does the wipe is src/platform2/init/clobber-state

That should be called by src/platform2/init/chromeos_startup, lines 223-268.

(Note that Esc+Refresh+Power should NOT clean all data.  That's simply going into recovery mode.  Toggling the developer switch state is what should trigger the clobber.  So the bug title is currently a little misleading.)
Cc: apronin@chromium.org
Sounds like wiping stateful partition may not be freeing up disk space from previously created user cryptohomes, which is weird to me. I wonder what can cause that - disk corruption? Who best understands how user cryptohomes get freed up post clobber-state?
And, aha, that's the problem.  chromeos_startup line 247:

elif [ -O "${DEV_MODE_FILE}" ]; then
  if crossystem 'devsw_boot?0' && ! crossystem 'mainfw_type?recovery'; then
    # We're transitioning from dev mode to verified boot.
    # When coming back from developer mode, we don't need to
    # clobber as aggressively.  Fast will do the trick.
    chromeos-boot-alert leave_dev
    echo "fast keepimg" >"${RESET_FILE}"
    clobber-log -- "Leave developer mode"
  fi

Developer mode blocking was first implemented in chromeos_startup.  That check is still there, at line 103: 

# Developer mode functions (defined in dev_utils.sh and will be loaded
# only when CROS_DEBUG=1).
dev_check_block_dev_mode() { true; }

So when you toggled dev mode on, the system would boot through the dev warning screen, start the OS, and get to that point, at which point the wipe would be triggered even if dev mode was disabled.

But now, developer mode is blocked more securely by the firmware, using a flag stored into the TPM at OOBE on enrolled devices.  So *the OS never boots*.  So it never calls dev_check_block_dev_mode(), and never creates ${DEV_MODE_FILE} either.

So when you return to normal mode, the OS has no memory of ever seeing developer mode, and the check at line 247 doesn't trigger.

So, no wipe.
As for how to fix this: In addition to checking the developer flag file, we should check for the TPM transitioning from owned to not-owned.

If the TPM owner has been cleared, then trigger a fast wipe.

This could be as simple as storing the TPM-owned state in a flag file similar to ${DEV_MODE_FILE}, and checking/updating that every boot.

Note that powerwash clears the TPM owner; we'd need to make sure that adding this check doesn't cause stateful to be wiped twice (once by the powerwash and then once by this new check).

Re#39: Note that we may break that flow too; see go/vboot-clear-tpm-owner.
So idea would be that we'd create a file when we own the TPM (isn't that already install_attributes?), and on boot of that file exists but TPM is unowned we wipe the device? That seems like it would avoid the double-wipe for powerwash.
Re#41: Yes.
Maybe a different approach to look at this: Clearing the TPM deletes the system salt and makes encrypted data unrecoverable. cryptohomed detects this because it fails to mount and then deletes the old cryptohome and creates a new one. Maybe we could add code there that deletes user directories as well because they have become unmountable for the same reason?
atwilson/dskaram - should we fix this now that we know what's going on (based on the discussion above)?  I know we're looking at other things for clearing data as well, but seems like this is the flow people are accustomed to?
Owner: igorcov@chromium.org
Yes, and assigning to Igor. Re comment #39, I think we *should* be OK wrt doing a double-wipe, since the first wipe will remove the "TPM_IS_OWNED" flag file so we won't think it's owned and do a second wipe, correct?
Project Member

Comment 46 by bugdroid1@chromium.org, Jan 22 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/7b1846e66cb13c907618785598924ccd3e604401

commit 7b1846e66cb13c907618785598924ccd3e604401
Author: Igor <igorcov@chromium.org>
Date: Mon Jan 22 18:13:40 2018

init: Included TPM transition case when deciding to wipe

When the devmode is blocked by firmware management parameters
from TPM NVRAM, after switching from devmode to verified boot or
back the ${DEV_MODE_FILE} is not created.
To clear the stateful partitions in this case additional check
is added to see if the TPM is not owned but install_attributes
file is present. This means the device was owned and the TPM was
cleared.

BUG= chromium:701292 
TEST=None

Change-Id: I6f2c83daf389d900faca8977f1386a94b19d1c63
Reviewed-on: https://chromium-review.googlesource.com/870312
Commit-Ready: Igor <igorcov@chromium.org>
Tested-by: Igor <igorcov@chromium.org>
Reviewed-by: Randall Spangler <rspangler@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/7b1846e66cb13c907618785598924ccd3e604401/init/chromeos_startup

Status: Fixed (was: Assigned)
Marking as fixed, but the change is starting from version 66.
Great work guys, looking forward to V66!
Thanks!! :)
Labels: M-66
Cc: ibezmenov@chromium.org
When attempted to verify this bug, I have followed the steps provided in c#31:

1. Force Re-Enrollment enabled, Dev mode blocked.
2. Initial State (attachment 1).
3. After Esc+Refresh+Power (attachment 2).
4. After USB recovery (attachment 3).

Looks like Esc+Refresh+Power still does not clean all data.

igorcov@: could you please clarify what user should expect with this fix?

Chrome OS: 10452.5.0
Chrome: 66.0.3359.21
Device: Santa
Screenshot 2018-03-09 at 3.10.25 PM.png
90.9 KB View Download
Screenshot 2018-03-09 at 3.17.52 PM.png
92.2 KB View Download
Screenshot 2018-03-09 at 3.31.24 PM.png
90.2 KB View Download
Labels: Needs-Feedback
Status: Assigned (was: Fixed)
@igorcov, could you please review #c50 and reply.
I've checked the code in version 66.0.3359.21 and the changes I've made in chromeos_startup are there. Will try to reproduce and check the clober-state logs.
I couldn't reproduce on Eve. The clobber logs show the device was cleared. ibezmenov@ also tried again to reproduce with version 66.0.3359.52 and couldn't.
After few attempts I was able to reproduce this issue.

Followed the same steps provided in c#31:

1. Force Re-Enrollment enabled, Dev mode blocked.
2. Initial State (attachment 1).
3. After Esc+Refresh+Power (attachment 2).

I both cases clobber-state.log and clobber.log are empty.

Debug logs attached.

Chrome OS: 10452.28.0
Chrome: 66.0.3359.62
Device: Santa
Screenshot 2018-03-27 at 10.10.49 AM.png
88.1 KB View Download
Screenshot 2018-03-27 at 10.24.36 AM.png
86.2 KB View Download
debug-logs_20180327-103811.tgz
716 KB Download

Comment 55 by loyso@chromium.org, Mar 28 2018

Labels: CrOSFilesFeature-LowStorage
I've managed to reproduce, using the steps from #31. But, if we change step 9 and instead of pressing space, we press ctrl+D forcing the device to check and give us the message that the device is forbidden to go in devmode, then the clobber is executed.

So, for people to clear the device they need to change the step 9 in comment #31, to press ctrl+D instead of space.

I confirm that if we change step 9 (c#31) and instead of pressing space we press ctrl+D, it clears the device.

1. Force Re-Enrollment enabled, Dev mode blocked.
2. Initial State (attachment 1, clobber-state.log and clobber.log are empty).
3. After Esc+Refresh+Power (attachment 2, clobber-state.log and clobber.log are created).

The clobber-state.log and clobber.log attached.

Chrome OS: 10452.44.0
Chrome: 66.0.3359.79
Device: Santa
Screenshot 2018-04-04 at 10.33.08 AM.png
130 KB View Download
Screenshot 2018-04-04 at 10.42.18 AM.png
134 KB View Download
clobber-state & clobber.log
3.6 KB View Download
Labels: Hotlist-Enterprise
Looks like the problem is that install_attributes file is not present in home when we check, because home is not mounted yet. It happens few lines after. But the install_attributes from /mnt/stateful_partition/ is present.

Created https://chromium-review.googlesource.com/c/chromiumos/platform2/+/999636 to fix this.
Project Member

Comment 60 by bugdroid1@chromium.org, Apr 18 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/913b862c4aae9c3260b5599283cb49bf123cbc49

commit 913b862c4aae9c3260b5599283cb49bf123cbc49
Author: Igor <igorcov@chromium.org>
Date: Wed Apr 18 16:30:31 2018

init: Fix location for install_attributes,

The folder /home is not mounted yet, when we need to check if
install_attributes file is present. This fixes the location to check
in /mnt/stateful_partition instead.

BUG= chromium:701292 
TEST=Manual test

Change-Id: I7ec79056b3c8cfd5f3d1565c783203d7d3fd1a04
Reviewed-on: https://chromium-review.googlesource.com/999636
Commit-Ready: Igor <igorcov@chromium.org>
Tested-by: Igor <igorcov@chromium.org>
Reviewed-by: Igor <igorcov@chromium.org>
Reviewed-by: Randall Spangler <rspangler@chromium.org>
Reviewed-by: Sonny Rao <sonnyrao@chromium.org>

[modify] https://crrev.com/913b862c4aae9c3260b5599283cb49bf123cbc49/init/chromeos_startup

Comment 61 by loyso@chromium.org, Apr 19 2018

Status: Started (was: Assigned)
Status: Fixed (was: Started)
This should be fixed in 68.

Comment 64 by loyso@chromium.org, Apr 20 2018

Labels: -M-66 M-68
Project Member

Comment 65 by bugdroid1@chromium.org, Apr 20 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/2ff880c35e9d6036c07804ee19cbbf67ed1335a1

commit 2ff880c35e9d6036c07804ee19cbbf67ed1335a1
Author: Igor <igorcov@chromium.org>
Date: Fri Apr 20 09:48:41 2018

Revert "init: Fix location for install_attributes,"

This reverts commit 913b862c4aae9c3260b5599283cb49bf123cbc49.

Reason for revert: This commit caused canary 68.10598.0 to fail.
If a user had set up the system already, shutting down or rebooting
will cause a dev-to-normal wipe, despite the system never leaving
normal mode.

Original change's description:
> init: Fix location for install_attributes,
>
> The folder /home is not mounted yet, when we need to check if
> install_attributes file is present. This fixes the location to check
> in /mnt/stateful_partition instead.
>
> BUG= chromium:701292 
> TEST=Manual test
>
> Change-Id: I7ec79056b3c8cfd5f3d1565c783203d7d3fd1a04
> Reviewed-on: https://chromium-review.googlesource.com/999636
> Commit-Ready: Igor <igorcov@chromium.org>
> Tested-by: Igor <igorcov@chromium.org>
> Reviewed-by: Igor <igorcov@chromium.org>
> Reviewed-by: Randall Spangler <rspangler@chromium.org>
> Reviewed-by: Sonny Rao <sonnyrao@chromium.org>

BUG= chromium:834958 
TEST=OS image on a normal mode Chromebook does not result in a
dev-to-normal wipe every time after a reboot or shutdown.

Change-Id: I9d2c8bcaafa6d50afba3a970be3164a5488f660e
Reviewed-on: https://chromium-review.googlesource.com/1019181
Commit-Ready: Benson Leung <bleung@chromium.org>
Tested-by: Benson Leung <bleung@chromium.org>
Reviewed-by: Benson Leung <bleung@google.com>

[modify] https://crrev.com/2ff880c35e9d6036c07804ee19cbbf67ed1335a1/init/chromeos_startup

Status: Started (was: Fixed)
Seems like the cryptohome daemon is not started yet when chromeos_startup is running and the status of TPM is unknown. Will check other options, like existence of salt or Thiemo's proposal in #43 to remove the user folder when we create the new cryptohome.
I am seeing a dev-to-normal wipe on 68.0.3400.0 / 10598.0.0 (Official Build) canary-channel samus, both on reboot and resume-from-suspend. Screen saying "Transitioning from dev mode to verified." Device is not in dev mode. Account is jamescook@google.com. I filed feedback, but cannot provide a link since I don't have a cert yet.

The reboot issue sounds like this bug, but I don't see references to resume. Is this bug the cause?

Re CL https://crrev.com/c/870312 from comment #46: it attempts to run cryptohome CLI utility to determine if the tpm is owned too early in the boot sequence, when cryptohomed is not running yet and can't provide the tpm status. As the result, is_tpm_owned() in chromeos_startup must always be false. Left comment on the CL.

Blockedon: 846371
Re #66: cryptohomed is definitely not started yet when chromeos_startup is running. Alternative means for checking if tpm is owned at this point include:
1) tpmc getpf (and checking ownerAuthSet/ownership for tpm 2.0/1.2)
2) checking /sys/class/(misc|tpm)/tpm0/device/owned - the exact sysfs path depends on the kernel version.
So far, testing on a reks device
tpmc getpf gives ownership 1 in both scenarios when chromeos_startup runs.
/sys/class/tpm/tpm0/device/owned is present in both scenarios.

Maybe the TPM is not cleared at that point yet.
Actually it works well by checking the contents of the device/owned file. I've created https://chromium-review.googlesource.com/c/chromiumos/platform2/+/1075274
based on this.
Project Member

Comment 72 by bugdroid1@chromium.org, May 30 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/6b42d1a29aba007b5399b5eeed977a6de2ef154b

commit 6b42d1a29aba007b5399b5eeed977a6de2ef154b
Author: Igor <igorcov@chromium.org>
Date: Wed May 30 00:15:27 2018

Revert "init: Included TPM transition case when deciding to wipe"

This reverts commit 7b1846e66cb13c907618785598924ccd3e604401.

Reason for revert: The cryptohome is not initialized when called here and the CL has no effect Just causes cryptohome to crash.
https://bugs.chromium.org/p/chromium/issues/detail?id=846371

Original change's description:
> init: Included TPM transition case when deciding to wipe
>
> When the devmode is blocked by firmware management parameters
> from TPM NVRAM, after switching from devmode to verified boot or
> back the ${DEV_MODE_FILE} is not created.
> To clear the stateful partitions in this case additional check
> is added to see if the TPM is not owned but install_attributes
> file is present. This means the device was owned and the TPM was
> cleared.
>
> BUG= chromium:701292 
> TEST=None
>
> Change-Id: I6f2c83daf389d900faca8977f1386a94b19d1c63
> Reviewed-on: https://chromium-review.googlesource.com/870312
> Commit-Ready: Igor <igorcov@chromium.org>
> Tested-by: Igor <igorcov@chromium.org>
> Reviewed-by: Randall Spangler <rspangler@chromium.org>
> Reviewed-by: Mike Frysinger <vapier@chromium.org>

Bug:  chromium:701292 
Change-Id: Ie64a2d3defad76be9242292e911e0e24d6a80444
Reviewed-on: https://chromium-review.googlesource.com/1071967
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: Igor <igorcov@chromium.org>
Reviewed-by: Igor <igorcov@chromium.org>
Reviewed-by: Andrey Pronin <apronin@chromium.org>

[modify] https://crrev.com/6b42d1a29aba007b5399b5eeed977a6de2ef154b/init/chromeos_startup

Project Member

Comment 73 by bugdroid1@chromium.org, Jun 2 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/3a685f341582d24df30fffbc0161df4e724b236f

commit 3a685f341582d24df30fffbc0161df4e724b236f
Author: Igor <igorcov@chromium.org>
Date: Sat Jun 02 00:44:24 2018

init: Fix the check if clobber has to run after clear tpm

The previous version was checking if the TPM is owned using the cryptohome
command. That doesn't work when the chrome OS starts, and it was always
returning false. Now it's fixed by checking the contents of device/owned
file.

Also the location of install attributes file that was checked in previous
version was wrong, since /home/.shadow/ is now mounted yet at that point.
This is fixed now too.

BUG= chromium:701292 
TEST=Build a image that boots in verified mode and check clobber state logs
after a normal boot and after a boot in devmode switched to verified mode.

Change-Id: I4d3d593fa29e42ea1cd94682962c7e07ea10b695
Reviewed-on: https://chromium-review.googlesource.com/1075274
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: Igor <igorcov@chromium.org>
Reviewed-by: Igor <igorcov@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/3a685f341582d24df30fffbc0161df4e724b236f/init/chromeos_startup

Status: Fixed (was: Started)
This should be fixed now. Checked with version 69.0.3463.0 on platform 10791.0.0.
Status: Verified (was: Fixed)
I have verified this pressing Space in step 9 (c#31), it clears the device.

1. Force Re-Enrollment enabled, Dev mode blocked.
2. Initial State (screenshot 1, clobber-state & clobber logs (before) attached).
3. After Esc+Refresh+Power (screenshot 2, clobber-state & clobber logs (after) attached).

Chrome OS: 10805.0.0
Chrome: 69.0.3464.0
Device: Robo
Screenshot 1.png
138 KB View Download
Screenshot 2.png
139 KB View Download
clobber-state & clobber logs (before)
3.8 KB View Download
clobber-state & clobber logs (after)
3.9 KB View Download
Cc: sjg@chromium.org
Project Member

Comment 77 by bugdroid1@chromium.org, Oct 22

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/a6794b2c4ec4c8c16e9e02fd4566046ad1f77422

commit a6794b2c4ec4c8c16e9e02fd4566046ad1f77422
Author: Jonathan Brandmeyer <jbrandmeyer@chromium.org>
Date: Mon Oct 22 13:16:39 2018

init: Correctly clobber-state on entry to developer mode

A narrow logic hole exists on the very first time that chromiumos enters
developer mode, when the TPM is unowned and the install_attributes file
exists.  Reorder the logic such that a wipe is triggered.

BUG=b:112369945, b:117163103, b:113345814
BUG= chromium:701292 
TEST=On Grunt.Careena hardware, construct a recovery image with this
change, and install it.  Enter developer mode, and observe that the
stateful partition is clobbered exactly once with a 5 min wait on the
first reboot.  Exit developer mode, and observe that the stateful
partition is clobbered exactly once without an extra wait.

Change-Id: I39df9d311fde9241aa3b33bb0e012826b2d294cd
Signed-off-by: Jonathan Brandmeyer <jbrandmeyer@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/1289632
Commit-Ready: Mattias Nissler <mnissler@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Andrey Pronin <apronin@chromium.org>

[modify] https://crrev.com/a6794b2c4ec4c8c16e9e02fd4566046ad1f77422/init/chromeos_startup

Project Member

Comment 78 by bugdroid1@chromium.org, Oct 22

Labels: merge-merged-release-R70-11021.B
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/570de651116641637e9327c9a19754c8f87b6614

commit 570de651116641637e9327c9a19754c8f87b6614
Author: Jonathan Brandmeyer <jbrandmeyer@chromium.org>
Date: Mon Oct 22 14:58:27 2018

init: Correctly clobber-state on entry to developer mode

A narrow logic hole exists on the very first time that chromiumos enters
developer mode, when the TPM is unowned and the install_attributes file
exists.  Reorder the logic such that a wipe is triggered.

BUG=b:112369945, b:117163103, b:113345814
BUG= chromium:701292 
TEST=On Grunt.Careena hardware, construct a recovery image with this
change, and install it.  Enter developer mode, and observe that the
stateful partition is clobbered exactly once with a 5 min wait on the
first reboot.  Exit developer mode, and observe that the stateful
partition is clobbered exactly once without an extra wait.

Change-Id: I39df9d311fde9241aa3b33bb0e012826b2d294cd
Signed-off-by: Jonathan Brandmeyer <jbrandmeyer@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/1289632
Commit-Ready: Mattias Nissler <mnissler@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Andrey Pronin <apronin@chromium.org>
(cherry picked from commit a6794b2c4ec4c8c16e9e02fd4566046ad1f77422)
Reviewed-on: https://chromium-review.googlesource.com/c/1293730

[modify] https://crrev.com/570de651116641637e9327c9a19754c8f87b6614/init/chromeos_startup

Project Member

Comment 79 by bugdroid1@chromium.org, Oct 22

Labels: merge-merged-release-R71-11151.B
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/0b682411cb8933111a6951d908a80519b1e38ff2

commit 0b682411cb8933111a6951d908a80519b1e38ff2
Author: Jonathan Brandmeyer <jbrandmeyer@chromium.org>
Date: Mon Oct 22 15:46:17 2018

init: Correctly clobber-state on entry to developer mode

A narrow logic hole exists on the very first time that chromiumos enters
developer mode, when the TPM is unowned and the install_attributes file
exists.  Reorder the logic such that a wipe is triggered.

BUG=b:112369945, b:117163103, b:113345814
BUG= chromium:701292 
TEST=On Grunt.Careena hardware, construct a recovery image with this
change, and install it.  Enter developer mode, and observe that the
stateful partition is clobbered exactly once with a 5 min wait on the
first reboot.  Exit developer mode, and observe that the stateful
partition is clobbered exactly once without an extra wait.

Change-Id: I39df9d311fde9241aa3b33bb0e012826b2d294cd
Signed-off-by: Jonathan Brandmeyer <jbrandmeyer@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/1289632
Commit-Ready: Mattias Nissler <mnissler@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Andrey Pronin <apronin@chromium.org>
(cherry picked from commit a6794b2c4ec4c8c16e9e02fd4566046ad1f77422)
Reviewed-on: https://chromium-review.googlesource.com/c/1293731

[modify] https://crrev.com/0b682411cb8933111a6951d908a80519b1e38ff2/init/chromeos_startup

Sign in to add a comment