New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 713556 link

Starred by 2 users

Issue metadata

Status: Archived
Owner:
Last visit > 30 days ago
Closed: May 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug

Blocked on:
issue 714075

Blocking:
issue 706017



Sign in to add a comment

User profile not removed after a failed direncryption migration

Project Member Reported by dspaid@chromium.org, Apr 20 2017

Issue description

Chrome Version: 60.0.3077.0
OS: ChromeOS 9478
Device: Kevin

What steps will reproduce the problem?
(1) Modify migration_helper.cc to return false immediately on Migrate
(2) Install on ChromeOS with direncryption USE flags
(3) Attempt to migrate to ext4-crypto
(4) See failure screen, click through
(5) Attempt to log in

What is the expected result?
Profile should have been removed and login should succeed

What happens instead?
Browser crashes returns to the login screen
 
messages
659 KB View Download
ui.LATEST
11.7 KB Download

Comment 1 by fukino@chromium.org, Apr 20 2017

Cc: dspaid@chromium.org
Status: Started (was: Untriaged)
There seems to be several issues:

1) After migration failure, everything is still mounted for migration.
  => Dan, is it possible to unmount them on failure?
2) "Restart" button on the failure screen only restarts Chrome, not the system.
  => We should request restart to power manager. It makes migration process a bit longer, but worth doing for safety. I'll do it.
3) Chrome can crash on login attempt when everything is mounted for migration.
  => I'll look into this further. if 1) or 2) is fixed, this should not happen, though.

Comment 2 by fukino@chromium.org, Apr 20 2017

Correction: Fixing only 2) will not solve the problem.
Even when we unmount user directories by restarting the system, we can enter the restart loop if we don't clean up the user directory which we failed to migrate.

Comment 3 by dspaid@chromium.org, Apr 20 2017

I believe we unmount on success, so unmounting on failure seems reasonable.

Comment 4 by fukino@chromium.org, Apr 21 2017

Blockedon: 714075

Comment 5 by fukino@chromium.org, Apr 21 2017

Cc: -dspaid@chromium.org fukino@chromium.org
Owner: dspaid@chromium.org
Separated 2) in comment #1 as  Issue 714075 .
Let me assign this issue to Dan for fixing 1) in comment #1.

Comment 6 by uekawa@google.com, Apr 24 2017

Labels: ArcExt4Migration

Comment 7 by fukino@chromium.org, Apr 25 2017

Blocking: 706017
Cc: -fukino@chromium.org dspaid@chromium.org
Owner: fukino@chromium.org
We've decided that Chrome removes user directory when it is notified migration failure.
I'll take this.

Comment 8 by fukino@chromium.org, Apr 25 2017

dspaid@,

When the migration filed in intermediate state (i.e. we have both old tree and new tree),
Does AsyncRemove DBUS call remove both trees?
Project Member

Comment 9 by bugdroid1@chromium.org, Apr 25 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/467df89c1c669f0fb875a6bf445076bcdc51cf55

commit 467df89c1c669f0fb875a6bf445076bcdc51cf55
Author: Dan Spaid <dspaid@google.com>
Date: Tue Apr 25 21:03:58 2017

cryptohome: UnmountAll if crypto migration fails.

UnmountAll is currently called only when the dircrypto migration
succeeds.  For consistency it should also be called if the migration
fails.

BUG= chromium:713556 
TEST=cros_workon_make --board=samus --test cryptohome

Change-Id: I65c49f3e5abc727160b086ea5db19957318eda63
Reviewed-on: https://chromium-review.googlesource.com/484179
Commit-Ready: Dan Spaid <dspaid@chromium.org>
Tested-by: Dan Spaid <dspaid@chromium.org>
Reviewed-by: Ryo Hashimoto <hashimoto@chromium.org>

[modify] https://crrev.com/467df89c1c669f0fb875a6bf445076bcdc51cf55/cryptohome/mount.cc

Project Member

Comment 10 by bugdroid1@chromium.org, Apr 25 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/467df89c1c669f0fb875a6bf445076bcdc51cf55

commit 467df89c1c669f0fb875a6bf445076bcdc51cf55
Author: Dan Spaid <dspaid@google.com>
Date: Tue Apr 25 21:03:58 2017

cryptohome: UnmountAll if crypto migration fails.

UnmountAll is currently called only when the dircrypto migration
succeeds.  For consistency it should also be called if the migration
fails.

BUG= chromium:713556 
TEST=cros_workon_make --board=samus --test cryptohome

Change-Id: I65c49f3e5abc727160b086ea5db19957318eda63
Reviewed-on: https://chromium-review.googlesource.com/484179
Commit-Ready: Dan Spaid <dspaid@chromium.org>
Tested-by: Dan Spaid <dspaid@chromium.org>
Reviewed-by: Ryo Hashimoto <hashimoto@chromium.org>

[modify] https://crrev.com/467df89c1c669f0fb875a6bf445076bcdc51cf55/cryptohome/mount.cc

Yes, it should remove everything under /home/.shadow/<id> (as well as the mount points at /home/{root,user}/<id>
Thank you for the confirmation!
I'll call AsyncRemove when the UI receives migration failure.

Comment 13 by uekawa@google.com, Apr 26 2017

Labels: M-60
Can this be done before first dev push?
Project Member

Comment 14 by bugdroid1@chromium.org, Apr 27 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1339ec6addf0faa53da2c7418a0de5b6803d89cf

commit 1339ec6addf0faa53da2c7418a0de5b6803d89cf
Author: fukino <fukino@chromium.org>
Date: Thu Apr 27 06:48:40 2017

cros: Remove user directory when encryption migration fails.

When the migration UI is notified by cryptohome that cryptohome gets unexpected
errors during migration and it gives up migration, we should remove the
user's cryptohome to make sure that the user will be able to log in to
the Desktop next time.

After the user's cryptohome is removed, we get password verification errors
on the existing user pod three times.
It seems an independent issue, so I filed  crbug.com/715474  and will look into separately.

BUG= 713556 
TEST=manually tested by forcing cryptohome fail during migration and confirming the cryptohome is gone.

Review-Url: https://codereview.chromium.org/2838303003
Cr-Commit-Position: refs/heads/master@{#467608}

[modify] https://crrev.com/1339ec6addf0faa53da2c7418a0de5b6803d89cf/chrome/browser/ui/webui/chromeos/login/encryption_migration_screen_handler.cc
[modify] https://crrev.com/1339ec6addf0faa53da2c7418a0de5b6803d89cf/chrome/browser/ui/webui/chromeos/login/encryption_migration_screen_handler.h

Labels: -M-60 Merge-Request-59 M-59
Requesting a merge of the commit in #14 to M59, since removing user's cryptohome is essential to handle migration failures.
Labels: Merge-Approved-59
Project Member

Comment 17 by bugdroid1@chromium.org, Apr 28 2017

Labels: -merge-approved-59 merge-merged-3071
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fb47d19268582cdc5a16222901fc937472eb91be

commit fb47d19268582cdc5a16222901fc937472eb91be
Author: Naoki Fukino <fukino@chromium.org>
Date: Fri Apr 28 06:00:47 2017

cros: Remove user directory when encryption migration fails.

When the migration UI is notified by cryptohome that cryptohome gets unexpected
errors during migration and it gives up migration, we should remove the
user's cryptohome to make sure that the user will be able to log in to
the Desktop next time.

After the user's cryptohome is removed, we get password verification errors
on the existing user pod three times.
It seems an independent issue, so I filed  crbug.com/715474  and will look into separately.

BUG= 713556 
TEST=manually tested by forcing cryptohome fail during migration and confirming the cryptohome is gone.
TBR=xiyuan@chromium.org

Review-Url: https://codereview.chromium.org/2838303003
Cr-Commit-Position: refs/heads/master@{#467608}
(cherry picked from commit 1339ec6addf0faa53da2c7418a0de5b6803d89cf)

Review-Url: https://codereview.chromium.org/2851683002 .
Cr-Commit-Position: refs/branch-heads/3071@{#285}
Cr-Branched-From: a106f0abbf69dad349d4aaf4bcc4f5d376dd2377-refs/heads/master@{#464641}

[modify] https://crrev.com/fb47d19268582cdc5a16222901fc937472eb91be/chrome/browser/ui/webui/chromeos/login/encryption_migration_screen_handler.cc
[modify] https://crrev.com/fb47d19268582cdc5a16222901fc937472eb91be/chrome/browser/ui/webui/chromeos/login/encryption_migration_screen_handler.h

Status: Fixed (was: Started)
Project Member

Comment 19 by sheriffbot@chromium.org, Apr 29 2017

Labels: -Merge-Request-59 Hotlist-Merge-Approved Merge-Approved-59
Your change meets the bar and is auto-approved for M59. Please go ahead and merge the CL to branch 3071 manually. Please contact milestone owner if you have questions.
Owners: amineer@(Android), cmasso@(iOS), gkihumba@(ChromeOS), Abdul Syed@(Desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Project Member

Comment 20 by sheriffbot@chromium.org, May 2 2017

Cc: gkihumba@google.com
This issue has been approved for a merge. Please merge the fix to any appropriate branches as soon as possible!

If all merges have been completed, please remove any remaining Merge-Approved labels from this issue.

Thanks for your time! To disable nags, add the Disable-Nags label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Project Member

Comment 21 by sheriffbot@chromium.org, May 5 2017

This issue has been approved for a merge. Please merge the fix to any appropriate branches as soon as possible!

If all merges have been completed, please remove any remaining Merge-Approved labels from this issue.

Thanks for your time! To disable nags, add the Disable-Nags label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Owner: dspaid@chromium.org
Status: Assigned (was: Fixed)
cherry pick is missing.
Project Member

Comment 23 by bugdroid1@chromium.org, May 12 2017

Labels: merge-merged-release-R59-9460.B
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/66e748524a062efedd67b8901a468696ed45df88

commit 66e748524a062efedd67b8901a468696ed45df88
Author: Dan Spaid <dspaid@google.com>
Date: Fri May 12 09:04:09 2017

cryptohome: UnmountAll if crypto migration fails.

UnmountAll is currently called only when the dircrypto migration
succeeds.  For consistency it should also be called if the migration
fails.

BUG= chromium:713556 
TEST=cros_workon_make --board=samus --test cryptohome

Change-Id: I65c49f3e5abc727160b086ea5db19957318eda63
Reviewed-on: https://chromium-review.googlesource.com/484179
Commit-Ready: Dan Spaid <dspaid@chromium.org>
Tested-by: Dan Spaid <dspaid@chromium.org>
Reviewed-by: Ryo Hashimoto <hashimoto@chromium.org>
(cherry picked from commit 467df89c1c669f0fb875a6bf445076bcdc51cf55)
Reviewed-on: https://chromium-review.googlesource.com/503888
Reviewed-by: Dan Spaid <dspaid@chromium.org>
Commit-Queue: Junichi Uekawa <uekawa@chromium.org>
Tested-by: Junichi Uekawa <uekawa@chromium.org>
Trybot-Ready: Junichi Uekawa <uekawa@chromium.org>

[modify] https://crrev.com/66e748524a062efedd67b8901a468696ed45df88/cryptohome/mount.cc

Status: Fixed (was: Assigned)
is this fixed?
Yes
Labels: -Hotlist-Merge-Approved -Merge-Approved-59
Cc: dhadd...@chromium.org
Trying to verify this bug. Could you provide verification steps?

Comment 28 by dspaid@google.com, May 21 2017

If you have terminal access on the machine the easiest way is:
1. Flash an ecryptfs-based image (m58 should work for any board)
2. Login to some user (foo@gmail.com)
3. Update to image where migration should happen
4. without logging in, in a terminal run:
dd if=/dev/zero of=$(find/home/.sahdow/$(cryptohome --action=obfuscate_user --user=foo@gmail.com)/vault -type f | head -n1) bs=12k count=1
5. Attempt to log in and go through the migration flow.  It should finish with an error.
6. Click through the dialog to restart, verify that there is no migration the next time you login (because the old profile was removed).

Comment 29 by uekawa@google.com, May 21 2017

AFAIK #28 step 4 (generate a file with known I/O error) is not sufficient to fail a migration now.
you need something else to fail the migration.


It should still be valid in M59, which is probably what should be verified since this is targeting M59.  It is not valid on M60 dev, and it probably won't be valid on M59 beta after https://chromium-review.googlesource.com/c/499948/ is cherry-picked.

After that patch I don't think we've got any reproduceable way to verify this.
Updated a device (terra) having old build (M53) with single user account to M60 9592.3.0, 60.0.3112.10

Logged in to user account and migration started
Seen failure screen after migration
Clicked Restart button

Expected:
From the bug description, I expected that the user account will be removed.

Actual:
Device rebooted and user account is still there and not removed.

Attached screenshot which says "Unfortunately, you'll need to add your account to this Chromebook again"
migration_failure.JPG
2.8 MB View Download
Was the content of your profile (e.g. Downloads, android app contents, etc) still there?
We do keep the profile pod on the login screen, so if you just checked that you wouldn't see evidence of removal.

Also did you file a feedback report after the failure?
Device was on M53 before updating to M60. So I was not opted-in. 

Here is the feedback report link: http://feedback/#/Report/64071718276

Comment 35 by uekawa@google.com, Jun 2 2017

Cc: fukino@chromium.org
as an aside, wow, this feedback is NOT_FOUND.
 listxattr: /home/.shadow/ad44920984a00b2df6128fc3fcc97417578b469c/temporary_mount/user/GCache/v1/tmp/R8vt93/tmp.gdoc: No such file or directory
Failed to migrate "user/GCache/v1/tmp/R8vt93/tmp.gdoc"

Did you generate this intentionally or did this just happen? 





as for the main topic, what log should I be looking for when profile gets removed?
uekawa@ It just happened. Not sure about the file tmp.gdoc

As for where to check to see if the profile is removed:
It looks like we don't have any explicit logs, however if youcheck the dbus_details section of system_logs.txt you can see that AsyncRemove has been called at least once.

The not found error definitely seems to be a new/unknown bug, and it worries me.  

Comment 38 by uekawa@google.com, Jun 2 2017

forked off a bug for NOT_FOUND https://bugs.chromium.org/p/chromium/issues/detail?id=728892

Comment 39 by uekawa@google.com, Jun 2 2017

#31 from the login screen, the user account will look like it is still there but it is listed in the roster but will recreate the profile. At least that's what should happen, so I am curious for answers to questions in #33, #37, and possibly your logs after logging in again (which should have logs about recreating profile)

Comment 40 by dchan@chromium.org, Jan 22 2018

Status: Archived (was: Fixed)

Sign in to add a comment