New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 779567 link

Starred by 2 users
Status: WontFix
Owner:
█ uekawa@chromium.org
Last visit > 30 days ago
Closed: Jan 2018
Cc:
xiy...@chromium.org
gwendal@chromium.org
█ bhthompson@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug

Blocked on:
issue 761911
issue 773474


Sign in to add a comment

Login failure on guado with 62 and ext4

Project Member Reported by jayhlee@google.com, Oct 30 2017 
Chrome Version: 62.0.3202.74
Chrome OS Version: 9901.54.0 (Official Build) beta-channel guado

Steps To Reproduce:
(1) Logged into corp profile (worked).
(2) Attempt to login to personal account as secondary profile on device (failed).
(3) Logout and attempt to login with personal account as primary (failed).

Expected Result:
Login success for personal profile (profile was logged in and working on Friday).

Actual Result:

Logs (attached) show:

[6153:6153:1030/102937.468270:ERROR:device_event_log_impl.cc(156)] [10:29:37.468] Login: homedir_methods.cc:336 HomedirMethods MountEx error (CryptohomeErrorCode): 1
[6153:6153:1030/102937.468366:ERROR:device_event_log_impl.cc(156)] [10:29:37.468] Login: cryptohome_authenticator.cc:931 Cryptohome failure: state(AuthState)=2, code(cryptohome::MountError)=32
[6153:6153:1030/102937.468384:VERBOSE1:cryptohome_authenticator.cc(774)] Resolved state to: 2
[6153:6153:1030/102937.468453:ERROR:device_event_log_impl.cc(156)] [10:29:37.468] Login: cryptohome_authenticator.cc:707 Login failed: Could not mount cryptohome.
[6153:6153:1030/102937.468500:ERROR:login_performer.cc(63)] Login failure, reason=1, error.state=0
[6153:6153:1030/102937.468608:VERBOSE1:existing_user_controller.cc(1431)] Could not mount cryptohome.
[6153:6153:1030/102937.468629:VERBOSE1:webui_login_display.cc(109)] Show error, error_id: 6393, attempts:1, help_topic_id: 188036
[6153:6153:1030/102937.468660:ERROR:core_oobe_handler.cc(194)] CoreOobeHandler::ShowSignInError: error_text=Sorry, your password could not be verified. Please try again.

Password is correct. On Friday device was logged in, locked and working fine. Over weekend, building may have lost power (NYC office-wide maintenance). Login worked fine Mon. morning to corp profile but gave bad password on personal. Logged out and retried with known-correct password. After 3-4 failures was pushed to full GAIA online flow. Password and security key were accepted but then got thrown back to username entry.

How frequently does this problem reproduce? (Always, sometimes, hard to
reproduce?)
10/10

What is the impact to the user, and is there a workaround? If so, what is
it?
Possible ext4 profile corruption? User is effectively DoS on login attempts (profile removal being non-obvious workaround).

Feedback submitted from corp profile after personal login failures. Full logs are also at:

https://drive.google.com/a/google.com/file/d/1usN030gVMTUNNH8XjugIX8e9ssGBD6zJ/view?usp=sharing

This could very well be a hardware issue with my guado but I've not seen issues like it before, marking as RBS in case it's larger issue.

Comment 1 by bhthompson@google.com, Oct 31 2017

Cc: xiy...@chromium.org bhthompson@chromium.org
Labels: -ReleaseBlock-Stable
Owner: uekawa@chromium.org
Junichi, any chance this is ext4 related? We should not be migrating this one actively, so I would not have thought so.

Chromeboxes are known to have issues when power cycled in certain cases, specifically if they have a Kingston SSD, so this might be related to that. 

If we have reason to believe this is seen past this particular Chromebox we can block stable, otherwise this is just another bug.

Comment 2 by xiy...@chromium.org, Oct 31 2017

Cc: gwendal@chromium.org
Looks like there is a power loss over the weekend (10/27 - 10/30) and causes ext4 corruption on cryptohome files.

eventlog.txt:
180 | 2017-10-26 11:14:54 | System Reset
181 | 2017-10-26 11:14:54 | Wake Source | GPIO | 27
182 | 2017-10-28 01:45:52 | System boot | 144
183 | 2017-10-28 01:45:52 | SUS Power Fail
184 | 2017-10-28 01:45:52 | System Reset
185 | 2017-10-28 01:45:52 | ACPI Wake | S5
186 | 2017-10-28 01:45:52 | Wake Source | Power Button | 0
187 | 2017-10-28 02:15:56 | ACPI Enter | S3
188 | 2017-10-30 10:20:25 | ACPI Wake | S3
189 | 2017-10-30 10:20:25 | Wake Source | Power Button | 0
190 | 2017-10-30 10:20:25 | Wake Source | GPIO | 27

The device lost power on 10/27 and regain power on 10/28.

2017-10-27T23:15:08.541561-04:00 INFO kernel: [129717.569246] tpm_tis tpm_tis: command 0x65 (size 22) returned code 0x0
2017-10-28T01:45:55.721467-04:00 INFO kernel: [    0.000000] Initializing cgroup subsys cpuset

In the boot on 10/28, ext4 performed recover:
2017-10-28T01:45:55.723829-04:00 INFO kernel: [    2.011109] EXT4-fs (sda1): recovery complete
...
2017-10-28T01:45:55.723844-04:00 INFO kernel: [    2.263111] EXT4-fs (dm-1): recovery complete

Then on 10/30, cryptohome mount started to fail:
2017-10-30T10:29:02.215961-04:00 CRIT kernel: [ 2324.237006] EXT4-fs error (device sda1): ext4_iget:4229: inode #132518: comm MountThread: bogus i_mode (0)
2017-10-30T10:29:02.216961-04:00 CRIT kernel: [ 2324.237630] EXT4-fs error (device sda1): ext4_iget:4229: inode #132518: comm MountThread: bogus i_mode (0)
2017-10-30T10:29:02.243043-04:00 INFO kernel: [ 2324.263213] tpm_tis tpm_tis: command 0x15 (size 14) returned code 0x0
2017-10-30T10:29:02.243983-04:00 CRIT kernel: [ 2324.264523] EXT4-fs error (device sda1): ext4_iget:4229: inode #132518: comm MountThread: bogus i_mode (0)
2017-10-30T10:29:02.244000-04:00 CRIT kernel: [ 2324.264904] EXT4-fs error (device sda1): ext4_lookup:1593: inode #524291: comm MountThread: deleted inode referenced: 132516
2017-10-30T10:29:02.244004-04:00 CRIT kernel: [ 2324.265061] EXT4-fs error (device sda1): ext4_lookup:1593: inode #524291: comm MountThread: deleted inode referenced: 132516
2017-10-30T10:29:02.244307-04:00 ERR cryptohomed[2183]: Can't create: /home/root/4a43e1553cb80cdb560e61a03dc95e1a5323c032: Input/output error
2017-10-30T10:29:02.244347-04:00 ERR cryptohomed[2183]: Couldn't ensure root path: /home/root/4a43e1553cb80cdb560e61a03dc95e1a5323c032
2017-10-30T10:29:02.244369-04:00 ERR cryptohomed[2183]: Error creating cryptohome.
2017-10-30T10:29:02.244551-04:00 WARNING cryptohomed[2183]: PKCS#11 initialization requested but cryptohome is not mounted.
2017-10-30T10:29:02.245126-04:00 CRIT kernel: [ 2324.265163] EXT4-fs error (device sda1): ext4_lookup:1593: inode #524291: comm MountThread: deleted inode referenced: 132516
2017-10-30T10:29:02.245136-04:00 CRIT kernel: [ 2324.265257] EXT4-fs error (device sda1): ext4_lookup:1593: inode #524291: comm MountThread: deleted inode referenced: 132516

Comment 3 by uekawa@google.com, Oct 31 2017

Blockedon: 761911
Labels: ArcExt4Migration
Yeah I think the file system is corrupted.
I wonder if we were better off with fsck, related bug would be crbug.com/761911

Comment 4 by uekawa@google.com, Oct 31 2017

Labels: -ArcExt4Migration

Comment 5 by uekawa@google.com, Oct 31 2017 

Currently the only way to make the file system back to a sane state is to do a powerwash, AFAIK otherwise the corrupted file system will be used forever without even a fsck.

Comment 6 by uekawa@google.com, Nov 8 2017

Blockedon: 773474
might it be a imageloader issue ?

Comment 7 by uekawa@google.com, Jan 22 2018

Status: WontFix (was: Unconfirmed)
I think unfortunately we can't do anything about this device and one needs to powerwash once in this state.
►

Sign in to add a comment