Version 56.0.2924.87 (64-bit)
Platform 9000.82.0 (Official Build) stable-channel quawks
Firmware Google_Quawks.5216.204.61

Version 56.0.2924.87 (64-bit)
Plate-forme 9000.82.0 (Official Build) stable-channel quawks
Micrologiciel Google_Quawks.5216.204.61

CBC RS/TC forum topic https://goo.gl/nQxcYL
#CBC-RS/TC-watchlist 
With all the recent sign-in issues it would be really helpful if alt+shift+i worked from the sign in screen.

I've updated all the topics with this message. 

Right now the only fix is a powerwash. 
Support/help page https://support.google.com/chromebook/answer/183084?source=genius-rts

I  started a bug and you can help the devs by giving your device info. 692802 - Sign in bootloop after auto update to V56 - chromium - Monorail 

Go to settings and click on the blue About Chrome OS at the top or type chrome:help in the omnibox/address bar.

Copy and paste the three lines that look like this:
Version 56.0.2924.87 
Platform xxxxxxxx (Official Build) stable-channel xxxxxxxxx
Firmware Google_xxxxxxxxxx
 
Sign in at the top right of the issue site. You can star the issue to get email updates. Please, do not spam with comments like "me too". Give them the data.The devs may ask for more information.

steel@ there's not a lot of detail to go on here, but can someone on your team dig into this?

everyone else: If you encounter this issue or see someone who does, the single most valuable thing we can get is an ID from chrome://crashes.

comment#5
Sign in as a guest from the sign-in screen > type chrome://crashes in the address bar

Does reporting the issue with alt+shift+i from guest sign-in give you the logs as well?

dymphep@ sadly, no. For privacy reasons we can't couple feedback reports and crash IDs. Starting in 57 there's some limited correlation, but the actual IDs are far more useful.

Have you noticed any particular devices crashing more than others?

5 different devices from the topics, 2 squawks, the same candy both on Beta and Stable, 1 peach, terra, HP14

Version 56.0.2924.87
Platform 9000.82.0 (Official Build) stable-channel peach_pit
Firmware Google_Peach_Pit.4482.95.0

Verzija 56.0.2924.67 beta (64-bit)
Platforma 9000.66.0 (Official Build) beta-channel candy
Oprema Google_Candy.5216.310.57

Verzija 56.0.2924.87 (64-bit)
Platforma 9000.82.0 (Official Build) stable-channel candy
Oprema Google_Candy.5216.310.57

56.0.2924.87 (64-bit)
Platform 9000.82.0 (Official Build) stable-channel terra
ARC Version 3691121
Firmware Google_Terra.7287.154.56

Version 56.0.2924.87 (64-bit)
Plate-forme 9000.82.0 (Official Build) stable-channel quawks
Micrologiciel Google_Quawks.5216.204.61

 56.0.2924.87 (Official Build) (64-bit) my Chromebook HP 14

From https://bugs.chromium.org/p/chromium/issues/detail?id=690710

Comment 31 by pucchakayala@chromium.org, Today (22 hours ago)
⚐
Song & I were able to see this Profile error today when the devices are auto-updated from M56 [56.0.2924.79] --> M57 [57.0.2987.54] on Daisy & Candy devices.

Ok, doesn't sound device specific.  Powerwashing does seem to recover it, so I'm guessing something is running off the rails in version migration for some users.

Good call on noticing the profile error issue as well dymphep@. That looks like it only affected 57 though, so probably not the same root cause.  Adding sebsg@ though for any thoughts.

All we know for sure right now is that this isn't hitting most users, but those that ARE hit are getting really screwed by it.

The other serious crash we know of at the moment is bug 690517 (I'm trying to get it unrestricted). The symptoms sound different so I'm thinking these two aren't related.

Feedback reports might be helpful here as it doesn't seem to be able to grab a crash id. I'm going to look through the reports and see what I can find.

https://feedback.corp.google.com/product/208/neutron?lView=rd&lRSort=1&lROrder=2&lRFilter=1&lReportSearch=bootloop&lReport=51905315212 looks like it MIGHT be the same issue.

Interesting, it seems similar to the migration that happened in 57 beta but I don't think it could affect 56. The problem in my case should only affect people who had 57 beta during a short time span.

Since this is in the wild already we can't hold 56 refresh for it.  If we can find a fix we'll merge back if possible.

Verzija 56.0.2924.87 (64-bit)
Platforma 9000.82.0 (Official Build) stable-channel candy
Oprema Google_Candy.5216.310.57

My Chromebook restarts/boots everytime I try to login into my gmail account (while second user on this Chromebook has no similar problem). I tried factory reset, powerwash, also switched between administrator/standars user for this account, uninstalled all chrome plugins, and still the same. Firstly it was solved by switching from Beta to Stable mode, but the problem came back after a certain time. Then I did powerwash, it worked for few days, now it is happening again. Please note, the other user on this Chromebook has no this issue. Thanks

gandlar@ Thank you for the info. Can you go into the working account and check chrome://crashes and post any info from there here?  

Increasing reports of this.  Bumping up to Pri-0

Jacob, could you pick this up?

So far this is only affecting one account.  The student can be working and then it crashes, sometimes it goes black, then I push a key, then the screen pops up.  other times it fully crashes.  I need to hit the power button to turn it back on.  I have done the resyncing chrome, updating the browser, and it would still crash.  I have tried wiping the original student account, backing up his files, then recreating the same student account, and putting his files back in his drive.  The chromebook still crashes.  I am going to try this thing with the powerwash now, although Since I am enterprise managed, I don't think it does powerwash, it does something else.  will update

kkamelamela@: Can you submit a crash report when this happens and record the crash id here?

To anyone who can reproduce: are you able to sign into the account using a secondary user? ie, first sign into a different account, then open the status menu, click the user icon, and add the failing user to the existing session. See https://support.google.com/chromebook/answer/6088201?hl=en.

I've tried reproducing but haven't had luck.

I took a look at the feedback in comment #11 but it was from a device running 55.0.2883.105. The only things I noticed was that there was an extremely large number of unreported crashes (we stop recording at 32 dumps). The sign-in also happened offline.

Since this is a stable update the change list is pretty huge (git log 55.0.2883.105..56.0.2924.67). I see there were a number of active directory changes - rsorokin@, is it possible these are related?

WE've been having the issue with our Dell 1320 chromebooks.  Not all users experience going to a black screen and a reboot.  The fix seems to be this:

1.  Powerwash the device back to chrome 54
2.  Move the device to an ou with a limit of the chrome os of 55
3.  login with an account that doesn't have extensions or data in it and update to chrome 55
4.  reboot the device login with test account and verify version 55
5.  move the device to its original ou
6.  reboot device and login with test account
7.  update to chrome 56

On the next reboot the user that was having the issue should be able to login to the device again.  The issue seems to be when you jump chrome versions 54 -->56 every time I powerwash and go from 54 --> 56 I can replicate the issue.  On a gradual upgrade it seems to work fine.  

Re #19: Active Directory changes seems harmless. But it's hard to tell - I failed to reproduce the bug.

I've tried reproducing by updating directly from 54.0.2840.44 to 56.0.2924.110, but I did not encounter any crashes. I've attempted with both managed and regular accounts.

Re #20: Can you try following the steps in comment #19? A crash report would be extremely helpful here. (See chrome://crashes).

We are seeing this issue as well.  On the most current as well as several previous OS builds. It impacts unmanaged chromebooks on our gsuite domain.  In other words, if you are a domain account-holder on a managed device, there is no issue.  But if you're a domain account-holder logging into the domain's gsuite on your PERSONAL or another UNMANAGED device, this problem reliably recurs.  Our workaround, for now, is to have the sync her chrome sync on the dashboard and then powerwash the device.

Re #23: I'm unable to reproduce on 56.0.2924.110 - what version are you on? Does this happen with any user in the domain? Can you file a feedback report or upload crashes?

I've found a feedback report, but it was reported from a guest session so there are no chrome logs. However, I do see this in the logs:


2017-03-01T11:07:29.312710-05:00 WARNING crash_reporter[10307]: Received crash notification for chrome[10245] user 1000 (called directly)
2017-03-01T11:07:29.314507-05:00 WARNING crash_reporter[10307]: Crash directory /home/user/be8f731ff029b7f5e82831ddbcfc559a2babff72/crash already full with 32 pending reports
2017-03-01T11:07:29.314537-05:00 ERR crash_reporter[10307]: Can't create crash directory for uid 1000
2017-03-01T11:07:29.321065-05:00 INFO kernel: [11359.137253] traps: chrome[10245] trap invalid opcode ip:7f0007c0a61c sp:7ffe401ddf10 error:0 in chrome[7f0003e90000+6dc7000]
2017-03-01T11:07:29.336421-05:00 WARNING crash_reporter[10308]: [user] Received crash notification for chrome[10245] sig 4, user 1000 (ignoring call by kernel - chrome crash; waiting for chrome to call us directly)
2017-03-01T11:07:29.426548-05:00 INFO session_manager[9900]: [INFO:child_exit_handler.cc(77)] Handling 10245 exit.
2017-03-01T11:07:29.426581-05:00 ERR session_manager[9900]: [ERROR:child_exit_handler.cc(85)]   Exited with signal 4
2017-03-01T11:07:29.426881-05:00 INFO session_manager[9900]: [INFO:session_manager_service.cc(298)] Exiting process is chrome.
2017-03-01T11:07:29.427043-05:00 INFO session_manager[9900]: [INFO:browser_job.cc(116)] Terminating process group: Ensuring browser processes are gone.
2017-03-01T11:07:29.427193-05:00 INFO session_manager[9900]: [INFO:system_utils_impl.cc(110)] Sending 9 to -10245 as 1000


The "traps: chrome[10245] trap invalid opcode ip:7f0007c0a61c sp:7ffe401ddf10 error:0 in chrome[7f0003e90000+6dc7000]" is very suspicious. I'm going to disassemble chrome and see what the instructions around there look like.

Assuming I figured things out right, here's the disassembly of 7f0007c0a61c (AKA 0x3d7a61c from the start of Chrome):

===

(gdb) disass 0x3d7a61c
Dump of assembler code for function syncer::Ordinal<syncer::StringOrdinalTraits>::CreateBetween(syncer::Ordinal<syncer::StringOrdinalTraits> const&) const:
   0x0000000003d7a5a0 <+0>:     push   %rbp
   0x0000000003d7a5a1 <+1>:     mov    %rsp,%rbp
   0x0000000003d7a5a4 <+4>:     push   %r13
   0x0000000003d7a5a6 <+6>:     push   %r12
   0x0000000003d7a5a8 <+8>:     push   %rbx
   0x0000000003d7a5a9 <+9>:     sub    $0x8,%rsp
   0x0000000003d7a5ad <+13>:    cmpb   $0x0,0x8(%rsi)
   0x0000000003d7a5b1 <+17>:    je     0x3d7a61c <syncer::Ordinal<syncer::StringOrdinalTraits>::CreateBetween(syncer::Ordinal<syncer::StringOrdinalTraits> const&) const+124>
   0x0000000003d7a5b3 <+19>:    cmpb   $0x0,0x8(%rdx)
   0x0000000003d7a5b7 <+23>:    je     0x3d7a61c <syncer::Ordinal<syncer::StringOrdinalTraits>::CreateBetween(syncer::Ordinal<syncer::StringOrdinalTraits> const&) const+124>
   0x0000000003d7a5b9 <+25>:    mov    %rsi,%rbx
   0x0000000003d7a5bc <+28>:    mov    %rdi,%r12
   0x0000000003d7a5bf <+31>:    mov    %rdx,%rsi
   0x0000000003d7a5c2 <+34>:    mov    %rbx,%rdi
   0x0000000003d7a5c5 <+37>:    mov    %rdx,%r13
   0x0000000003d7a5c8 <+40>:    callq  0x3d78a90 <syncer::Ordinal<syncer::StringOrdinalTraits>::Equals(syncer::Ordinal<syncer::StringOrdinalTraits> const&) const>
   0x0000000003d7a5cd <+45>:    test   %al,%al
   0x0000000003d7a5cf <+47>:    jne    0x3d7a61c <syncer::Ordinal<syncer::StringOrdinalTraits>::CreateBetween(syncer::Ordinal<syncer::StringOrdinalTraits> const&) const+124>
   0x0000000003d7a5d1 <+49>:    mov    %r13,%rsi
   0x0000000003d7a5d4 <+52>:    mov    %rbx,%rdi
   0x0000000003d7a5d7 <+55>:    callq  0x3d79020 <syncer::Ordinal<syncer::StringOrdinalTraits>::LessThan(syncer::Ordinal<syncer::StringOrdinalTraits> const&) const>
   0x0000000003d7a5dc <+60>:    test   %al,%al
   0x0000000003d7a5de <+62>:    jne    0x3d7a600 <syncer::Ordinal<syncer::StringOrdinalTraits>::CreateBetween(syncer::Ordinal<syncer::StringOrdinalTraits> const&) const+96>
   0x0000000003d7a5e0 <+64>:    mov    %rbx,%rdx
   0x0000000003d7a5e3 <+67>:    mov    %r13,%rsi
   0x0000000003d7a5e6 <+70>:    mov    %r12,%rdi
   0x0000000003d7a5e9 <+73>:    callq  0x3d7a410 <syncer::Ordinal<syncer::StringOrdinalTraits>::CreateOrdinalBetween(syncer::Ordinal<syncer::StringOrdinalTraits> const&, syncer::Ordinal<syncer::StringOrdinalTraits> const&)>
   0x0000000003d7a5ee <+78>:    add    $0x8,%rsp
   0x0000000003d7a5f2 <+82>:    mov    %r12,%rax
   0x0000000003d7a5f5 <+85>:    pop    %rbx
   0x0000000003d7a5f6 <+86>:    pop    %r12
   0x0000000003d7a5f8 <+88>:    pop    %r13
   0x0000000003d7a5fa <+90>:    pop    %rbp
   0x0000000003d7a5fb <+91>:    retq   
   0x0000000003d7a5fc <+92>:    nopl   0x0(%rax)
   0x0000000003d7a600 <+96>:    mov    %r13,%rdx
   0x0000000003d7a603 <+99>:    mov    %rbx,%rsi
   0x0000000003d7a606 <+102>:   mov    %r12,%rdi
   0x0000000003d7a609 <+105>:   callq  0x3d7a410 <syncer::Ordinal<syncer::StringOrdinalTraits>::CreateOrdinalBetween(syncer::Ordinal<syncer::StringOrdinalTraits> const&, syncer::Ordinal<syncer::StringOrdinalTraits> const&)>
   0x0000000003d7a60e <+110>:   add    $0x8,%rsp
   0x0000000003d7a612 <+114>:   mov    %r12,%rax
   0x0000000003d7a615 <+117>:   pop    %rbx
   0x0000000003d7a616 <+118>:   pop    %r12
   0x0000000003d7a618 <+120>:   pop    %r13
   0x0000000003d7a61a <+122>:   pop    %rbp
   0x0000000003d7a61b <+123>:   retq   
   0x0000000003d7a61c <+124>:   ud2    
End of assembler dump.

===

I see a lot of jumps to the illegal instruction, so my first guess from the code search (https://cs.chromium.org/chromium/src/components/sync/base/ordinal.h?type=cs&q=CreateBetween+package:%5Echromium$&l=298) is that we're hitting one of the CHECKs here?

Ordinal<Traits> Ordinal<Traits>::CreateBetween(const Ordinal& other) const {
  CHECK(IsValid());
  CHECK(other.IsValid());
  CHECK(!Equals(other));


...but I could have got that wrong.

BTW: my magic for getting the disassembly was based on stuff I figured out in b/35647846 (there I combined with a minidump):

--

From goldeneye, grab:

quawks/9000.91.0/ChromeOS-test-R56-9000.91.0-quawks.tar.xz
quawks/9000.91.0/debug-quawks.tgz

--

Untar somewhere chroot accessible:

cd /b/tip/tmp/quawks
tar xf ~/Downloads/stable-channel%2Fquawks%2F9000.91.0%2FChromeOS-test-R56-9000.91.0-quawks.tar.xz
tar xf ~/Downloads/stable-channel%2Fquawks%2F9000.91.0%2Fdebug-quawks.tgz

--

In chroot, get ready:

cd /b/tip/tmp/quawks
START=$(($(cgpt show chromiumos_test_image.bin  | grep ROOT-A | awk '{print $1};') * 512)) 
mkdir r 
sudo mount -oro -oloop,offset=${START} chromiumos_test_image.bin r 

--

Start gdb:

x86_64-cros-linux-gnu-gdb

--

Inside gdb, get ready:

set sysroot r 
set debug-file-directory debug/ 
file r/opt/google/chrome/chrome 

--

Disassemble:

disass 0x3d7a61c

I suspect CL https://codereview.chromium.org/2614803002 is causing the crash based on some crash reports involving CreateBetween symbol (go/cyyry).

Passing to khmel@ who is more familiar with that code.

Example feedback report where this happens: go/oyjrk (no associated crashes yet - usually crashes do not get uploaded since the user can never sign in).

Adding this CBC topic. The very helpful user was able to sign in once after a powerwash. He's trying again and will try to send feedback.

https://productforums.google.com/forum/?utm_medium=email&utm_source=footer#!msg/chromebook-central/v5ko0gN8tAE/oBYxLQeVEAAJ

CL: crrev.com/2732633003.

This was hidden legacy problem. With CL mentioned in #27 it comes to surface.

crrev.com/2732633003 in CQ. Requesting to merge to M57/M58. Once merged I will track crash reports to make sure that problem is gone.

Approving merge to M57 Chrome OS.

khmel@ please merge by eod today if possible.

CQ is flaky (windows builder). I am on it

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/11f9ee62d5cb9fbc3d3f1458e145b06be67f78ae

commit 11f9ee62d5cb9fbc3d3f1458e145b06be67f78ae
Author: khmel <khmel@chromium.org>
Date: Tue Mar 07 20:20:53 2017

arc: Handle position conflict in app list items.

It is not guaranteed that sync items have unique positions. This may
cause crash in case web store app has conflicting pos with some other
app.

TEST=Unit test added
BUG= 692802 

Review-Url: https://codereview.chromium.org/2732633003
Cr-Commit-Position: refs/heads/master@{#455195}

[modify] https://crrev.com/11f9ee62d5cb9fbc3d3f1458e145b06be67f78ae/chrome/browser/ui/app_list/app_list_syncable_service.cc
[modify] https://crrev.com/11f9ee62d5cb9fbc3d3f1458e145b06be67f78ae/chrome/browser/ui/app_list/app_list_syncable_service.h
[add] https://crrev.com/11f9ee62d5cb9fbc3d3f1458e145b06be67f78ae/chrome/browser/ui/app_list/app_list_syncable_service_unittest.cc
[modify] https://crrev.com/11f9ee62d5cb9fbc3d3f1458e145b06be67f78ae/chrome/test/BUILD.gn

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/da3c6b99047306619e66e5a369c5e7ba8ae057a0

commit da3c6b99047306619e66e5a369c5e7ba8ae057a0
Author: khmel <khmel@chromium.org>
Date: Tue Mar 07 20:36:53 2017

[Merge M57] arc: Handle position conflict in app list items.

It is not guaranteed that sync items have unique positions. This may
cause crash in case web store app has conflicting pos with some other
app.

TEST=Unit test added
BUG= 692802 

NOTRY=true
NOPRESUBMIT=true
TBR=stevenjb@chromium.org

Review-Url: https://codereview.chromium.org/2732633003
Cr-Commit-Position: refs/heads/master@{#455195}
(cherry picked from commit 11f9ee62d5cb9fbc3d3f1458e145b06be67f78ae)

Review-Url: https://codereview.chromium.org/2732333002
Cr-Commit-Position: refs/branch-heads/2987@{#788}
Cr-Branched-From: ad51088c0e8776e8dcd963dbe752c4035ba6dab6-refs/heads/master@{#444943}

[modify] https://crrev.com/da3c6b99047306619e66e5a369c5e7ba8ae057a0/chrome/browser/ui/app_list/app_list_syncable_service.cc
[modify] https://crrev.com/da3c6b99047306619e66e5a369c5e7ba8ae057a0/chrome/browser/ui/app_list/app_list_syncable_service.h
[add] https://crrev.com/da3c6b99047306619e66e5a369c5e7ba8ae057a0/chrome/browser/ui/app_list/app_list_syncable_service_unittest.cc
[modify] https://crrev.com/da3c6b99047306619e66e5a369c5e7ba8ae057a0/chrome/test/BUILD.gn

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/651a2d0a721ad7779451275b595ba3ad30d3a3c5

commit 651a2d0a721ad7779451275b595ba3ad30d3a3c5
Author: khmel <khmel@chromium.org>
Date: Tue Mar 07 20:42:28 2017

[Merge M58] arc: Handle position conflict in app list items.

It is not guaranteed that sync items have unique positions. This may
cause crash in case web store app has conflicting pos with some other
app.

TEST=Unit test added
BUG= 692802 

NOTRY=true
NOPRESUBMIT=true
TBR=stevenjb@chromium.org

Review-Url: https://codereview.chromium.org/2732633003
Cr-Commit-Position: refs/heads/master@{#455195}
(cherry picked from commit 11f9ee62d5cb9fbc3d3f1458e145b06be67f78ae)

Review-Url: https://codereview.chromium.org/2740603002
Cr-Commit-Position: refs/branch-heads/3029@{#51}
Cr-Branched-From: 939b32ee5ba05c396eef3fd992822fcca9a2e262-refs/heads/master@{#454471}

[modify] https://crrev.com/651a2d0a721ad7779451275b595ba3ad30d3a3c5/chrome/browser/ui/app_list/app_list_syncable_service.cc
[modify] https://crrev.com/651a2d0a721ad7779451275b595ba3ad30d3a3c5/chrome/browser/ui/app_list/app_list_syncable_service.h
[add] https://crrev.com/651a2d0a721ad7779451275b595ba3ad30d3a3c5/chrome/browser/ui/app_list/app_list_syncable_service_unittest.cc
[modify] https://crrev.com/651a2d0a721ad7779451275b595ba3ad30d3a3c5/chrome/test/BUILD.gn

Merged, I will keep this opened for a while to make sure that crash does not happen in new build.

crash info: ID rušenja ChromeOS (ID poslužitelja: d20e891940000000)

Version 56.0.2924.110
Platform 9000.91.0 (Official Build) stable-channel veyron_minnie
ARC Version 3756504
Firmware Google_Veyron_Minnie.6588.237.0

Hadn't run updates last month. Just ran updates this morning, now having the same issue (login screen looping back) as the others reported mid-February.

Have not yet run the powerwash steps to recover. Version information above taken from Guest login. 

CBC OP:
1. entering my password the spinning wheel is jerky and not smooth, the screen goes black and flashes briefly a fault page and goes black again.

2. I can log in as a guest, this works if I enter email details but I cannot return and log in my normal way.

Version 57.0.2987.123 (64-bit)
Platform 9202.56.1 (Official Build) stable-channel falco
Firmware Google_Falco.4389.92.0

https://productforums.google.com/forum/#!topic/chromebook-central/EPa_oKpDnvc;context-place=forum/chromebook-central

not reproduced on minnie M58 9334.69.0, 58.0.3029.112. Please reopen if its reproduced later builds.

9592.96.0 (Official Build) stable-channel veyron_speedy
Firmware
Google_Veyron_Speedy.6588.237.0
Channel
Currently on stable
CHANGE CHANNEL
ARC Version
4340384
Blink
537.36 (@)
V8
6.0.286.57

Has reoccurred for me on 66.0.3359.67 beta 64-bit on Acer 14

might be crbug.com/824630