New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 871759 link

Starred by 6 users

Issue metadata

Status: Verified
Owner:
Last visit > 30 days ago
Closed: Oct 10
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

Switch channel shows rollback notification

Project Member Reported by hunyadym@chromium.org, Aug 7

Issue description

Chrome Version: 69.0.3497.2
OS: Chrome OS

What steps will reproduce the problem?
(1) Switch to dev M69
(2) Switch to stable channel

What is the expected result? You don't see the rollback notification, just for powerwash.

What happens instead? You see a notification "Device will be rolled back. Your administrator is rolling back your device. All data will be deleted when the device is restarted." Which is not true (device is not even enrolled).


 
I'm also intermittently seeing this notification on canary (70.0.3519.3) on Caroline, though it doesn't seem that there's any update available or channel changing happening, also not enterprise enrolled.

@Zen: The device is on my desk if you'd like to take a look.
Hi, I cannot reproduce the canary issue. If you see this again, can you please comment the Chrome OS platform version number from the login screen top right corner (currently 10991.0.0-18.08.22 for me) before and after update?
Project Member

Comment 3 by bugdroid1@chromium.org, Aug 28

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c7439ad965458aab49ec47cc8181856c623e3fd9

commit c7439ad965458aab49ec47cc8181856c623e3fd9
Author: Marton Hunyady <hunyadym@chromium.org>
Date: Tue Aug 28 15:55:16 2018

Only show rollback notification if we're not doing channel switch.

This fixes the issue that we're showing a misleading "Your
administrator rolled back this device" notification on non-enrolled
devices too if they switch the channel to a more stable version.

Bug:  871759 
Change-Id: I75e9e3faacaed0499fedef1aafda290cf0b87e19
Reviewed-on: https://chromium-review.googlesource.com/1188665
Reviewed-by: Greg Thompson <grt@chromium.org>
Commit-Queue: Marton Hunyady <hunyadym@chromium.org>
Cr-Commit-Position: refs/heads/master@{#586723}
[modify] https://crrev.com/c7439ad965458aab49ec47cc8181856c623e3fd9/chrome/browser/upgrade_detector/upgrade_detector.cc
[modify] https://crrev.com/c7439ad965458aab49ec47cc8181856c623e3fd9/chrome/browser/upgrade_detector/upgrade_detector.h
[modify] https://crrev.com/c7439ad965458aab49ec47cc8181856c623e3fd9/chrome/browser/upgrade_detector/upgrade_detector_chromeos.cc

Project Member

Comment 4 by bugdroid1@chromium.org, Aug 28

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8f26669fb2113ff56e033fc6604d92dc7c1b803f

commit 8f26669fb2113ff56e033fc6604d92dc7c1b803f
Author: Marton Hunyady <hunyadym@chromium.org>
Date: Tue Aug 28 16:00:54 2018

Add histogram for the reason of the rollback

Send a histogram value from UpgradeDetector when the device did
install a rollback image and shows a notification before reboot.

The metric records whether the rollback is happening because of a
channel switch or because of an admin-initiated enterprise
rollback.

Bug:  871759 
Change-Id: I38fad1792ba87b6bdbd788b3cd0f736dfa2d363c
Reviewed-on: https://chromium-review.googlesource.com/1190703
Commit-Queue: Marton Hunyady <hunyadym@chromium.org>
Reviewed-by: Ilya Sherman <isherman@chromium.org>
Reviewed-by: Greg Thompson <grt@chromium.org>
Cr-Commit-Position: refs/heads/master@{#586724}
[modify] https://crrev.com/8f26669fb2113ff56e033fc6604d92dc7c1b803f/chrome/browser/upgrade_detector/upgrade_detector_chromeos.cc
[modify] https://crrev.com/8f26669fb2113ff56e033fc6604d92dc7c1b803f/tools/metrics/histograms/enums.xml
[modify] https://crrev.com/8f26669fb2113ff56e033fc6604d92dc7c1b803f/tools/metrics/histograms/histograms.xml

Project Member

Comment 5 by bugdroid1@chromium.org, Aug 28

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ae7f92ce622419469843461e2fd2c4014eae7823

commit ae7f92ce622419469843461e2fd2c4014eae7823
Author: Marton Hunyady <hunyadym@chromium.org>
Date: Tue Aug 28 17:01:29 2018

Add logging when a rollback or powerwash notification is shown

Add warning level logging when:
- It's detected that update_engine is downloading a rollback
  image or switching channel.
- Notification about the rollback or channel switch to a more
  stable channel is shown.

Bug:  871759 
Change-Id: I930b7a23def82ff239ec8da4feeff9daaf960b37
Reviewed-on: https://chromium-review.googlesource.com/1190882
Commit-Queue: Marton Hunyady <hunyadym@chromium.org>
Reviewed-by: Xiyuan Xia <xiyuan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#586754}
[modify] https://crrev.com/ae7f92ce622419469843461e2fd2c4014eae7823/chrome/browser/upgrade_detector/upgrade_detector_chromeos.cc
[modify] https://crrev.com/ae7f92ce622419469843461e2fd2c4014eae7823/chromeos/dbus/update_engine_client.cc

Status: Fixed (was: Untriaged)
Marking this as fixed, since the original issue (channel switch shows rollback notification) should be fixed with this.

I still wasn't able to reproduce the issue in #1 (random rollback notifications), it might be fixed with this change. If anyone sees this problem in a version containing these changes, please contact me, the additional logging should help investigating this (and the metrics to check how big the problem is).
Hi,

Is this bug by any chance affecting Meowth devices as well? I just received a bug for the latest OS version 70.0.3532.8 on dev channel for a Nocturne device. http://b/113704345 - Let me know if you need more info. 

Thanks!
-Neha
Yes, this is the same issue. The fixes (and the additional logging in case this is not fixed yet) landed in 70.0.3536.0.
Status: Assigned (was: Fixed)
Unfortunately, I still see this happening in UMA, mostly on Eves: https://uma.googleplex.com/p/chrome/histograms?endDate=latest&dayCount=7&histograms=UpgradeDetector.RollbackReason&fixupData=true&showMax=true&filters=isofficial%2Ceq%2CTrue&implicitFilters=isofficial

If anyone can repro it in 70.0.3536 or later, please contact me to gather logs. Thanks!
Project Member

Comment 10 by bugdroid1@chromium.org, Sep 5

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1b8f9b7ec826a357af2b75ec03723c667e9781b6

commit 1b8f9b7ec826a357af2b75ec03723c667e9781b6
Author: Marton Hunyady <hunyadym@chromium.org>
Date: Wed Sep 05 10:04:49 2018

Add canary channel to the list of Chrome OS channels

This is needed to correctly identify when we are moving to a more
stable channel.

Bug:  871759 
Change-Id: Ia159fc133356dd2850378e54132aa386b8ce7930
Reviewed-on: https://chromium-review.googlesource.com/1188569
Reviewed-by: Ryo Hashimoto <hashimoto@chromium.org>
Commit-Queue: Marton Hunyady <hunyadym@chromium.org>
Cr-Commit-Position: refs/heads/master@{#588812}
[modify] https://crrev.com/1b8f9b7ec826a357af2b75ec03723c667e9781b6/chromeos/BUILD.gn
[modify] https://crrev.com/1b8f9b7ec826a357af2b75ec03723c667e9781b6/chromeos/dbus/update_engine_client.cc
[add] https://crrev.com/1b8f9b7ec826a357af2b75ec03723c667e9781b6/chromeos/dbus/update_engine_client_unittest.cc

The UMA values might be because of channel switch from canary to a more stable one (where a powerwash will actually happen). No reports from dev channel yet, only from canary.

Canary fix landed in 71.0.3544.0, after this we shouldn't see notifications on canary either.
Ran into this bug on an unmanaged Swanky on M70 beta. Upon clicking on the "Restart and reset" button, the device restarted with no issues and brough me to the login page with no modification to user profiles.

Sent feedback at about 22:09 ET with 'CRBUG: 871759' in first line and '#wtlm-feedback' to help find it with ease.
Cc: willland...@gmail.com
Thanks, willlandrieu, for the bug report!

Unfortunately, the logs don't contain the additional logging I added when showing this message.
The Chrome logs start at 21:01:28 and stop at 21:56:21. The feedback report was indeed sent at 22:09 EDT. Do you remember when did the notification started showing?

Can you please attach the logs corresponding to that time from the file:///var/log/chrome/ and the file:///var/log/update_engine/ directory? (Type this to the URL bar, the interesting one is the latest one where last modified is later than the time when the issue occurred.)

Thank you!
Not sure which files you needed, so here are the contents of the directories you requested:

https://drive.google.com/drive/folders/127z7ydxFdY3SkT73IkZczoN9VhLUxOHw?usp=sharing
Great, thanks a lot!

Can you please share file:///home/chronos/user/log/ too?

Since this might contain user data, you can also share/email it directly to me (hunyadym@chromium.org), or grep the data for "upgrade" and "update".

Thanks!
hunyadym, sent the shareable link to your email.
Thank you!

Based on your logs, the notification showed up directly after a Chrome crash.

[30237:30237:1007/205930.588244:WARNING:upgrade_detector_chromeos.cc(235)] Device is rolling back, will require powerwash. Reason: 0, current_channel: beta-channel, target_channel: beta-channel

After crashing Chrome multiple times, I could reproduce the issue, the fix is under review.
Project Member

Comment 19 by bugdroid1@chromium.org, Oct 9

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/272b638008f5d585dd02ec218674a73059e9fc1d

commit 272b638008f5d585dd02ec218674a73059e9fc1d
Author: Marton Hunyady <hunyadym@chromium.org>
Date: Tue Oct 09 11:32:54 2018

Fix incorrect rollback notifications

|status.is_rollback| was left uninitialized when Chrome crashed.
This CL is initializing it as false. A follow-up CL is setting it
properly based on the version numbers.

Bug:  871759 
Change-Id: I1d705e7a406547e6463c310cb811b166deab5cff
Reviewed-on: https://chromium-review.googlesource.com/c/1269534
Reviewed-by: Achuith Bhandarkar <achuith@chromium.org>
Commit-Queue: Marton Hunyady <hunyadym@chromium.org>
Cr-Commit-Position: refs/heads/master@{#597880}
[modify] https://crrev.com/272b638008f5d585dd02ec218674a73059e9fc1d/chromeos/dbus/update_engine_client.h

Labels: -Pri-2 -Target-69 Target-70 Merge-Request-70 Pri-1
I would like to merge this simple fix to M70.

Without this, if Chrome OS is waiting for reboot, and Chrome crashes, it sometimes (based on my tests, ~50%) shows a scary "Your administrator is rolling back your device. All data will be deleted when the device is restarted." error message (even on unmanaged devices).
Project Member

Comment 21 by sheriffbot@chromium.org, Oct 9

Labels: -Merge-Request-70 Merge-Review-70 Hotlist-Merge-Review
This bug requires manual review: We are only 6 days from stable.
Please contact the milestone owner if you have questions.
Owners: benmason@(Android), kariahda@(iOS), geohsu@(ChromeOS), abdulsyed@(Desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Labels: -Merge-Review-70 Merge-Approved-70
Project Member

Comment 23 by bugdroid1@chromium.org, Oct 9

Labels: -merge-approved-70 merge-merged-3538
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8c843612665103ca30bb14534c944370359cf49a

commit 8c843612665103ca30bb14534c944370359cf49a
Author: Marton Hunyady <hunyadym@chromium.org>
Date: Tue Oct 09 16:56:50 2018

Fix incorrect rollback notifications

|status.is_rollback| was left uninitialized when Chrome crashed.
This CL is initializing it as false. A follow-up CL is setting it
properly based on the version numbers.

Bug:  871759 
Change-Id: I1d705e7a406547e6463c310cb811b166deab5cff
Reviewed-on: https://chromium-review.googlesource.com/c/1269534
Reviewed-by: Achuith Bhandarkar <achuith@chromium.org>
Commit-Queue: Marton Hunyady <hunyadym@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#597880}(cherry picked from commit 272b638008f5d585dd02ec218674a73059e9fc1d)
Reviewed-on: https://chromium-review.googlesource.com/c/1271005
Reviewed-by: Marton Hunyady <hunyadym@chromium.org>
Cr-Commit-Position: refs/branch-heads/3538@{#922}
Cr-Branched-From: 79f7c91a2b2a2932cd447fa6f865cb6662fa8fa6-refs/heads/master@{#587811}
[modify] https://crrev.com/8c843612665103ca30bb14534c944370359cf49a/chromeos/dbus/update_engine_client.h

Labels: Merge-Merged-70-3538
The following revision refers to this bug: 
https://chromium.googlesource.com/chromium/src.git/+/8c843612665103ca30bb14534c944370359cf49a

Commit: 8c843612665103ca30bb14534c944370359cf49a
Author: hunyadym@chromium.org
Commiter: hunyadym@chromium.org
Date: 2018-10-09 16:56:50 +0000 UTC

Fix incorrect rollback notifications

|status.is_rollback| was left uninitialized when Chrome crashed.
This CL is initializing it as false. A follow-up CL is setting it
properly based on the version numbers.

Bug:  871759 
Change-Id: I1d705e7a406547e6463c310cb811b166deab5cff
Reviewed-on: https://chromium-review.googlesource.com/c/1269534
Reviewed-by: Achuith Bhandarkar <achuith@chromium.org>
Commit-Queue: Marton Hunyady <hunyadym@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#597880}(cherry picked from commit 272b638008f5d585dd02ec218674a73059e9fc1d)
Reviewed-on: https://chromium-review.googlesource.com/c/1271005
Reviewed-by: Marton Hunyady <hunyadym@chromium.org>
Cr-Commit-Position: refs/branch-heads/3538@{#922}
Cr-Branched-From: 79f7c91a2b2a2932cd447fa6f865cb6662fa8fa6-refs/heads/master@{#587811}
Project Member

Comment 25 by bugdroid1@chromium.org, Oct 10

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/18dd4fad020b819faab03c3a333a7514e6090af8

commit 18dd4fad020b819faab03c3a333a7514e6090af8
Author: Marton Hunyady <hunyadym@chromium.org>
Date: Wed Oct 10 09:42:09 2018

Set is_rollback based on the version numbers after Chrome crash

Rollback notifications were not properly shown after a Chrome crash.

This CL is adding the logic from StatusUpdateReceived to OnGetStatus
too.

Bug:  871759 
Change-Id: I8c07c4d4f61ebcd49498af08f3d9c1a6e000bb25
Reviewed-on: https://chromium-review.googlesource.com/c/1270896
Commit-Queue: Marton Hunyady <hunyadym@chromium.org>
Reviewed-by: Zentaro Kavanagh <zentaro@chromium.org>
Reviewed-by: Achuith Bhandarkar <achuith@chromium.org>
Cr-Commit-Position: refs/heads/master@{#598267}
[modify] https://crrev.com/18dd4fad020b819faab03c3a333a7514e6090af8/chromeos/dbus/update_engine_client.cc

Status: Fixed (was: Assigned)
Status: Verified (was: Fixed)
unable to repro this issue on M71 nocturne device flashed with 71.0.3578.13
Issue 896753 has been merged into this issue.
I think this is one the bugs I have on Version 72.0.3626.30 (Official Build) dev (64-bit) Excuse the impromptu but Im still learning my way around things. Not sure what info you need, so please feel free to ask.  
Screenshot 2018-12-22 at 2.25.51 AM.png
358 KB View Download

Sign in to add a comment