Related to/duplicate of  bug 706535 ?

ovanieva@ feel free to set initial priority. If something like this comes it at P3, it's easy to get missed.

thanks Albert! will do next time

All of the feedback is R58, is the problem in 58 as well as 59?

I confirmed with T3 support team: users attempted to update to M59, got black screen and submitted report from M58.

Just tried on my nyan_big with and without the DP patches from the blaze black screen issue. Both builds successfully booted.

Is this another case of some panels working while others don't? Do we have model numbers for the affected devices?

most reports are for: Acer Chromebook CB5-311

@ovanieva: I have a CB5-311 and it works fine. There are likely multiple SKUs of this model with different panels. I suspect one of these SKUs is susceptible to failure while the others are not. It would be useful to figure out which panels are out there so we can isolate the problem.

If we can find a feedback report, or get the HWID from a failing system we can determine what panel may be broken here.

adding James form T3 support who can update this bug based on info we get back from users. 

Here are (2) HWIDs from affected users:

BIG E2G-J4F-A3D-A8H
BIG E2H-A4F-B3D-A57

Consolidated bugs, CBC topics , RS/TC topics

https://docs.google.com/document/d/1srJTdAQGvOlPM5Nh2r-AHmIcAQD5XTJO66ZZR5ngNAI/edit?usp=sharing

These both appear to be 'display_panel_auo_1366_768' which maps to:

      display_panel_auo_1366_768:
        values:
          compact_str: AUO:2c13 [1366x768]
          height: '768'
          product_id: 2c13
          vendor: AUO
          width: '1366'

I think we might have some of these in the lab, looking at ssh root@chromeos4-row5-rack10-host5.cros ...
...
localhost ~ # modetest | grep 2c13
                        00ffffffffffff0006af2c1300000000
...

Is there a better way to find the '2c13' panel?

I don't see anything unusual in the message log on this DUT though (which happens to be running R60). 

I am attempting to get specific model numbers and screen resolutions from the users.
https://productforums.google.com/forum/#!topic/chromebook-central/I0Xo_BvUFWs

Can you pull the current update file for BIG? That way users could do a Recovery and not have to worry about automatic update breaking it again.

So far only the 1366x768 models are being reported on the CBC thread.

@bhthompson: the device I have here starts with the same EDID. My full edid is:

00ffffffffffff0006af2c1300000000
00180103801d10780abbf59455549027
23505400000001010101010101010101
010101010101261b5664500016303020
360025a4100000180000000f00000000
00000000000000000020000000fe0041
554f0a202020202020202020000000fe
004231333358544e30312e33200a004b

Running through (slightly modified) parse-edid gives:

$ ./parse-edid < nyan_big_edid
Checksum Correct

Section "Monitor"
        Identifier ""
        ModelName ""
        VendorName "AUO"
        # Monitor Manufactured week 0 of 2014
        # EDID version 1.3
        # Digital Display
        DisplaySize 290 160
        Gamma 2.20
        Option "DPMS" "false"
        Modeline        "Mode 0" 69.50 1366 1414 1446 1466 768 771 777 790 -hsync -vsync 
EndSection

Also, to clarify, that device works fine with 59 pre/post blaze dp changes.

There are a lot of users who have this problem –much more than 10. I've a CB5-311-T3MQ (ITA version), 1366-768 screen resolution.

See there: https://productforums.google.com/forum/#!topic/chromebook-central/I0Xo_BvUFWs;context-place=forum/chromebook-central

Hi, I have reported this as  bug 737267 .

As of this morning there are  new reports for Nyan-big (see reports below). All users report getting black screen after attempting to update to M59. 

Total 14 gFeedback reports from 26 June to date (note that gFeedback does not count replies to Forum thread, so to Giovanni's point, there are much more users affected).

https://feedback.corp.google.com/product/208/neutron?lView=rd&lReport=67076560816

https://feedback.corp.google.com/product/208/neutron?lView=rd&lReport=67074472564

https://feedback.corp.google.com/product/208/neutron?lView=rd&lReport=67068571918

https://feedback.corp.google.com/product/208/neutron?lView=rd&lReport=67062141356

https://feedback.corp.google.com/product/208/neutron?lView=rd&lReport=67045475171

https://feedback.corp.google.com/product/208/neutron?lView=rd&lReport=67043761084

https://feedback.corp.google.com/product/208/neutron?lView=rd&lReport=67016061254

https://feedback.corp.google.com/product/208/neutron?lView=rd&lReport=67014415449

https://feedback.corp.google.com/product/208/neutron?lView=rd&lReport=66988599073

https://feedback.corp.google.com/product/208/neutron?lView=rd&lReport=67000794019

https://feedback.corp.google.com/product/208/neutron?lView=rd&lReport=67002646811

Thanks for the feedback reports, they're all R58, though. It would be useful to get one from a device that is actively experiencing the issue. The following instructions should work:

- Plug the laptop into an external monitor (HDMI)
- Power the device on and wait for the wallpaper to appear on the display
- Close the laptop to enter "dock" mode, this should move everything to the external display
- Login and generate feedback report

@bhthompson: Can you start a scavenger hunt for an affected device in MTV (lab or otherwise)?

 Issue 737964  has been merged into this issue.

 Issue 737267  has been merged into this issue.

I am not sure how we effectively find one we expect to fail, the feedback reports all map to that 2c13 panel, so if your 2c13 panel system is not failing it means some may and may not be failing or there is something special about your system?

I am wondering if perhaps this is related to transitioning between the link training setups, like does it need an extra reboot when it gets the new AU?

If it is not something specific to the panel, any other ideas on what identifying characteristic we should look for?

Maybe there is something unique about mine, wouldn't hurt to give another device with the same panel a try. We should probably RMA one of the reported failed units ASAP just in case we can't find a unit in house.

Multiple reboots will probably not fix the issue

Is it possible for somebody with an affected nyan to try out a custom kernel? I have an idea as to what could be wrong that'd be nice to try out.

Here are Reports from m59 with an external monitor

http://hotsauce/ng#/Report/67119375119
http://hotsauce/ng#/Report/67123902955
http://hotsauce/ng#/Report/67119774571

Thanks! We now have our error message:

...
[    5.787106] [drm:drm_dp_link_train_full] *ERROR* clock recovery failed, downgrading link
[    5.793500] [drm:drm_dp_link_train_full] *ERROR* clock recovery failed, downgrading link
[    5.794075] [drm:drm_dp_link_train] *ERROR* full link training failed: -22
[    5.794086] tegra-sor 54540000.sor: DP link training failed: -22
...

re: #25 -- I added your request on the CBC thread with a bit more details for the users. If you close the lid, it's rather difficult to get to the keyboard to log in! Most would not have access to an external keyboard/mouse. I instructed them to hold down the "dim" key to enter docked mode.

I think we are ok on feedbacks from comment 32, now we need to find a unit we can reproduce this locally.

I have been unsuccessful with three units I was able to find that have this panel here, they all work fine, if someone has one of these failing systems out there and would be willing to ship it to us that would aid debugging.

Also if someone happens to have multiple of this type of system, but only some of them are failing that would be good to know for sure. 

So far we have not been able to reproduce this within Google, which implies either something is special about our units (for instance, they are all from early factory builds) or that the failure is random. 

We may have identified a workaround that might also point to the source of the problem. 

Doing a Recovery or powerwash/revert can get users back to v58. But, the system would quickly update and the user would be back to the black screen.

The workaround was discovered by a user and I have asked other users to test it. The key is to NOT allow the system to restart after the update. Instead, power down manually and then power back up.

Please see the CBC thread listed in the OP for details.

Hi, after reading this comment:

https://productforums.google.com/d/msg/chromebook-central/I0Xo_BvUFWs/-RvplP2TAgAJ

I thought Id try using the 'click arrow to upgrade' option and everything has worked fine; whereas two days ago I received the black screen.

I am now running Version 59.0.3071.91 (Official Build) (32-bit) on my C810-T6YB.

Thanks to everyone for their help and speedy resolution, still loving my Chromebook.

 I figured the problem out by the recovery utility, but today is obviously the same as I powered my Chromebook off and it has updated. Any solution? 

CBS users are reporting successful updates now - it appears that the problematic version has been pulled. Success on:

59.0.3071.91
Platform 9460.60.0 (Official Build) stable-channel nyan_big
Firmware Google_Nyan_Big.5771.63.0

So, just to clarify...

When the system warm reboots from R58 to R59, the screen is black. If the user powers down the device and cold boots, the display comes up?

I don't really think it's the case. I think users for whom this "trick" worked were just lucky, and upgraded their systems after 59.0.3071.113 was pulled from upgrade servers and the simply upgraded to 59.0.3071.91.

I tried it this morning and it didn't worked because the actual upgrade was done yesterday afternoon. After the black screen I did recovery again and then it upgraded to from R58 to 59.0.3071.91 which works without problems.

The last comment seems right. I tried the power down, cold boost yesterday and it didn't work. Had to recover again. But then then downgraded upgrade (.91 instead of .113) was loaded and I was able to reboot normally into that build. 

Ahh, ok. Thanks for clarifying. Yes, I expect anything before 59.0.3071.111 (CrOS 9460.66.0) will work on big.

So it doesn't look like we have a repro unit on site. Can we please reach out to one of the affected users and do a swap ASAP?

I was hoping we could get something in the mail before the long weekend, but that doesn't seem to be possible. Please reach out to me if there's any red tape that prevents us from getting a unit quickly.

What would a swap involve? I'm in Chicago. Not sure if you have people working here. Happy to help if it can clear up the bug. 

We don't have a way to actually swap units I am aware of, at least not a way of getting a replacement unit out there (this is all the engineering team, the RMA folks at Acer or support teams at Google might have more options), but rather we would borrow one and ship it back.

Chicago is a bit far out, the engineers whom could help with this would be in California, Oregon, or North Carolina. 

I just had this happen with a nyan-big CB5-311 when updating today (stable channel) - not sure when it actually download the update. Didn't matter cold vs warm boot, pressed the battery disconnect button (which usually fixes the weirdnesses of this model), etc. Sorry I don't know the version it have before the update. USB recovery worked but despite "checking for updates" during first login, put me back on 58.0.3029.140.

Is the long string from the recovery screen (BIG E2G-A4F-A3D-A35) that I had to enter to get the recovery image useful to you in any way? Or is it just a matter of waiting for someone with one of these units near the engineers to hand it over so a fix can be tested before being pushed?

(For what it's worth, this same device has also recently - maybe last 6 months or so - had "blocks" of the chrome startup screen displaced/small horizontal black lines, and some web video decoding ends up exceedingly blocky - like 20px by 20px. Had tried a powerwash then a recovery previously to try to repair that. Realized after the fact that I can't edit a message, so I hope this is useful enough to merit the emails...)

I tried to copy all of the dmesg log of this unit in case that would be helpful, but apparently I can't copy more than a screenful and there's of course much more than than. Let me know if there's any data I can get from my device that would be helpful in diagnosing or solving this issue. It's never been in developer mode and I've just used it basically as a "consumer", but I do have a Jetson Tegra board in the other room with a UART console hooked up to it so I do know my way around (closer to typical, anyway) linuxes and some hw/driver debugging...

i have a Acer Chromebook CB5-311 .screen still dark ..i have tried multiple times rest ..restart ..recover but nothing worked so far 

We are getting a fresh batch of users reporting problems on CBC. Don't know if they are late in restarting after the old update, or if the current update is an issue.

Same problem. Acer CB5-311 gets blank screen on update. Multiple recoveries hoping the latest OS patch is the fix. Still persists today
Model BIG E2G-J4F-A3D-A8H

Latest update worked! 
Model Model BIG E2G-J4F-A3D-A8H now updating to Version 59.0.3071.91 (Official Build) (32-bit). I knew Google Chromium wouldn't let me down ;-)

re @comment47

@jgomezjr - can we get one of the units to be shipped to ENG?

As an update as of this morning we are down to 2 reports per day for Nyan-Big,
 which is pre-spike levels. 

We stopped M59 updates on glimmer last week, hence the decreased # of reports.

@ovanieva  unfortunately all of our affected users are located out side of the US, mainly in GB.  Please let us know if you'd like us to reach out to them to borrow an affected unit. 

@seanpaul please advise?

We would very much still like to get our hands on an affected device. If you have a device that works with 59.0.3071.91 but is broken with 59.0.3071.113 please try to arrange to ship it to us.  If necessary, coordinate with me directly and I'll provide my shipping address and can reimburse the shipping costs if that's an issue.

ok!

@jgomezjr how we do typically do this? Let's arrange for shipment of at least one device to ENG

i am affected by this issue and i am in the us 

Echoing hoegsberg@'s comments, let's get a device ASAP regardless of where it comes from.

@christineclarkwfh - James will be reaching out to you with instructions shortly. 
thanks!

i can not ship it out ...i can not walk ,don't drive and can not make it to the ups shop

ok then. We are arranging for device from GB user. I'll keep you posted.

@sean.johnson.andrews I've reached out to you in hopes we can borrow your unit. 

#CBC-RS/TC-watchlist

FWIW, I'm in Wisconsin, and since this machine is essentially bricked until the issue gets fixed (and since it also has those weird panel lines on the startup screen, which also seem panel-driver-related since they arrived with an update and persist through powerwash and recovery), you can actually have mine to keep for about $150 (or if it's easier a new Chromebook of comparable capability)...  Feel free to contact me directly.

Issue 737945 has been merged into this issue.

Pri-0 bugs are critical regressions or serious emergencies, and this bug has not been updated in three days. Could you please provide an update, or adjust the priority to a more appropriate level if applicable?

If a fix is in active development, please set the status to Started.

Thanks for your time! To disable nags, add the Disable-Nags label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

We have our hands on a chromebook that reproduces the regression and have a way to fix it that we're currently finishing up.

Problem on mine too :
BIG E2H-I4E-B3L-A8A

cb5-311p-t4s8
with 1366x768

Issue is still reproducible on latest dev # 61.0.3158.0/9751.0.0 Candy.

If you see this on a Candy (Dell 11") system, that is probably a very different problem worthy of a separate bug. This particular problem is specific to Nyan (Tegra) based systems. 

As per comment # 75 , un duping issue 737945 as it is seen on Candy.
It seems to be recent Regression in M-61(Issue 737945, C#10)

Thanks !

hoegsberg@, any luck with the fix for this issue?

We have a CL: https://chromium-review.googlesource.com/c/575697/

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/kernel/+/fbd237dcf52b4d39189df241efb3ef0749ae3726

commit fbd237dcf52b4d39189df241efb3ef0749ae3726
Author: Kristian H. Kristensen <hoegsberg@google.com>
Date: Wed Jul 19 19:34:31 2017

CHROMIUM: drm/tegra: Assume success for clock recovery and equalization

The old fbdev code had a bug where it failed to properly read link
status for 1 lane link configurations. From
tegra_dp_clock_recovery_status() in drivers/video/tegra/dc/dp.c:

	for (cnt = 0; cnt < n_lanes / 2; cnt++) {
		/* check status, maybe return false.. */
	}

	return true;

which for n_lanes == 1 never actually checks the status and always
returns true.  This bug obscures a bug in certain panels that fail to
indicate successful clock recovery or equalization, unless the DC and
PE MAX_REACHED flags are set. This panel also don't request any
adjustments, so the max will never be reached. The only way this
worked was because the fbdev driver effectively always assumed
success.

This patch restores that behaviour. While certainly not ideal, this
used to work and there's not much we can do given that the devices are
out there with faulty panels.

BUG= chromium:737618 
TEST=Tested on even more (4) nyan devices (lowres and fhd) including
  flakey panels, display comes up on all

Change-Id: I4279e61aa9e5bfdca28d2b4a77d66a527797a4f1
Reviewed-on: https://chromium-review.googlesource.com/575697
Commit-Ready: Kristian H. Kristensen <hoegsberg@chromium.org>
Tested-by: Kristian H. Kristensen <hoegsberg@chromium.org>
Tested-by: Sean Paul <seanpaul@google.com>
Reviewed-by: Sean Paul <seanpaul@google.com>

[modify] https://crrev.com/fbd237dcf52b4d39189df241efb3ef0749ae3726/drivers/gpu/drm/tegra/sor.c

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/kernel/+/c952d582035ca595f94fb2e19bf98274f5cb6bc4

commit c952d582035ca595f94fb2e19bf98274f5cb6bc4
Author: Kristian H. Kristensen <hoegsberg@google.com>
Date: Wed Jul 19 22:49:53 2017

CHROMIUM: drm/tegra: Assume success for clock recovery and equalization

The old fbdev code had a bug where it failed to properly read link
status for 1 lane link configurations. From
tegra_dp_clock_recovery_status() in drivers/video/tegra/dc/dp.c:

	for (cnt = 0; cnt < n_lanes / 2; cnt++) {
		/* check status, maybe return false.. */
	}

	return true;

which for n_lanes == 1 never actually checks the status and always
returns true.  This bug obscures a bug in certain panels that fail to
indicate successful clock recovery or equalization, unless the DC and
PE MAX_REACHED flags are set. This panel also don't request any
adjustments, so the max will never be reached. The only way this
worked was because the fbdev driver effectively always assumed
success.

This patch restores that behaviour. While certainly not ideal, this
used to work and there's not much we can do given that the devices are
out there with faulty panels.

BUG= chromium:737618 
TEST=Tested on even more (4) nyan devices (lowres and fhd) including
  flakey panels, display comes up on all

Change-Id: I4279e61aa9e5bfdca28d2b4a77d66a527797a4f1
Reviewed-on: https://chromium-review.googlesource.com/575697
Commit-Ready: Kristian H. Kristensen <hoegsberg@chromium.org>
Tested-by: Kristian H. Kristensen <hoegsberg@chromium.org>
Tested-by: Sean Paul <seanpaul@google.com>
Reviewed-by: Sean Paul <seanpaul@google.com>
(cherry picked from commit fbd237dcf52b4d39189df241efb3ef0749ae3726)
Reviewed-on: https://chromium-review.googlesource.com/578569
Reviewed-by: Kristian H. Kristensen <hoegsberg@chromium.org>
Commit-Queue: Kristian H. Kristensen <hoegsberg@chromium.org>

[modify] https://crrev.com/c952d582035ca595f94fb2e19bf98274f5cb6bc4/drivers/gpu/drm/tegra/sor.c

Issue(Issue 737945 duped here in C#70) is still reproducible on latest dev # 62.0.3164.0/9773.0.0 on Candy Device i.e. Black screen is seen after recovering build.
Please let us know if any further information required about Device.

Thanks..!! 

Issue 737945 was un-duped in C#76 as this seems to be different problem.

Is this resolved on M60?

Yes, and we have confirmation from at least one user in the field, we can probably mark this as fixed. 

Verified on M60 with version 9592.71.0, 60.0.3112.80:

1. Auto update from M59 beta to M60 beta 
2. Install Recovery build with USB stick

I haven't seen anything promoted in stable channel - assuming 59 will stay at the .91 for these devices until M60 is stable? Anyway, switching from stable channel to dev channel on my affected device, which gets me to version 60.0.3112.80 (platform 9592.71.0 beta-channel nyan_big, firmware Google_Nyan_Big.5771.63.0), and does work so far - display powers on, at least.

I'll open a bug (or a CBC thread?) for the other display-related issues I have seen if I continue to see them.

I started a topic in RS/TC but don't know if these have anything to do with this specific issue. Different chromebooks besides Nyan_Big.

Black/blank screen > can't USB recover - Google Product Forums
https://productforums.google.com/forum/#!private-topic/chromebook-central+rs-mentor/9Yl0poMGMN4;context-place=private-forum/chromebook-central+rs-mentor