DTseng reports not reproducing on Canary

Does NOT reproduce on 
Eve Stable 67.0.3396.99 (Official Build) (64-bit)
Eve Beta 68.0.3440.87 (Official Build) beta (64-bit)

(Will continue checking on other channels/devices in another update.)

Does not reproduce on Eve Dev 70.0.3524.2 (Official Build) dev (64-bit)

Does not reproduce on Cave device (Cave has a touchscreen.)

Does not reproduce on another Eve device.

Does not reproduce on 70.0.3538.7 (Official Build) dev (64-bit)
Google Celes

(This machine does not have a touchscreen, so there goes that idea.)

At first it does not speak but that's because this is a very slow Chromebook. After a few seconds ChromeVox starts and then speaks normally as expected. 

Testing the hypothesis that it had to do with changing channels or updating did not repro the bug. I did the following on Google Eve:

Flash to 67.0.3396.66 (Official Build)(64-bit), ChromeVox speaks
Update to 68.0.3440.110 (Official Build) (64-bit) which is current stable, ChromeVox speaks
Update to 69 current beta, 69.0.3497.87 (Official Build) beta (64-bit), ChromeVox speaks
Update to 70.0.3538.7 dev channel, ChromeVox speaks. 

I first noticed this bug while on an airplane with the Lulu, so no Wifi connected. It reproed when I landed and connected to wifi again so I didn't mention the offline part in the bug above. 

I tested with wifi off on Eve and ChromeVox still spoke. 
70.0.3538.7 (Official Build) dev (64-bit) 

This revert should resolve this bug: https://chromium-review.googlesource.com/c/chromium/src/+/1235127

the change should have landed on 71.0.3557

Still looking for repro steps to trigger the bug so we can verify the revert worked. 

This bug started in https://chromium-review.googlesource.com/c/chromium/src/+/1220687 which landed in 71.0.3553.0 per this page: https://chromium.googlesource.com/chromium/src/+/c996527a66944e7e4498c3c112e62e407e2e5e99/chrome/VERSION

So the bug was introduced in 71.0.3553.0 and was reverted in 71.0.3557.0

dsexton@ was on 3554 and it reproed and fixed in 3558 when it updated. 

Assigning to @dtseng who is working on this. 

According to our update tracker, on 09/21/18 the dev channel was on 71.0.3554.0 which shows the bug behavior. On 09/26/18, the Dev channel was updated to 	71.0.3558.0 which should include the fix. 

I'm marking as fixed. If there are new issues that crop up, we should file new bugs. I have one new issue already filed (also fixed).

dtseng@, we've had reports of testers getting this bug while updating their dev channel and automatically receiving the bad builds. How can we be sure this bug doesn't slip into branch? 

Clarifying the above comment:

 Bug 891889  is for Google Text to speech crashes on signed out screen
Bug 878883 is for ChromeVox not speaking on Login screen

Both are as a result of regressing changes. Look at the commit range between when the bad change was landed and when it was reverted. Only builds within that window should be bad.

There are two such windows above.

David T, can you help us understand which builds make it into the Dev channel after the branch? That is, does the process itself ensure that these bad builds will never be seen again? 

Here's my understanding, please correct me where I am wrong: 

The day of branch, the launch team takes the existing Dev build for that day, labels it branch, and then the Dev channel moving forward contains the branch + any changes made to that branch, ignoring all milestone builds that came before branch. (Therefore the bad builds will be ignored forever.)

Canary's build numbers are incremented, for example, after the M71 branch Canary starts using M72 numbers. 

After all that development on the M71 branch + the changes specific to it, a certain day's Dev build is declared to be "Beta" and then moves to the Beta channel. 

At that time between beta promotion and another dev branch point, the Dev channel just starts getting Canary builds frequently. In this example, the Dev channel will keep getting M72 builds fresh off Canary until it's time for M72 branch. These "intermediate" M72 builds are all destined to be ignored at branch. 


However, this doesn't jive with the documentation, which says that the branch build is pulled from Canary, not from the Dev channel: https://chromium.googlesource.com/chromium/src.git/+/master/docs/process/release_cycle.md#Branch-Point

The documentation link you sent is correct.

Canary builds are based on top of trunk (in the Chrome repo). Each of these builds has a build number.

At branch point, an actual repo branch gets created. At this point, there are two "branches" -- a master branch and a branch for a specific version e.g. 70. A "branch" (in the git sense) means the history previous to the branch is entirely the same, but after the branch, they can and do diverge.

So, at this point, you have 70, and master. 70 can be annotated as dev, pormoted to beta, etc, but it will always be the 70 branch.

In the same way, there are branches for 69, ...

I'm not sure what you're asking here, but hopefully that clears things up.

In this bug specifically, there were two commits on master that caused regressions on the login screen. Both were reverted and merged to a branch. Merging is when we take a commit (in this case, master) and commit it to another branch.

So, in the builds for a particular branch, we would have seen this bug come back because there was a second regressing change...

Update: my corp chromebook is on Google Chrome
71.0.3578.8 (Official Build) dev (64-bit) and shows the bug behavior (as logged in separate bug 896978)

David S is seeing the bug in 70.0.3538.0 beta which he received through normal OTA updates. ChromeVox not speaking at Login. 

Reopening this as the master tracking bug for any of the 3 possible root causes leading to this behavior. Assigned to David T since he's actively working on it. 

+lpalmero@ and +kjbooker@ for tracking this bug. 

Issue is still reproducing, working with dtseng@ to get a test build showing the problem so that he can debug. 

Since this is a master bug, I'll mention that there are four instances of this bug behavior, all tracked in internal doc https://docs.google.com/document/d/1zHN-AfX3iDjBLUy_M7AuBakKiRSh5B00_DlPI1U_gq0/

Instance one, in M70, is still possibly in the wild. We didn't find the root cause yet. 
Instances two and three are found in M71 and have had CLs in M71. 
Instance four is currently found in M71. 

 Issue 753463  has been merged into this issue.

Just a quick ping to all on this thread. Please provide any more information regarding builds you're seeing this on. There was a block of time, in the latter part of 2018, where we had several regressions (all unrelated) that resulted in the loss of speech on the login screen. As of the latest 72/73 builds, speech should be working.