New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 800598 link

Starred by 4 users

Issue metadata

Status: Verified
Owner:
Closed: Jan 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug-Regression



Sign in to add a comment

Enrollment fails over a network with the SSL whitelist.

Project Member Reported by krishna...@chromium.org, Jan 10 2018

Issue description

Chrome Version: <From about:version: 64.0.3282.83
Chrome OS Version: 10176.43
Chrome OS Platform: Blaze
Network info: Wifi with a Firewall.

Summary: Enrollment over a locked Network (ie a network which has a firewall with the URLs mentioned in   https://support.google.com/chrome/a/answer/6334001?hl=en&ref_topic=3504941  whitelisted).

Steps To Reproduce:
1) On the OOBE of a chrome device connect to a Network with a proxy  that has a SSL whitelist (ie, the proxy/firewall has all the URLs blocked except for those mentioned in the SSL whitelist.) 
2) Step thru OOBE, on the Sign in screen press Ctrl+Alt+E to navigate to the Enrollment screen
3) Attempt to enroll the device.
Expected Result:
The device should be able to Enroll successfully 
Actual Result:
The device displays the Spinner of Death in the Enrollment frame on attempting to Enroll.
How frequently does this problem reproduce? (Always, sometimes, hard to
reproduce?)
Always.
What is the impact to the user, and is there a workaround? If so, what is
it?
Schools/Enterprises that have implemented the SSL whitelist will not be able to enroll their devices


This works fine on M63.
Attached device logs(verbose level set) and netlog. 

 
netlog
1.9 MB View Download
debug-logs_20180109-152459.tgz
371 KB Download

Comment 1 by roy...@google.com, Jan 10 2018

Labels: Hotlist-Enterprise ReleaseBlock-Stable

Comment 2 by roy...@google.com, Jan 10 2018

[Krishna/Harpreet said its possible the device is making request to maps.google.com which is critical for enrollment to succeed in M64]

Comment 3 by roy...@google.com, Jan 10 2018

Labels: -Pri-1 Pri-0
The same issue on 64.0.3282.41/10176.22.0 build, Candy.
Cc: igorcov@chromium.org
Owner: pmarko@chromium.org
Igor/Pavol - we should look at this today. Any idea why the heck we'd need to access maps.google.com to do enrollment (if that is indeed the problem)?

Comment 6 by pmarko@chromium.org, Jan 10 2018

Last time I've seen the spinner of death on enrollment, it was due to tlsdate not syncing the date.

Reason: during the enterprise enrollment process, chrome requests server-backed state key from session_manager. Since  bug 649422 , sesssion_manager waits for tlsdate to sync the date when state keys are requested. When tlsdate can't sync the date session_manager waits forever.

Symptom: Enrollment screen hangs forever on the spinner after entering enrollment data.

I've seen this happen behind an auth-proxy (see my comment https://bugs.chromium.org/p/chromium/issues/detail?id=127019#c92 ) but in theory this could be the same issue, I'll verify.

Comment 7 by pmarko@chromium.org, Jan 10 2018

So, the tlsdate.log from the original report seems to have trouble talking to clients3.google.com:443, which is on the list.

Krishna, are you sure this host was whitelisted?


Comment 8 by pmarko@chromium.org, Jan 10 2018

Status: Assigned (was: Available)
Just checked that the host (clients3.google.com) used by tlsdate hasn't changed in a long time, so it should have been used on M-63 too, but worth double-checking anyway.

Another thing that could go wrong is propagation of proxy details from chrome to tlsdated. I see calls to ResolveProxy in the chrome log, and according to chrome they complete synchronously, but I don't see the data transferred in the response or if this data arrives back in tlsdated.

I'll try on my machine today.
The 'maps.google.com' endpoint(which is used for timezone detection(/src/chromeos/timezone/timezone_request.cc)  is prolly not the cause of the failure, since. it's not a blocking request and ChromeOS has been using this URL for a couple of releases before M64. (this issue does not repro on M63). 

Clients3. is Whitelisted in the proxy this issue does not repro over M63 thru the same firewall



Not sure how helpful this might be but:
From the time I clicked Enroll on the UI, it took over a minute for the request to Gaia (ie, accounts.google) to be sent. 
The Gaia request was successful. 
The checklicense to the DM server(ie, m.google.com) using the Gaia token was successful as well. 
The subsequent register/enrollment requests were never sent to the DM Server. The attached netlog documents the timestamps and requests. 
Cc: icoolidge@chromium.org
+icoolidge

I could repro this on M-64, but not on M-65.

It seems that the tlsdate CL https://chromium-review.googlesource.com/c/chromiumos/third_party/tlsdate/+/717021 landed in Chrome OS platform 10045.0.0 and got reverted in 10220.0.0. So 10176.* which we both reproduced on has this CL.

I also tested that chromium replies to proxy resolution requests correctly through dbus-send while tlsdate was still failing, so I assume tlsdate is able to get the proxy info through dbus.

Ian, could it be that this CL breaks tlsdate's ability to talk through a HTTPS CONNECT proxy?
If yes, the solution would be to just merge the revert https://chromium-review.googlesource.com/#/c/807284/ to M-64.

Thanks!
Status: Started (was: Assigned)
I've used a tlsdate version with the revert https://chromium-review.googlesource.com/#/c/807284/ on M-64 and confirmed that tlsdate syncs date correctly behind a proxy and that enrollment works fine then.

I'll prepare a merge CL to merge it back the revert to M-64.

Re >1 minute in Comment #10: This is probably the auto-enrollment check ("Determining device configuration") timing out, as the AutoEnrollmentController also requests server-backed state keys from session, but it has a timeout of 90s. I've also observed the long delay when running with the tlsdate version containing the bug, but not when running with the fixed one.
Thanks Pavol for fixing this.
The request for server backed state keys has infinite timeout. The 90 seconds is the timeout for overall auto_enrollment_controller if some process is stuck to notify a the user with a network error.
Cc: kbleicher@chromium.org
+kbleicher@

Requesting merge of the revert CL (see Comment #14) to M-64 (release-R64-10176.B).
FYI: The CL being merged was submitted to ToT on Dec 16.
Labels: Merge-Request-64
Project Member

Comment 17 by sheriffbot@chromium.org, Jan 10 2018

Labels: -Merge-Request-64 Hotlist-Merge-Review Merge-Review-64
This bug requires manual review: We are only 12 days from stable.
Please contact the milestone owner if you have questions.
Owners: cmasso@(Android), cmasso@(iOS), kbleicher@(ChromeOS), abdulsyed@(Desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
To confirm, this request appears to be for a revert, fair?
Yes, the CL we'd like to merge back reverts to M-63 state of tlsdate, which is also the ToT state.
Labels: -Merge-Review-64 Merge-Approved-64
Approving merge to M64 Chrome OS.

Project Member

Comment 21 by bugdroid1@chromium.org, Jan 10 2018

Labels: merge-merged-release-R64-10176.B
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/tlsdate/+/a65ce86233ac17c1094b40117ef0d71416b0c202

commit a65ce86233ac17c1094b40117ef0d71416b0c202
Author: Matt Stokes <matthewbstokes@google.com>
Date: Wed Jan 10 17:25:40 2018

[M64] Revert "tlsdate: Support IPv6-only networks"

Reason for merge: Without the revert, tlsdate does not respect
proxy servers (obtained from chromium through DBus).
Bug:  800598 

This reverts commit 411bfbd54e5eea784596ef6622010d37483d8058.

Reason for revert:
No longer needed. Revisit IPv6 support at a later date.
Bug=b:70175610

Original change's description:
> tlsdate: Support IPv6-only networks
>
> This is based on a patch that buildroot applies to tlsdate,
> originally authored by Avery Pennarun (apenwarr@gmail.com).
> https://github.com/jameshilliard/buildroot/blob/master/package/tlsdate/tlsdate-0020-openssl-ipv6.patch
>
> BUG=b:63137232
> TEST=tlsdate is able to sync time on IPv6 only system
>
> Change-Id: I8529ea54fbcc1e77f2cdc2787debbe793cd68f53
> Reviewed-on: https://chromium-review.googlesource.com/717021
> Commit-Ready: Ian Coolidge <icoolidge@google.com>
> Tested-by: Ian Coolidge <icoolidge@google.com>
> Reviewed-by: Dan Erat <derat@chromium.org>

Bug: b:63137232
Change-Id: I0bf8918f121f533ad065963c26357e1d533ca3b8
Reviewed-on: https://chromium-review.googlesource.com/807284
Commit-Ready: Joshua Emele <jemele@google.com>
Commit-Ready: Ian Coolidge <icoolidge@google.com>
Tested-by: Joshua Emele <jemele@google.com>
Tested-by: Ian Coolidge <icoolidge@google.com>
Reviewed-by: Dan Erat <derat@chromium.org>
Reviewed-by: Ian Coolidge <icoolidge@google.com>
(cherry picked from commit c4e7e80a1c601c68a602598a8192da1daf3fea35)
Reviewed-on: https://chromium-review.googlesource.com/859797
Tested-by: Pavol Marko <pmarko@chromium.org>
Trybot-Ready: Pavol Marko <pmarko@chromium.org>
Commit-Queue: Pavol Marko <pmarko@chromium.org>

[modify] https://crrev.com/a65ce86233ac17c1094b40117ef0d71416b0c202/src/tlsdate-helper.c

Status: Fixed (was: Started)
Thanks, merged to M-64. I'll monitor the builds and ping when a M-64 build with this change is ready for verification.
Labels: -Pri-0 Pri-1
Downgrading Priority to P1
The first M-64 build with this change is 10176.45.0.
Status: Verified (was: Fixed)
Verified on Candy and Eve, 10176.53.0/64.0.3282.97.
Was able to successfully enroll and login against the network with a proxy that has a SSL whitelist. Also verified that the user cannot access the URLs that are not on the whitelist.

Chrome Version: 64.0.3282.97
Chrome OS Version: 10176.53
Chrome OS Platform: Eve/Candy
Network info: WiFi with a Firewall
Project Member

Comment 27 by sheriffbot@chromium.org, Jan 19 2018

This issue has been approved for a merge. Please merge the fix to any appropriate branches as soon as possible!

If all merges have been completed, please remove any remaining Merge-Approved labels from this issue.

Thanks for your time! To disable nags, add the Disable-Nags label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Labels: -Merge-Approved-64
Removing Merge-Approved label as this has been merged.

Sign in to add a comment