New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 793026 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner: ----
Closed: Oct 18
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 3
Type: Bug



Sign in to add a comment

err_ssl_version_interference - Fortigate FortiOS 5.0 - disabling TLS 1.3 Fixes

Reported by dav...@skyfactory.com, Dec 7 2017

Issue description

Chrome Version       : 63.0.3239.84
OS Version: 6.1 (Windows 7, Windows Server 2008 R2)
URLs (if applicable) : https://www.google.com/
Other browsers tested:
  Add OK or FAIL after other browsers where you have tested this issue:
     Safari:
    Firefox: OK
    IE/Edge:

What steps will reproduce the problem?
1. Can't access GMail or google.com
2. err_ssl_version_interference
3. Setting TLS 1.3 to disabled under chrome://flags works

What is the expected result?
web page is loaded

What happens instead of that?
err_ssl_version_interference

Please provide any additional information below. Attach a screenshot if
possible.
Two people had this problem today.  Possibly others once Chrome updates.  

FortiOS 5.0 firewall (an older version) - I am having other issues related to secure connections but this is the first time I've been able to fix it by disabling an encryption type.
 
Did not experience the issue with 62.0.3202.94 on my computer.
Updated to 63.0.3239.84 -- worked fine
Changed TLS 1.3 from "Default" to "Enabled (Draft)" -- causes this error
"Enabled (Experiment)" -- ERR_SSL_VERSION_INTERFERENCE error
"Enabled (Experiment 2)" -- ERR_SSL_VERSION_INTERFERENCE error
"Enabled (Experiment 3)" -- ERR_SSL_VERSION_INTERFERENCE error
"Disabled (Deprecated Setting)" - works
"Disabled (Deprecated Setting)" - there are two of these entries and both worked
"Disabled" - works
Cc: awhalley@chromium.org pbomm...@chromium.org davidben@chromium.org
Labels: M-63
+awhalley@ and davidben@, could you ptal and reassign if needed. Thank you.
Cc: svaldez@chromium.org agl@chromium.org
Components: Internals>Network>SSL
This indicates that you have a buggy firewall or other device. We weren't aware of issues remaining with FortiOS, but they were been quite unresponsive to our repeated contacts all year, so it's plausible there are unexplored issues here.

Could you attach a net-internals log per these instructions? Thanks!
https://dev.chromium.org/for-testers/providing-network-details

Upgrading to the latest versions of your firewall devices may also help with the issue.
Labels: Needs-Feedback
OK, I have two captures, 

chrome-net-export-log-draft-tls1.3.json
TLS 1.3 set to enabled (problem setting) 

chrome-net-export-log-no-tls1.3.json
TLS disabled (works fine)

chrome-net-export-log-draft-tls1.3.json
610 KB View Download
chrome-net-export-log-no-tls1.3.json
980 KB View Download
Project Member

Comment 6 by sheriffbot@chromium.org, Dec 7 2017

Labels: -Needs-Feedback
Thank you for providing more feedback. Adding requester "davidben@chromium.org" to the cc list and removing "Needs-Feedback" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
I can also confirm that TLS 1.3 works if I punch out of the firewall using a VPN.
Labels: Needs-Feedback
Oh! What happens if you set it to "Enabled (Experiment2)"? We're not actually shipping the draft-18 version because it trips loads of bugs in middleboxes, including FortiOS. The middleboxes are the ones that messed up but, alas, the breakage was too high.

We're actually shipping the somewhat misleadingly-named "Experiment2" setting which works around the bugs we know of. (At the time, the workarounds were still experimental.)
Additionally, do other HTTPS sites work? (E.g. https://www.facebook.com.) Normally we'd have expected this to only affect Gmail, not www.google.com.
At least www.hbo.com doesn't work (it seems to enforce https).  I didn't try more than a couple of sites.

I tried the Experiment2 setting and captured a log file.  It fails in the same way.
chrome-net-export-log-experiment-2b.json
910 KB View Download
Project Member

Comment 11 by sheriffbot@chromium.org, Dec 7 2017

Labels: -Needs-Feedback
Thank you for providing more feedback. Adding requester "davidben@chromium.org" to the cc list and removing "Needs-Feedback" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Labels: Needs-Feedback
Fascinating. What's your FortiOS firmware version? Could you upgrade it? We've had previous reports with Fortinet devices that worked with Experiment2, so perhaps there was some bug in an older revision.

I also notice that your log shows secure.livechatinc.com working, so not quite everything is broken? This is puzzling because most of the Google sites you list aren't running TLS 1.3 yet.

Thanks for your responsiveness, by the way!
Oh, also could you include a net-internals that tries to talk to https://tls.ctf.network/? (That's one of our test servers.) Thanks!
FG100D-5.00-build271 - built circa 2014

We don't have an active support contract with FortiNet right now so I can't upgrade the firmware, I'm vacillating back and forth between replacing the firewall with a Tomato-firmware Asus AC3200 or reactivating our support contract, hoping to fix the existing FortiGate fw, which is frankly well beyond the needs of our company.

I'm not sure if this date means anything to you, but I had some automated emails sent from my Ubuntu server here locally that stopped on precisely August 4th of this year.  Poking in /var/log/mail.log shows this error:

Dec  7 14:44:47 air2 postfix/smtp[19817]: F07485447A0: to=<davidr@skyfactory.com>, relay=smtp.gmail.com[74.125.70.108]:587, delay=369924, delays=369903/0.03/21/0, dsn=4.7.5, status=deferred (Cannot start TLS: handshake failure)

I'm sure this issue began on August 3rd or 4th, I'm 90% certain this is the same issue as we experienced in Chrome today, and I'm 99% sure the FortiGate is to blame.

As to your question about not everything being broken...
Is it possible that some TLS connections could persist temporarily across restarts?  Changing the setting to the "experiment2" setting, I had to stop and restart the capture, because it was working...  Then after a minute or so it stopped and that was when I logged the experiment 2 capture.

So, perhaps the secure.livechatinc.com was working using a previously-established connection, in a way the firewall doesn't interfere with?
Project Member

Comment 15 by sheriffbot@chromium.org, Dec 7 2017

Labels: -Needs-Feedback
Thank you for providing more feedback. Adding requester "davidben@chromium.org" to the cc list and removing "Needs-Feedback" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Here's a log of my attempted visit to https://tls.ctf.network/? with Experiment 2 enabled.

chrome-net-export-log-experiment-2-tls.ctf.network.json
774 KB View Download
Thanks! It looks like, from logs on the test server, that the TLS 1.3 ClientHellos aren't making it out of the FortiGate device at all, which is consistent with it affecting all sites, not just those that support TLS 1.3, though not something we'd seen before. (TLS works by having the client advertise everything it supports, and the server picking something. So a TLS 1.3 ClientHello is also a valid TLS 1.2 ClientHello. TLS 1.2 servers will then pick TLS 1.2, and TLS 1.3 servers pick TLS 1.3.)

It does appear to kind of inconsistently block things, so I'm not really sure what's going on there. One of your logs even shows a Google site getting through. TLS connections and resumption information don't persist across restarts, so that wouldn't be it. Perhaps the device is stateful and once it gets offended by something, it starts blackholing things?

I don't think the smtp.gmail.com thing could be related? We never deployed TLS 1.3 to that service, and presumably the client there is postfix or so, not Chrome, so they wouldn't be offering TLS 1.3, or whatever else caused them troubles.
By the way, I forgot to mention: you can use the SSLVersionMax administrative policy to disable TLS 1.3 in your organization in the meantime. Though it does sound like there is indeed *something* weird going on with your device that we ought to get to the bottom of.

https://www.chromium.org/administrators/policy-list-3#SSLVersionMax
Labels: Needs-Triage-M63
Cc: krajshree@chromium.org
Labels: Triaged-ET Needs-Feedback
davidr@ - Could you please try the SSLVersionMax administrative policy as per comment #18 and please let us know your observations.

Thanks...!!
davidr: Sorry about that. Please disregard the previous comment.

krajshree: No, there is no interesting observation needed from SSLVersionMax. It is known what affect that will have. That is a temporary workaround for the bug in the firewall.
Labels: -Needs-Feedback
Labels: TE-NeedsTriageHelp
As this issue is not reproducible from TE end, adding TE-NeedsTriageHelp label. Could someone from Internals>Network>SSL team please have a look at this.

Thanks!
Labels: Needs-Feedback
Are you still running into issues here?
Interesting but fairly unsatisfying conclusion to this issue, we finally replaced our firewall but that did not resolve the issue for us.  Our local ISP determined that our problem was the result of a bad port that we were connected through.  They swapped the port we were on, problem solved.
Project Member

Comment 26 by sheriffbot@chromium.org, Oct 18

Labels: -Needs-Feedback
Thank you for providing more feedback. Adding the requester to the cc list.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Status: WontFix (was: Unconfirmed)
Thanks for following up! I'll go ahead and close this then.

Sign in to add a comment