Same error using Chromium on Raspbian 8.0 after recent apt dist-upgrade.

Version 56.0.2924.84 Built on Ubuntu 14.04, running on Raspbian 8.0

Package: chromium-browser/stable, now 56.0.2924.84-0ubuntu0.14.04.1.1000 armhf

All included Google SSL Certificates have expired (2014).  I found similar bugs for Chromium from a year ago that had been fixed in newer patches.  Seems like a package issue.

Deleted: Chromium Expired Certificates.pdf 2.1 MB

Chromium Expired Certificates.pdf 2.1 MB Download

I don't think those certificates are related:
https://productforums.google.com/forum/#!topic/chromebook-central/aZnszydGkHw

This problem went away for me for a while yesterday, but has now returned. I'm no-longer seeing the problem on
 https://cloud.google.com
but am seeing it on
 https://console.cloud.google.com
and still on
 https://dev.chromium.org/for-testers/providing-network-details

The problem goes away if I set the maximum TLS version to 1.2 in chrome://flags. Firefox (which works) appears to negotiate TLS 1.2. Chrome on OS X appears to use QUIC and/or HTTP2 whether QUIC is enabled or not. Maybe something's wonky with TLS 1.3 negotiation.

Changing the max TLS version to 1.2 also fixed the problem for me.

Unable to reproduce the issue on Ubuntu 14.04 using chrome stable #56.0.2924.87 and latest dev #58.0.3013.3.

Attached a screen cast for reference.

Following are the steps followed to reproduce the issue.
------------
1. Navigated to https://cloud.google.com and https://dev.chromium.org/for-testers/providing-network-details
2. Observed that both pages loaded as expected.

Reporter@ - Could you please check this issue on latest stable #56.0.2924.87 by creating a new profile without any apps and extensions and please let us know if the issue still persist or not.

Thanks...!!

Deleted: 693943.ogv 18.3 MB

693943.ogv 18.3 MB Download

Are there any network proxies or firewalls running in the network that you're experiencing problems with?

Could you also check whether you are getting the same error when visiting: 
https://cert-test.sandbox.google.com


Chrome Stable is shipping a modern, more secure version of the TLS protocol, unfortunately some middle-boxes and network security solutions behave unexpectedly with the new protocol.

I don't think this is a generic TLS 1.3 issue.
cloudflare.com and other TLS 1.3 websites work perfectly fine (and show up in DevTools as using TLS 1.3)
This seems to be only impacting Google-owned domains.

We have another reproduction case from #chromium-support, this time with a pcap file (attached). The user asserts and the pcap file confirms that they do not have a proxy configured. Looking at the packet capture, the ServerHello has a sensible type and length, but then the contents after that point are gibberish. Disturbing.

Over to davidben@ for triage.

Deleted: malformed_hello.pcapng 9.0 KB

malformed_hello.pcapng 9.0 KB Download

 Issue 694072  has been merged into this issue.

 Issue 694077  has been merged into this issue.

The "malformed" SH is expected, as TLS 1.3 is using a different SH format, as expected.

ellyjones: Could you get a net-internals log from that user as well? If it's ERR_SSL_PROTOCOL_ERROR, net-internals should give a more precise error code.

kraush: Hrm, that's interesting. Could you get a net-internals log both against a broken TLS 1.3 server and against a working one? Thanks!
https://dev.chromium.org/for-testers/providing-network-details

kraush: Do you have the experimental token binding support enabled via about:flags?

It looks like some of this issue may be from unofficial builds picking up all the normally unsupported experiment flags.
https://groups.google.com/a/chromium.org/forum/#!topic/chromium-packagers/walQ6-0VjVM

People seeing this on unofficial builds: where did you get your builds from? It sounds like better communication may be needed about that flag.

@David:

"Maximum TLS version enabled" as well as "Token Binding" in about:flags are set to "Default"
Not sure what that would map to.

Can you elaborate on how those flags could be picked up?

My build DID use "base::FieldTrial::DisableByDefault();"
--> Is that a problem / bad idea? (Assuming it even has any effect)

kraush: What kind of a build are you using?

"Default" just means whatever field trials and whatnot set it to.

To elaborate, developer builds of Chromium default to a field trial testing config (rather than actually running field trials) which enables a lot of experimental features. In M56, the combination of TLS 1.3 and tokbind didn't work (tokbind wasn't intended to be shipped in M56). But it seems unofficial builds may have picked up this developer setting and got both enabled.

Thanks for the clarification! :)

It's a Release build. I've listed the GN flags below.

To be clear: We don't actually run field trials (the field trial remote server url is 0'ed out)
Would an official build then just disable all field trials or do release builds also use the local config until they receive a new one from the backend?
If so, is that a bad idea? (Wouldn't we want to use the default config baked into the branch, as that is also what Chrome proper uses?)

GN flags used:
is_debug=false
is_official_build=true
android_channel="stable"
target_os="android"
symbol_level=1
enable_webvr=false

I believe you want to build with fieldtrial_testing_like_official_build=true. This is somewhat confusingly named, but actually disables the fieldtrial testing config. (I'm going to see if I can rename that to something clearer...)

https://groups.google.com/a/chromium.org/forum/#!topic/chromium-packagers/walQ6-0VjVM

Does it do everything else official_build is doing?
If so, would it also be a viable solution to just null out fieldtrial_testing_config?

We've in the past intentionally kept it around to make sure we're not missing any features Chrome ships as an "on-by-default" field trial.

All that flag does is null out fieldtrial_testing_config. It moves every feature to their defaults in code, which is the safer thing to do. fieldtrial_testing_config enables *everything*.

I do not believe there is any option which corresponds to enabling features which get enabled as a field trial. But you're better off using the defaults in code rather than fieldtrial_testing_config.

Gotcha, thanks for explaining all of this David, much appreciated!! :)

Can  issue 693904  be merged over here?  Seems like a similar issue.  Sorry, newbie.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/96d085c8adbd2092696fdc6c622d61a23d5de0bf

commit 96d085c8adbd2092696fdc6c622d61a23d5de0bf
Author: nharper <nharper@chromium.org>
Date: Wed Feb 22 20:10:21 2017

Update max Token Binding version to 13

Drafts 10 thru 13 of Token Binding are all wire compatible, so we can advertise support for draft 13 in Chrome without breaking support with existing servers (or any other code changes than the max version). More importantly, by advertising support for draft 13, we can update servers to only accept draft 13. M56 (which only does draft 10) has a bug where if both TLS 1.3 and Token Binding are enabled and it makes a connection to a server which supports both, the connection fails. Token Binding is turned off in M56, but unofficial builds that weren't built with fieldtrial_testing_like_official_build set properly enabled all experiments, resulting in builds enabling both TLS 1.3 and Token Binding, and hitting this bug.

BUG= 693943 

Review-Url: https://codereview.chromium.org/2712563002
Cr-Commit-Position: refs/heads/master@{#452178}

[modify] https://crrev.com/96d085c8adbd2092696fdc6c622d61a23d5de0bf/net/socket/ssl_client_socket_impl.cc

 Issue 693904  has been merged into this issue.

Isn't it a problem that Google's servers seem to react to this behaviour by sending a malformed Server Hello packet?

http://imgur.com/a/qI5Kf

Starting with Version: Unknown (0x7f12) it looks like gibberish

No, that's TLS 1.3. That version of Wireshark hasn't been updated for TLS 1.3 yet.

Thanks for the fix!
I assume there won't be a 56 dot release with this, but are you planning to merge it into 57?

And is that everything that's required to support draft 13? (No functional change?)
(I know you mentioned drafts 10 and 13 are wire compatible which I assume means they look the same from a client perspective, but I just wanted to make sure! :D)

Oh, I think you misunderstood that commit. It will not be merged to M56. Do not merge that to M56. M56 should not have token binding enabled. Set fieldtrial_testing_like_official_build=true in your M56-based builds and all other versions. That is the fix. (Though I'm going to be working with the field trial folks to make that flag less confusing and communicate this better.)

The intent with bumping to draft 13 is that, in the future, we expect that both TLS 1.3 and tokbind will be enabled. Since there may be other mis-configured unofficial end-user builds of M56 lying around that didn't set fieldtrial_testing_like_official_build that will break, it's simplest to bump the tokbind draft version. Then servers which implement TLS-1.3-draft-18 can ensure they only accept tokbind-draft-13 and higher to avoid worrying about those builds.

(And, of course, in the further future, when final standardized versions of both TLS 1.3 and tokbind exist, this will also not be an issue since the versions will be different.)

Gotcha, thanks!
Wasn't planning on merging it into v56 (we already shipped a version with TLS 1.3 disabled to mitigate the issue, before we knew the exact root cause)
So you're saying we also shouldn't merge it into v57 though?

fieldtrial_testing_like_official_build=true is already set from now on - since we never had a lab repro of this issue though I figured also cherry-picking that commit over to v57 would give us some extra safety. Are you saying that's a bad idea?

That commit is primarily so that token binding (when enabled) can be deployed. There's no need to merge it early since token binding shouldn't be enabled before M58 anyway.

fieldtrial_testing_like_official_build=true is correct to ensure you don't accidentally enable features that shouldn't be enabled on stable and is fine to cherry-pick to other releases, though davidben@ is working on trying to get the flag renamed to something less confusing so other derivatives/unofficial builds will know to set it.

Don't merge it. It's not a fix.

I don't think it would do anything bad (tokbind + TLS 1.3 are not broken in M57), but it definitely wouldn't do anything good. Especially if you set that build flag, it should be a no-op since tokbind will just be disabled.

Gotcha, thanks both of you, won't merge then! :)

Network bug triager here.    fggl0@ can you respond to comment #5?  davidben@, I can't tell from skimming this bug, but if it is well understood enough, can you remove the Needs-Feedback label?

No feedback is needed. I'll go ahead and close this as we've bumped the minimum tokbind version which is the only immediate actionable thing on our end. Longer term we need to make the flag less confusing, but discussion's still pending on what to do there. Probably best to put that under a separate bug when people have decided what color to paint the bikeshed.

 Issue 788496  has been merged into this issue.