New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 638945 link

Starred by 1 user

Issue metadata

Status: Untriaged
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 2
Type: Bug



Sign in to add a comment

QUIC Protocol Error blocking all google domains

Reported by kudiab...@gmail.com, Aug 18 2016

Issue description

UserAgent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2831.0 Safari/537.36

Example URL:

Steps to reproduce the problem:

1. Open Chrome Canary normally (with the chrome://flags/#enable-quic flag set to default)
2. Use the address bar to quick search/Navigate to Google.com/Navigate to any Google domain

What is the expected behavior?
Google domain should load normally when any flags (including chrome://flags/#enable-quic) are set to default

What went wrong?
1. Receive the error: QUIC_Protocol_Error
2. Page does not load

Did this work before? N/A 

Chrome version: 54.0.2831.0  Channel: canary
OS Version: 6.1 (Windows 7, Windows Server 2008 R2)
Flash Version: Shockwave Flash 23.0 r0

The issue seems to be related to the bug:
https://bugs.chromium.org/p/chromium/issues/detail?id=398413
However I do not need to change access points for the issue to occur
 
Components: -Internals>Network Internals>Network>QUIC
Labels: Needs-Feedback
Could you please attach net-internals logs? Instructions: 

https://sites.google.com/a/chromium.org/dev/for-testers/providing-network-details
Status: Available (was: Unconfirmed)

Comment 3 by kudiab...@gmail.com, Aug 19 2016

I have shared the file with you'r email linked here. If there is another email specifically for sharing files please let me know.
Sharable link here for tracking 

https://drive.google.com/file/d/0By3Ng6H6cExTYzlzV1hRdjRzbkk/view?usp=sharing
Cc: ianswett@chromium.org
Thanks very much kudiaborm@ for net-internals.

Comment 5 by rch@chromium.org, Aug 19 2016

Can you please add rch@google.com and/or rch@chromium.org?

Comment 6 by rch@chromium.org, Aug 19 2016

OK, looking at the log we see a number of QUIC sessions set up in which Chrome receives several packets from the server and then no more. It's as if UDP connectivity drops out after .5 seconds. What ISP do you use? What router?

Comment 7 by rch@chromium.org, Aug 19 2016

We notice that you're in a QUIC experiment which we think is working well but we'd like to rule it out. Can you run chrome with the --enable-quic command line argument to force yourself into the default group?

Comment 8 by kudiab...@gmail.com, Aug 19 2016

Thank you for your prompt response. I have added both of your rch accounts.

Using the --enable-quic command line argument when the chrome://flags/#enable-quic is disabled functions. Setting that flag to enabled with and without the command line argument results in the same failure.

As for my router, it isn't beyond unfathomable that QUIC protocal itself is somehow blocked here at work. I attempted to read the log and timeline for something of that nature but didn't see anything.

Was the UDP drop out you were referring to in the QUIC > Connection UID [link] section?

I haven't had a chance to test this issue at home. I can test at home later today (a few hours) 

Comment 9 by rch@chromium.org, Aug 19 2016

Good to know that it's not the experiment.

You can see the lack of UDP connectivity in the Events tab. Pull up event 1136 which is a QUIC_SESSION. Looking just for received packets, you'll see:

t=11977 [st=   80]    QUIC_SESSION_PACKET_RECEIVED
t=12000 [st=  103]    QUIC_SESSION_PACKET_RECEIVED
t=12053 [st=  156]    QUIC_SESSION_PACKET_RECEIVED
t=12078 [st=  181]    QUIC_SESSION_PACKET_RECEIVED

And then nothing else. Meanwhile you'll see chrome sending packets and then eventually retransmitting:

t=12570 [st=  673]    QUIC_SESSION_PACKET_SENT
                      --> packet_number = "5"
...
t=12720 [st=  823]    QUIC_SESSION_PACKET_RETRANSMITTED
                      --> new_packet_number = "8"
                      --> old_packet_number = "5"

I do notice something a bit odd. The requests are being cancelled after a few seconds (possibly by the user or possibly by javascript). Since the requests are cancelled, we also cancel the retransmissions of the packets. This causes the connection to hang around longer that it otherwise would.

Ian, this sounds mildly undesirable. What do you think?
I agree that the lingering seemlingly dead connections are not optimal.  We could certainly close the connection once it may be dead and there were no active requests, but I'd want to think more about when to declare a connection "seemingly dead".
One clarification.  Has Chrome been working fine for you the last few weeks or months on your work network?  Or has this been a persistent issue for a long time?
Chrome Canary has been functioning properly up until this version. I'm not quite sure when QUIC was added/turned on in Canary. The issue began the same day as the bug report though.


Comment 13 by rch@chromium.org, Aug 19 2016

Interesting! We did recently bump to quic v35. Can let's see if that's causing your problems. Can you run with  --enable-quic --quic-version=QUIC_VERSION_34 ? If that still has problems, can you try running Chrome Stable? (If it's tied to a chrome release, then presumably that will still work)

Comment 14 Deleted

I was not able to replicate the issue on my home machine, I can safely say that needlessly excessive 2005 Norton-esque firewall practices are most likely the culprit of QUIC session drops.

But it looks like I was removed from the experiment, setting the chrome://flags/#enable-quic to default now turns it off.
Somewhere between versions:
54.0.2831.0
and
54.0.2833.0 

In addition the --enable-quic --quic-version=QUIC_VERSION_34 command line arguments don't seem to be functioning. No QUIC sessions appear in netinternals with the flags enabled, and quic is marked as disabled in netinternals as well.
The only thing that enables quic is the chrome flag, both on my home computer and at work.
The quic version flag does not set the quic version to 34 as sessions still use version 35.

Would you be able to point me in the direction of documentation for QUIC, so that I can hand it off to IT to get it unblocked when it is ready for primetime. If it gets enabled on stable I suspect the problem to pop up for people less daring than canary users.

QUIC has actually been enabled for most Stable users for about a year now, see the blog post from 2015:  http://blog.chromium.org/2015/04/a-quic-update-on-googles-experimental.html

Which is why it's so odd this has been working well for you for quite a while.  

The Chromium QUIC page is here: https://www.chromium.org/quic

QUIC is about to begin the IETF standardization process, so I would expect to see 
even more QUIC, increasingly from non-Google servers.

Ryan might have some better ideas about why the command line flags aren't working.

Good point on the firewall.  If there are firewalls that are known to cause issues, or cause issues with select ports, it'd be great to know about them.

Is there a reason that on chrome stable would return no QUIC sessions even when setting the chrome://flags/#enable-quic flag to enabled manually?
The only reason I can think of is that all domains are marked "broken".  QUIC will be disabled with exponential backoff if handshakes continue to fail.  

You can see whether domains are marked as broken in the Alt-Svc portion of the chrome://net-internals page.
Interesting. It would appear every domain gets marked broken in Chrome stable, and in Chrome Canary, (I presume) due to the QUIC experiment domains fail to be marked as broken and switch to another protocol and then fail to load.
I just realised that port 443 is blocked by our firewall apparently. 
216.58.217.10:443
vs
216.58.217.10:80 and 216.58.217.10 (with no port defined)

I'm not sure if that helps you at all but it could probably close the bug. Should I open another bug since the command line arguments fail to work (unless I was using them wrong)


Comment 21 by rch@chromium.org, Aug 22 2016

re: command line flags. Yeah, it's pretty surprising that they were not working. I think you are on windows, right? These instructions seem to suggest the "right" way to do things on Windows:

https://www.chromium.org/for-testers/command-line-flags

Is that what you did? But yeah, a follow up bug sounds good.

Comment 22 by rch@chromium.org, Aug 22 2016

re: stable vs canary. Are you saying that in your network you're able to run Stable with QUIC enabled and things load correctly, but when you run Canary they do not?
re: command line flags
I will create a new bug for the command line flags.

re: stable vs canary

Yes that is correct. Stable with the flag set to enable is capable of marking alternate service mappings as broken, where Canary is not capable of doing so. That being said it might be possible that Canary has different permissions than Stable

I'm not sure if chrome stable is jumping from one protocol to another but I have attached a log of the stable behaviour

If you would like me to keep digging I can, but I can't say it's not just a firewall

https://drive.google.com/file/d/0By3Ng6H6cExTMzR2c1FwZ1dORU0/view?usp=sharing

Comment 24 by rch@chromium.org, Aug 22 2016

Hm. So that net-internals (of Stable) shows only 1 QUIC session being marked as broken, beacons5.gvt2.com, which received a reset during the handshake. But other QUIC_SESSIONS have lots of QUIC_SESSION_PACKET_RETRANSMITTED events just before the connection terminates. This looks very much like the net-internals from Canary. So I think this suggests that Canary and Stable behave similarly. But, kudiaborm, I guess that's not what you see?
Apologies, for the second example I only attempted one site where I searched "this is a test" on canary I performed multiple searches and visited multiple domains.

Canary doesn't redirect and still connect after failing, stable does.

Comment 26 by rch@chromium.org, Aug 22 2016

Hm. In that case, can you collect a net-internals from stable and from canary (one after the other, probably) in which you're doing the same activity? I'm not aware of any differences between the two that I would expect to be causing these issues, so I'd really like to see the differences in net-internals.
kudiaborm@gmail.com:  Can you please provide the requested log?  Without it, we can't make forward progress here.
Apologies I've been on vacation.
I can collect them Monday the issue is only on my work machine.
Cc: rch@chromium.org
Monday was labor day woops.
Here are the net-internals from chrome canary and chrome stable as clean as I could get them.

I was unable to save a file after hitting "stop" on the capture page it got stuck at "preparing file" so it might have some extra stuff.

Process:
Close all tabs (except chrome://net-internals)
Reset net-internal capture
open a new tab and type "test" into the awesome bar to quick search.
wait for page to either load or fail for stable and canary respectively
save to file.

I SHOULD NOTE: I have confirmed that UDP port 443 has been blocked by our firewall.

Files here, you should all have access, please request it if not I should be able to respond to that quickly
https://drive.google.com/folderview?id=0By3Ng6H6cExTckZhS0tSZURBcFE&usp=sharing

Thanks for the traces.  I see both traces with similar behavior, where the QUIC handshakes succeed, and then the connections stop receiving packets at between 200-500ms in.

It appears that by blocking 443 the firewall has in fact blocked 443 after some point in a connection, not entirely blocked 443 UDP.  That is definitely causing the issues you're seeing.
Labels: -Needs-Feedback
(Removing Needs-Feedback since the logs were provided.)
Project Member

Comment 33 by sheriffbot@chromium.org, Sep 11 2017

Labels: Hotlist-Recharge-Cold
Status: Untriaged (was: Available)
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available. If you change it back, also remove the "Hotlist-Recharge-Cold" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Sign in to add a comment