New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 24 users
Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug

Blocked on:
issue 344685



Sign in to add a comment
Can't reach private IPv6 hosts on networks without global IPv6 connectivity
Reported by bkans...@firstgaming.com, Sep 11 2015 Back to list
UserAgent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36

Example URL:
http://h.searx.info/

Steps to reproduce the problem:
1. Chromium 45, 46
2. --enable-ipv6 switch
3. Private IPv6 network without global IPv6 connectivity
4. URL with domain pointing to IPv6 address in private network

What is the expected behavior?
URL is opening or something like ERR_CONNECTION_REFUSED

What went wrong?
ERR_NAME_NOT_RESOLVED

Did this work before? N/A 

Chrome version: 45.0.2454.85  Channel: stable
OS Version: 4.1.0-1-amd64
Flash Version: Shockwave Flash 18.0 r0

Worked on Chromium 44.0.2403.89. On Iceweasel and Firefox it works.
URl with literal ipv6 address also works.
 
44.0.2403.89.net-internals-log.json
274 KB Download
45.0.2454.85.net-internals-log.json
55.2 KB Download
Cc: mmenke@chromium.org
Owner: rtenneti@chromium.org
Status: Assigned
Will investigate asap. Thanks for the report.
Comment 3 by mmenke@chromium.org, Sep 11 2015
We removed the command line flag.  See  issue 344685 .
Blockedon: chromium:344685
Status: WontFix
Thanks mmenke.
So there is no way to use DNS for private IPv6 addresses in Chromium anymore?
Comment 6 by mmenke@chromium.org, Sep 11 2015
Cc: ttuttle@chromium.org pauljensen@chromium.org eroman@chromium.org
Labels: -Cr-Internals-Network Cr-Internals-Network-DNS
No, there's not.  I think it's worth asking if we care about this case, and if so, how we should support it.  Relying on command line flags for it seems rather bad, but we obviously don't want to slow down DNS resolutions for everyone on IPv4-only networks just to support this one case.
Is this an issue with Chrome's internal resolver?  Does disabling Chrome's internal resolver address the issue?  I'm never quite sure how AI_ADDRCONFIG reacts (Does it ignore loopback addresses? I think yes.  Does it ignore link-local addresses? I think no.).
Comment 8 by mmenke@chromium.org, Sep 11 2015
Cc: rtenneti@chromium.org
Owner: ----
Status: Available
Hrm...I was thinking the issue was that our IPv6 probe determines there's no IPv6 support, since it just checks support with a remote address.  That would affect both the internal and system resolver.
True, our IPv6 probe will fail without global IPv6 connectivity and then mask out any IPv6 results too.
Comment 10 by Deleted ...@, Oct 13 2015
Other solution would be to allow this address to be configurable.

So in case of local network you can specify IPv6 availability DNS server.
Labels: -OS-Linux OS-All Cr-Internals-Network-Connectivity
There's a pretty high bar for adding settings UI to Chrome, unfortunately, and I don't think this meets the bar.

We could try IPv6 if IPv4 fails, even if we don't think we have IPv6 connectivity.  That could significantly delay time until error pages are seen for people for whom IPv6 is badly broken, but I think that's the only downside (Other than extra code complexity).
Comment 12 by pmarks@google.com, Oct 24 2015
If a DNS A request responds with "no data" (which is distinct from NXDOMAIN), the resolver could try again with AAAA, even if global IPv6 connectivity is unavailable.

This should have zero impact on IPv4-only users, until they try to contact a server without an IPv4 address.

(Alternatively, restarting a connection with IPv6 enabled, after any kind of DNS+TCP failure, would catch more edge cases but incur longer timeouts.)
There's no way to tell if the response is "no data" while using the system resolver, to the extent of my knowledge.  The async resolver is basically disabled everywhere buy ChromeOS, since it doesn't quite match behavior of the system resolver.
Cc: -ttuttle@chromium.org juliatut...@chromium.org
Summary: Can't reach private IPv6 hosts on networks without global IPv6 connectivity (was: --enable-ipv6 switch is broken on Chromium 45)
 Issue 584028  has been merged into this issue.
Owner: pauljensen@chromium.org
Status: Assigned
Can this: https://cs.chromium.org/chromium/src/net/dns/host_resolver_impl.cc?l=2367 check just be removed and let Happy Eyeballs deal with misbehaving IPv6 connectivity?
I believe the reason for this is to save us the IPv6 resolve, which may (?) take a long time on non-IPv6 networks. 
I'm not sure why a AAAA query over IPv4 transport would take a long time?
Because IPv6-unaware dns servers can leave much to be desired, particularly those in home routers or captive portals.
Err...AAAA-unaware, rather.
Surely networks that are using recursors with that broken behaviour, are not handing out IPv6 GUA or ULA addresses, so the point is moot?  
(If the end host doesn't have a non-link-local IPv6 address, it shouldn't send a AAAA query)
@richard: The check you wanted to remove in comment #18 is the one that prevents Chrome from sending AAAA queries when Chrome detects no IPv6 connectivity, which if I'm not mistaken is what you proposed in comment #23.
Could we fix this bug by detecting the special case where the host has only site-local (private) (fec0::/48) IPv6 addresses and disabling the IPv6 connectivity probe?  Seems like DnsConfigService could set DnsConfig::use_local_ipv6 in this case to accomplish disabling the probe.
I'm not sure.

I also wonder if determining if we have only site-local addresses is a blocking operation or not - "connecting" UDP sockets presumably doesn't tend to block (much?), so we can do it pretty cheaply.  Getting local IP addresses is generally done off thread, I believe, implying it can be a blocking operation.
@paul Sorry you're right, the reference was too high-level.  Removing the dummy socket creation here: https://cs.chromium.org/chromium/src/net/dns/host_resolver_impl.cc?rcl=1484267005&l=231 is probably better suited as that's what breaks when there's no default route. Leave the link-local and teredo checking below it intact. 

However then the function name is a bit misleading and should change.
@mmenke: My proposal was "DnsConfigService could set DnsConfig::use_local_ipv6 in this case", so it would be calculated very occasionally when DnsConfigService detected an IP address change (IIRC DnsConfigService already recalculates on OnIPAdddressChanged) and recomputes the DnsConfig, not at every resolution time like our IPv6 connectivity probe.  That being said, getting the local IP addresses is a also essentially non-blocking (just a few syscalls like our IPv6 connectivity probe (side note: on Android these syscalls are actually IPCs to netd...)); I believe libc's getaddrinfo() implementation calls getifaddrs() to implement AI_ADDRCONFIG.

@richard: When you connect a UDP socket on linux it checks that there is a route from the local machine to the peer; I think the kernel runs a dummy packet through the routing logic.  So the result of the Connect() call is the main component of our IPv6 connectivity check; it answers the question "do we have a global IPv6 route?".  I don't think we can remove it without breaking that function's functionality.
@paul:  That's exactly what we want to remove.  At least for this "Should I do a AAAA query or not" check. 

If after it receives a valid AAAA response, then Chrome's Happy Eyeballs implementation should make that "do I have global connectivity" decision by sending SYNs over both IPv6 and v4 simultaneously, not a presumptuous dummy UDP socket check to an arbitrary hardcoded IPv6 address.  

If this function is used elsewhere, perhaps the functionality can be split out into two?  One for "Do I have a non-link-local IPv6 address", and another for "Do I have an IPv6 default route" (if that's indeed needed)
Comment 30 by pmarks@google.com, Jan 13 2017
> "do I have global connectivity" decision by sending SYNs over both IPv6 and v4 simultaneously, not a presumptuous dummy UDP socket

You're describing Happy Eyeballs, which should never be *required* unless the network is broken (e.g. it advertises a route to ::/0 but blackholes the packets).  The primary "do I have a route?" decision occurs during the RFC6724 sort, which works by UDP connect()ing to each address listed in DNS.

I think it sounds reasonable to enable AAAA lookups whenever a non-{::1,fe80::,Teredo} address is listed in getifaddrs(), instead of looking for a default route.
> You're describing Happy Eyeballs, which should never be *required* unless the network is broken (e.g. it advertises a route to ::/0 but blackholes the packets).

Or if IPv6 is drastically more latent.

> The primary "do I have a route?" decision occurs during the RFC6724 sort, 

Yup

> which works by UDP connect()ing to each address listed in DNS.

and *before* it even attempts to do a DNS query, when this: https://goo.gl/WI52ZA connect()s to the hardcoded Google Public DNS recursor: https://goo.gl/XRte7E   Which there-in lies the problem.

> I think it sounds reasonable to enable AAAA lookups whenever a non-{::1,fe80::,Teredo} address is listed in getifaddrs(), instead of looking for a default route.

Excellent, that sounds like a valid solution.  Will still require adjusting the above though.


If DNS returns an AAAA record, and doesn't return an A record, the application should attempt a connection to the AAAA result, and inform the user that it couldn't connect if that fails, rather than informing the user that DNS lookup failed, which is false. The current behavior is extremely confusing when debugging, because it points a finger at DNS when DNS is fine. Don't lie to the user, and don't give up without trying.

I'm on a v4 connection right now, with a v6-enabled VPN tunnel, and I can't get to sites that I have a perfectly valid AAAA record for and route to. Trusting the network stack would fix this. Happy Eyeballs would fix this. There's an obviously correct behavior here that doesn't require testing anything.
Note that DNS doesn't return a AAAA records in this case, since we only do A lookups when we think AAAA is unavailable.  AAAA lookups on some intolerant networks just timeout, so if we delay A lookups on those networks until AAAA is complete, that can lead to performance issues.
In that case, if A lookup fails, you should try an AAAA lookup, in case there's a partial v6 configuration or your test endpoint is firewalled. That would allow correct behavior on correct partial v6 setups like VPNs with no impact on performance for the broken configurations you're attempting to accommodate.
Comment 35 by pmarks@google.com, Mar 22 2017
csnook: That's similar to what I suggested in Comment 12, but with "no data" replaced by "any failure".

That means IPv4-only users navigating to nonexistent domain names will incur an extra DNS round trip, which doesn't sound too bad to me.
So now that I understand what the problem actually is, how about first fixing the error reporting.

Here's how I got here:

* local domain .dev didn't resolve, but works in all other tools and browsers
* changed it to .test since that is what I should be using in the first place
* no change in behavior
* tcpdump on port 53 on loopback interface => chksum errors. Research that first. Red herring: other lookups also display chksum errors but answers are accepted by chrome
* changed CNAME record to AAAA record to simplify the setup
* finally noticed when I isolated the request/response cycle in tcpdump that there was no AAAA lookup tried
* searched and got here via earlier bug report

So, my use case is that I simply run local development on [::1] (it's shorter, sockstat -6l lists all dev stuff, there's a plethora of reasons for it). This whole mechanism of testing for ipv6 routing is bogus to begin with as the vast majority *nix machines out there will have [::1] whether or not global or neighborhood IPv6 connectivity exists.

It's disturbing to see that code complexity is posted as a negative for fixing this, while instead of simply trusting the OS to the right thing and use `getaddrinfo(3)` and friends, complex code has been introduced to do things differently. I do understand the performance issues, but they are not actually that severe and in total save me less time then this whole chase into the rabbit hole has cost me, for a setup that is in fact completely correct!

So, the proposed fix:

> I think it sounds reasonable to enable AAAA lookups whenever a non-{::1,fe80::,Teredo} address is listed in getifaddrs()

would still bite me, unless I put up a bogus interface with a bogus address that falls outside above scopes. Or switch to 127.0.0.1 (except that I'm not the one with the flawed setup).
Owner: ----
Status: Available
We're having the same issue in our organisations IPv6-only VPN. We don't route all traffic through the VPN but only have a route into our internal network. All our internal devices have DNS names with only AAAA records.
Clients which don't have IPv6 connectivity through their Internet connection trigger this issue in Chrome. Accessing internal sites via Firefox and using Samba shares works just fine due to them using the OS's resolver. A solution like mentioned in Comment #34 would solve this without any negative impacts.

Our current workaround is to push a route to 2001:4860:4860::/112 to the VPN client to fool Chrome into thinking it has full IPv6 connectivity. It doesn't cause any noticeable slowdowns so far but we'd still prefer a cleaner solution.
Cc: mge...@chromium.org
Adding Miriam who may be interested in this DNS issue.
Cc: -juliatut...@chromium.org
I can't use chrome/chromium in my ULA IPv6 network without global connectivity because of this, and I just refuse to add a fake route to cover for an issue that is made up in the first place! 
This is of course not a show stopper in any way as Edge and Firefox works flawlessly, but I am disappointed to realize that I must keep away from chrome/chromium for IPv6 networks when I thought google was actually encouraging IPv6 deployments, offline or not.

Everything else on my computer resolves A and AAAA records in parallell and I get that this browser is used by an insane amount of people that knows nothing about computers, so you optimize for that, but I find it strange that chromium is the only thing not handling this without massive time delays.

My vote goes to completely removing this feature. It is not logical because nothing else behaves like this, it slows down working IPv6 deployments by encuraging bad or non-functional setups, and I see no real reason to mess this up and not behave like every other browser in this aspect. Also it wastes time for some of us!
Sign in to add a comment