This is typically a very brief edge case that we log (in case it causes problems) but so far have been able to ignore.

Is this state either not brief or causing problems? Please describe the specific problem.

If it is now causing problems, we may need to remove an idle Ethernet network from the network list. You should be familiar enough with the code now to fix this, or maybe pass it along to an Arc dev. I am in the process of trying to work through my bug list so that I can focus on mus+ash work.

The specific problem is that ARC++ apps no longer have access to the Internet due to this bug. ARC++ relies on the DefaultNetworkChanged() callback to route the connection, and since this bug causes the new default network to be null, ARC++ thinks that there is no connection.

Okay, I can look into fixing this for M-63. So, the correct solution is removing idle Ethernet networks from the network list? Why is the network still even present in this case? I would expect the network to be gone entirely (not just idle) when the cable is unplugged from the device.

Please clarify "no longer have access to the Internet". DefaultNetworkChanged() should be called when the Tether network correctly becomes the default network. i.e. the default network may briefly be Ethernet+idle, but should immediately become the correct default network.

If that is not happening then this is a Tether bug, i.e. the Tether code should make sure to call DefaultNetworkChanged() when it becomes the default network.

Also, the Ethernet network should go away, the "idle" state should be very brief (as in, a frame or two) does it not? This is trivial to reproduce, yes?

By "no longer have access to the Internet," I mean that ARC++ apps think that they are offline, even though the Chrome browser does have network access.

The situation what you described (default network correctly changes) is what is *supposed* to happen, but that does not end up happening. The way that this is supposed to happen is that once Ethernet becomes disconnected, we rely on Shill to promote the underlying Wi-Fi connection for the Tether network to the new default network, then when we get this update, we look to see if this network corresponds to a Tether network. If it does, we switch the default network to be the Tether network instead of the underlying Wi-Fi network.

This never occurs in practice, though. We just get this "unexpected state" error and the default network remains null without ever switching over.

Yes, this is trivial to reproduce.

[1] https://cs.chromium.org/chromium/src/chromeos/network/network_state_handler.cc?l=1404-1412

OK, so the bug is not the "unexpected state", the bug is that DefaultNetworkChanged() is not getting called with the WiFi network after Ethernet is disconnected (I assume that Ethernet is actually going away though?).

If this is a Shill issue, I'm surprised this hasn't caused problems before. It seems more likely that this is an artifact of the recent NetworkStateHandler changes.

FWIW, on a Pixel 1 without a Tether network I am seeing DefaultNetworkChanged events for Ethernet then GoogleGuest when I remove the Ethernet connection, and no "unexpected state" error. I would definitely suggest testing without Tether first to establish a baseline, using a standard WiFi network and using a hospot without Tether.

Good point - I've renamed the bug accordingly.

Okay, will try that. Thanks for the advice.

This actually has come up before, presenting itself in a different way (see  crbug.com/738542 ).

Hope this is helpful!

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/23a2216be7192733d86f5cf2fdac401ab96b6f65

commit 23a2216be7192733d86f5cf2fdac401ab96b6f65
Author: Kyle Horimoto <khorimoto@google.com>
Date: Thu Dec 07 01:50:54 2017

[CrOS Tether] Notify default network changes for Ethernet -> Tether.

Previously, NetworkStateHandler contained the following bug:
  (1) Connect Ethernet network (the default network).
  (2) Connect Tether network (Ethernet is still default).
  (3) Disconnect Ethernet network (Tether should be the default).
  (4) NSH notifies that the new default is null, which is incorrect.

This issue was caused by an early return in
NetworkStateHandler::DefaultNetworkServiceChanged() which prevented the
default network change event from being propagated.

This CL fixes an issue which prevents ARC++ apps from utilizing a Tether
connection, since ARC++ relies on default network change events to
forward network connections to Android apps.

Bug:  766933 , 672263
Change-Id: Ib4824bfef1d3e7faa86b631a7cc87870c818ab42
Reviewed-on: https://chromium-review.googlesource.com/809535
Commit-Queue: Kyle Horimoto <khorimoto@chromium.org>
Reviewed-by: Steven Bennetts <stevenjb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#522299}
[modify] https://crrev.com/23a2216be7192733d86f5cf2fdac401ab96b6f65/chromeos/network/network_state_handler.cc
[modify] https://crrev.com/23a2216be7192733d86f5cf2fdac401ab96b6f65/chromeos/network/network_state_handler_unittest.cc

Actually, since this is a non-essential bugfix, let's just wait for M-65.