For reference, this test will start timing out, with --site-per-process only, after Pavel lands his https://chromium-review.googlesource.com/c/535022/ to kick devtools extensions out of process.

The test times out while navigating to chrome-devtools://devtools/bundled/inspector.html in a regular tab.  I think the root cause is some confusion about WebUI bindings for devtools pages.  

We start out by forcing a BrowsingInstance swap because ShouldSwapBrowsingInstancesForNavigation thinks the dest URL requires WebUI:

    if (WebUIControllerFactoryRegistry::GetInstance()->UseWebUIBindingsForURL(
            browser_context, new_effective_url)) {
      return true;
    }

The WebUIFactoryFunction needed is DevToolsUI.  

Next, we create a pending RenderFrameHost and call UpdatePendingWebUI() on it.  This should normally grant the necessary WebUI bindings, except that for DevToolsUI, the bindings are 0:

DevToolsUI::DevToolsUI(content::WebUI* web_ui)
    : WebUIController(web_ui), bindings_(web_ui->GetWebContents()) {
  web_ui->SetBindings(0);
  ...

This causes UpdatePendingWebUI to actually skip the call AllowBindings here (which sets them on ChildProcessSecurityPolicy), because enabled bindings (0) match the new ones:
    int new_bindings = pending_web_ui_->GetBindings();
    if ((GetEnabledBindings() & new_bindings) != new_bindings) {
      AllowBindings(new_bindings);

Next, the navigation proceeds, we hit WillProcessResponse and examine whether there should be a transfer in IsRendererTransferNeededForNavigation, for a rfh->GetSiteInstance()->GetSiteURL() of "chrome-devtools://devtools/" (pending RFH) and dest_url of "chrome-devtools://devtools/bundled/inspector.html".

Before Pavel's CL, there was no transfer because we just returned early for the chrome-devtools:// scheme.  Now that this hack is removed, we fall through.  IsCurrentlySameSite (incorrectly) returns *false* even though dest_url matches the current SiteInstance.  This happens because we go to IsCurrentlySameSite -> HasWrongProcessForURL -> RPHI::IsSuitableHost, which returns false here:
  if (ChildProcessSecurityPolicyImpl::GetInstance()->HasWebUIBindings(
          host->GetID()) !=
      WebUIControllerFactoryRegistry::GetInstance()->UseWebUIBindingsForURL(
          browser_context, site_url)) {
    return false;
  }

I.e., we haven't assigned bindings for the pending RFH's process in ChildProcessSecurityPolicy because they weren't needed for this WebUI type, yet UseWebUIBindingsForURL still thinks bindings are needed for this URL.  

Later, with --site-per-process, DoesSiteRequireDedicatedProcess returns true and forces the transfer to happen when it really shouldn't.  The transfer then hangs (haven't tried to understand why).

Interestingly, all this works with PlzNavigate, so might be another bug where we just wait for PlzNavigate to ship. :)

The test is fairly new, added in r476937.  I don't think this is the test's fault though: the same thing seems to happen in regular Chrome with --site-per-process enabled just by navigating a regular tab to chrome-devtools://devtools/bundled/inspector.html from the omnibox.  I suspect it didn't work before we added the RFHM hack for devtools extensions in r460876, and the test just came to rely on the hack before we could remove it.

Also, the SetBindings(0) is only used by DevTools, so this shouldn't affect other WebUI pages.

Wow, thanks for looking into it.

It seems like RenderProcessHostImpl::IsSuitableHost assumes nobody would do SetBindings(0) and DevTools explicitly violates that assumption. There is also a TODO from nick on the same block saying:

// TODO(nick): Consult the SiteIsolationPolicy here.  https://crbug.com/513036 

which makes sense to me - why would we use non-site-isolation-policy hints on determining the isolation...