Just to add more info about this issue.

Tested the issue using #63.0.3239.0 on Mac 10.12.6, Win 10 and Linux Ubuntu 14.04 and could reproduce the issue as per the steps mentioned in comment #0.

Note: Same behavior is seen in M50 as well by launching Chrome using above mentioned command. Observed new tab is opening on mid clicking or clicking open link in new tab.

Thanks!!

I may have found a lead here - when we create a new WebContents with a brand new SiteInstance, we always call SiteInstance::GetProcess() (1) *before* calling SiteInstance::SetSite (2).  If the site url is not set, then process reuse will be prevented by IsSuitableHost origin lock checks (3).

(1a): When middle-clicking as in the repro steps above:

#2 0x7f42a2353f94 content::RenderProcessHostImpl::GetProcessHostForSiteInstance()
#3 0x7f42a25d37bd content::SiteInstanceImpl::GetProcess()
#4 0x7f42a266f0b6 content::WebContentsImpl::Init()
  scoped_refptr<SiteInstance> site_instance = params.site_instance;
  if (!site_instance)
    site_instance = SiteInstance::Create(params.browser_context);
  ...
  if (main_frame_widget_routing_id == MSG_ROUTING_NONE) {
    view_routing_id = main_frame_widget_routing_id =
        site_instance->GetProcess()->GetNextRoutingID(); <- GETTING A PROCESS WITH NULL SITE
  }
#5 0x7f42a265def9 content::WebContentsImpl::CreateWithOpener()
#6 0x7f42a265db7c content::WebContents::Create()
#7 0x5601c71aee48 (anonymous namespace)::CreateTargetContents()
#8 0x5601c71ad29c chrome::Navigate()
#9 0x5601c71855d6 Browser::OpenURLFromTab()
#10 0x7f42a2677e57 content::WebContentsImpl::OpenURL()
#11 0x7f42a1db21aa content::NavigatorImpl::RequestOpenURL()
#12 0x7f42a1dc2d35 content::RenderFrameHostImpl::OnOpenURL()
...

(1b): When running ProcessReuseVsBrowsingInstance from https://crrev.com/c/764487:

#2 0x7f077657af94 content::RenderProcessHostImpl::GetProcessHostForSiteInstance()
#3 0x7f07767fa7bd content::SiteInstanceImpl::GetProcess()
#4 0x7f0776896036 content::WebContentsImpl::Init()
#5 0x7f0776884e79 content::WebContentsImpl::CreateWithOpener()
#6 0x7f0776884afc content::WebContents::Create()
#7 0x000001e795c1 content::Shell::CreateNewWindow()
...


(2):

#2 0x7f6d7a74c4ab content::SiteInstanceImpl::SetSite()
#3 0x7f6d79f1fbc6 content::NavigationRequest::OnResponseStarted()
#4 0x7f6d7a1c0311 content::NavigationURLLoaderImpl::NotifyResponseStarted()


(3): 

#2 0x7f045b3db648 content::RenderProcessHostImpl::IsSuitableHost()
  auto lock_state = policy->CheckOriginLock(host->GetID(), site_url);
  if (lock_state !=
      ChildProcessSecurityPolicyImpl::CheckOriginLockResult::NO_LOCK) {
    // If the process is already dedicated to a site, only allow the destination
    // URL to reuse this process if the URL has the same site.
    if (lock_state !=
        ChildProcessSecurityPolicyImpl::CheckOriginLockResult::HAS_EQUAL_LOCK)
      return false;  <- THE PROCESS IS LOCKED, BUT SITE INSTANCE IS NOT (YET)
  } else if (...
#3 0x7f045b3dc022 content::RenderProcessHost::GetExistingProcessHost()
#4 0x7f045b3dd3c4 content::RenderProcessHostImpl::GetProcessHostForSiteInstance()
#5 0x7f045b65c83d content::SiteInstanceImpl::GetProcess()
#6 0x7f045b6f8136 content::WebContentsImpl::Init()
#7 0x7f045b6e6f79 content::WebContentsImpl::CreateWithOpener()
#8 0x7f045b6e6bfc content::WebContents::Create()
#9 0x55c8f7474e48 (anonymous namespace)::CreateTargetContents()
#10 0x55c8f747329c chrome::Navigate()
#11 0x55c8f744b5d6 Browser::OpenURLFromTab()
#12 0x7f045b700ed7 content::WebContentsImpl::OpenURL()
#13 0x7f045ae3b1aa content::NavigatorImpl::RequestOpenURL()
#14 0x7f045ae4bd35 content::RenderFrameHostImpl::OnOpenURL()
...

I have a CL in progress to fix this https://chromium-review.googlesource.com/c/chromium/src/+/794952.

Removing Needs-triage-Mobile, as we don't target --site-per-process on mobile at this time.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/deb6c7eade28984a7952134532e28998dca1c8f0

commit deb6c7eade28984a7952134532e28998dca1c8f0
Author: Nasko Oskov <nasko@chromium.org>
Date: Thu Nov 30 18:18:11 2017

Respect the process limit on same-site new tabs.

When over the process limit, same-site navigations in new tabs
(ctrl+click, middle click, context menu "open in new tab") should
try to reuse an existing process suitable for hosting the initial
target of the navigation. This CL fixes the current behavior by
creating a SiteInstance for the URL the new tab is created with.

Bug:  774723 
Change-Id: I2e35b7bc1d62826a9a220f2e021c6119d9abcccf
Reviewed-on: https://chromium-review.googlesource.com/794952
Reviewed-by: Avi Drissman <avi@chromium.org>
Reviewed-by: Scott Violet <sky@chromium.org>
Reviewed-by: Łukasz Anforowicz <lukasza@chromium.org>
Reviewed-by: Devlin <rdevlin.cronin@chromium.org>
Commit-Queue: Nasko Oskov <nasko@chromium.org>
Cr-Commit-Position: refs/heads/master@{#520605}
[modify] https://crrev.com/deb6c7eade28984a7952134532e28998dca1c8f0/chrome/browser/chrome_navigation_browsertest.cc
[modify] https://crrev.com/deb6c7eade28984a7952134532e28998dca1c8f0/chrome/browser/extensions/api/web_request/web_request_apitest.cc
[modify] https://crrev.com/deb6c7eade28984a7952134532e28998dca1c8f0/chrome/browser/tab_contents/tab_util.cc
[modify] https://crrev.com/deb6c7eade28984a7952134532e28998dca1c8f0/chrome/test/ppapi/ppapi_browsertest.cc
[modify] https://crrev.com/deb6c7eade28984a7952134532e28998dca1c8f0/content/browser/site_instance_impl.cc
[modify] https://crrev.com/deb6c7eade28984a7952134532e28998dca1c8f0/content/browser/site_instance_impl.h
[modify] https://crrev.com/deb6c7eade28984a7952134532e28998dca1c8f0/content/public/browser/site_instance.h

This should be fixed. Feel free to file a new bug if something isn't working correctly.