It would seem we have a bunch of different leaks? The big ones would be:

media::mojom:Remote* (677 leaks) (with service_manager::mojom::InterfaceProvider leaks most likely a side effect leak)
service_manager::mojom::Connector (321 leaks)
content::mojom::Frame (392 leaks)
content::mojom::ImageDownloader (189 leaks)

Oh, I just realized a flaw in our methodology. Some of these Binding objects live in renderer, and we always just kill renderers.

content::mojom::Frame and content::mojom::ImageDownloader for example are exclusively bound in render processes.

That's true for the binding vs impl side, but as far as freeing objects on the browser side our methodology should be valid?

Ok updated numbers on the browser side only:

Interfaces that are created:

     10 service_manager::mojom::Resolver
     18 service_manager::mojom::Service
     38 rappor::mojom::RapporRecorder
     56 translate::mojom::ContentTranslateDriver
     91 device::mojom::TimeZoneMonitor
     91 metrics::mojom::LeakDetector
     91 startup_metric_utils::mojom::StartupMetricHost
     94 device::mojom::PowerMonitor
     94 discardable_memory::mojom::DiscardableSharedMemoryManager
     99 service_manager::mojom::PIDReceiver
    120 blink::mojom::AppBannerService
    171 ui::mojom::Gpu
    188 chrome::mojom::FieldTrialRecorder
    515 service_manager::mojom::Connector
    706 autofill::mojom::PasswordManagerDriver
    707 autofill::mojom::AutofillDriver
    723 content::mojom::FrameHost
    896 service_manager::mojom::InterfaceProvider
   1319 media::mojom::RemoterFactory
   1330 media::mojom::Remoter

Interfaces that are destroyed:

     19 rappor::mojom::RapporRecorder
     28 translate::mojom::ContentTranslateDriver
     43 startup_metric_utils::mojom::StartupMetricHost
     46 discardable_memory::mojom::DiscardableSharedMemoryManager
     46 metrics::mojom::LeakDetector
     47 device::mojom::TimeZoneMonitor
     48 device::mojom::PowerMonitor
     58 service_manager::mojom::PIDReceiver
     86 ui::mojom::Gpu
     90 chrome::mojom::FieldTrialRecorder
    108 blink::mojom::AppBannerService
    242 service_manager::mojom::Connector
    657 media::mojom::Remoter
    661 media::mojom::RemoterFactory
    711 content::mojom::FrameHost
    712 autofill::mojom::AutofillDriver
    716 autofill::mojom::PasswordManagerDriver
    887 service_manager::mojom::InterfaceProvider

#6 was the numbers for Binding, here are the same numbers for InterfacePtr:

Interfaces that are created:

     24 blink::mojom::PermissionObserver
     97 blink::mojom::AppBannerEvent
    100 blink::mojom::AppBannerController
    198 translate::mojom::Page
    225 web_cache::mojom::WebCache
    230 IPC::mojom::ChannelBootstrap
    233 service_manager::mojom::PIDReceiver
    333 content::mojom::URLLoaderFactory
    412 device::mojom::PowerMonitorClient
    490 service_manager::mojom::Connector
    492 autofill::mojom::AutofillAgent
    493 autofill::mojom::PasswordAutofillAgent
    493 autofill::mojom::PasswordGenerationAgent
    625 content::mojom::FrameFactory
    625 content::mojom::FrameHost
    625 content::mojom::ImageDownloader
    629 content::mojom::Frame
    647 device::mojom::TimeZoneMonitorClient
   2856 service_manager::mojom::InterfaceProvider
   3321 service_manager::mojom::Service
   3784 media::mojom::RemotingSource


Interfaces that are destroyed:

     24 blink::mojom::PermissionObserver
    100 blink::mojom::AppBannerController
    100 blink::mojom::AppBannerEvent
    205 translate::mojom::Page
    225 web_cache::mojom::WebCache
    230 IPC::mojom::ChannelBootstrap
    230 service_manager::mojom::PIDReceiver
    335 content::mojom::URLLoaderFactory
    416 device::mojom::PowerMonitorClient
    475 service_manager::mojom::Connector
    496 autofill::mojom::PasswordAutofillAgent
    496 autofill::mojom::PasswordGenerationAgent
    498 autofill::mojom::AutofillAgent
    627 content::mojom::FrameFactory
    627 content::mojom::FrameHost
    628 content::mojom::ImageDownloader
    629 content::mojom::Frame
    647 device::mojom::TimeZoneMonitorClient
   2835 service_manager::mojom::InterfaceProvider
   3221 service_manager::mojom::Service
   3793 media::mojom::RemotingSource

Two main things worthy of investigation here and are likely signs of real
leaks:

media remoting bindings
service manager connector bindings

will investigate both in more detail

https://bugs.chromium.org/p/chromium/issues/detail?id=682534 will be about the media stuff, let's keep this one for the connector leak.

Bad methodology again - we're double logging in several Binding ctors, leading to incorrect 2x ctor/dtor ratio. There's no Connector leak.

There is however a Service Manager instance leak for renderers, and this could lead to several other dependent message pipes being held open indefinitely, in turn leading to their message queues being kept alive indefinitely.

Could explain all the leaks!

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fe2fecc147dceea5d6445140448d1ae8a53ecfcb

commit fe2fecc147dceea5d6445140448d1ae8a53ecfcb
Author: rockot <rockot@chromium.org>
Date: Thu Jan 19 20:16:19 2017

Always notify Mojo EDK of incomplete content child process launch

There are some cases where RenderProcessHostImpl creates a
ChildConnection - and therefore allocates a partial message pipe
with a corresponding pending reservation for the other end,
associated with a child token - but never launches a child process
or even attempts to use a ChildProcessLauncher.

This is intentional due to the fact that ChildConnection
facilitates IPC Channel creation, and we support horribly
subtle legacy expectations regarding the existence of said
Channel at various points in the RPHI's lifetime.

In such cases, neither mojo::edk::ChildProcessLaunched nor
mojo::edk::ChildProcessLaunchFailed is called for the child
token, as this is normally done by ChildProcessLauncher.
This leaves the child's ServiceRequest pipe reservation pending
indefinitely and thus leaves the corresponding ServiceManager
Instance with an indefinitely open Service pipe, effectively
causing the Service Manager to leak content_renderer Instances.

This CL fixes the problem by having ChildConnection notify the
EDK of launch failure if the ChildConnection is destroyed before
its user provides a child process handle.

BUG= 682459 
R=ben@chromium.org

Review-Url: https://codereview.chromium.org/2645713004
Cr-Commit-Position: refs/heads/master@{#444830}

[modify] https://crrev.com/fe2fecc147dceea5d6445140448d1ae8a53ecfcb/content/browser/browser_child_process_host_impl.cc
[modify] https://crrev.com/fe2fecc147dceea5d6445140448d1ae8a53ecfcb/content/common/service_manager/child_connection.cc
[modify] https://crrev.com/fe2fecc147dceea5d6445140448d1ae8a53ecfcb/content/common/service_manager/child_connection.h

Yup that fixes that problem for me, thanks! I'll mark verified since I don't think the test team can verify this one.

 Issue 682534  has been merged into this issue.

we should get this fix in M-56 if there is still time - josafat@ ?

Should be a trivial merge, so if I can get the approval I'll land it ASAP
on 2924.

Your change meets the bar and is auto-approved for M56. Please go ahead and merge the CL to branch 2924 manually. Please contact milestone owner if you have questions.
Owners: amineer@(clank), cmasso@(bling), gkihumba@(cros), bustamante@(desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/99fdcd508bf021c25171f9e4c411a8e4a3f6eaed

commit 99fdcd508bf021c25171f9e4c411a8e4a3f6eaed
Author: Ken Rockot <rockot@chromium.org>
Date: Fri Jan 20 21:23:09 2017

Always notify Mojo EDK of incomplete content child process launch

There are some cases where RenderProcessHostImpl creates a
ChildConnection - and therefore allocates a partial message pipe
with a corresponding pending reservation for the other end,
associated with a child token - but never launches a child process
or even attempts to use a ChildProcessLauncher.

This is intentional due to the fact that ChildConnection
facilitates IPC Channel creation, and we support horribly
subtle legacy expectations regarding the existence of said
Channel at various points in the RPHI's lifetime.

In such cases, neither mojo::edk::ChildProcessLaunched nor
mojo::edk::ChildProcessLaunchFailed is called for the child
token, as this is normally done by ChildProcessLauncher.
This leaves the child's ServiceRequest pipe reservation pending
indefinitely and thus leaves the corresponding ServiceManager
Instance with an indefinitely open Service pipe, effectively
causing the Service Manager to leak content_renderer Instances.

This CL fixes the problem by having ChildConnection notify the
EDK of launch failure if the ChildConnection is destroyed before
its user provides a child process handle.

BUG= 682459 
R=ben@chromium.org

Review-Url: https://codereview.chromium.org/2645713004
Cr-Commit-Position: refs/heads/master@{#444830}
(cherry picked from commit fe2fecc147dceea5d6445140448d1ae8a53ecfcb)

Review-Url: https://codereview.chromium.org/2644843007 .
Cr-Commit-Position: refs/branch-heads/2924@{#822}
Cr-Branched-From: 3a87aecc31cd1ffe751dd72c04e5a96a1fc8108a-refs/heads/master@{#433059}

[modify] https://crrev.com/99fdcd508bf021c25171f9e4c411a8e4a3f6eaed/content/browser/browser_child_process_host_impl.cc
[modify] https://crrev.com/99fdcd508bf021c25171f9e4c411a8e4a3f6eaed/content/common/service_manager/child_connection.cc
[modify] https://crrev.com/99fdcd508bf021c25171f9e4c411a8e4a3f6eaed/content/common/service_manager/child_connection.h