Thank you, Erik. Here is the full list. More info: I highly suspect Chrome Remote Desktop, as I manually uninstalled and reinstalled it just before starting to observe the issues. Also, I am now removing two extensions that shouldn't be even installed on my machine: Solitaire and the Economist. These have been there for a while, though.

Automatic Room Finder 1.4.3
BeyondCorp 4.8
Chrome RDP (Internal) 4.5.1.23
Chrome Remote Desktop 51.0.2704.53
CL Monitor 0.51
gnubbyd 0.9.42
Google Art Project 1.0.37 (Disabled)
Google Cast 15.1120.0.4
Google Corporate Extension Reporter 1.7.2
Google Docs 0.9
Google Docs Offline 1.4
Google Hangouts 2016.617.425.3
Google OpenVPN NG (Management Client) 2.4.25
Google OpenVPN NG (User Interface) 2.4.25
Google Sheets 1.1
Google Slides 0.9
Google Tone 2.0.6
Password Alert 1.22
Secure Shell 0.8.34
Shortn 1.0.2
Solitaire 1.4.4
The Economist 1.0.33.3

Assigning to sergeyu to investigate further. 

+rdevlin just as a heads up, and to possibly provide more information about how to trace down an extension that's acting up.

Can you please provide more information about the symptoms? Is it browser or renderer process hanging? Looks like the trace file is for renderer, but the bug title says it's a browser. If you see a renderer handing, then what url/extension does it run?

After ~30 min of usage the browser hangs. It doesn't matter what pages are open. There is also correlation with memory usage. In one instance it hanged after eating all of my memory. I have temporarily disabled the Chrome Remote Desktop extension to see if this fixes the problem. The issue impacts only the stable version of Chrome (so far). The canary seems to be functional.

One more piece of information - it is possible that the renderer uses enough memory to hang the browser. If this is the case, the only extension that has an open window is the Hangouts.

Is it renderer or browser process that eats memory? You can open the task manager and keep it running to see PIDs of each chrome process when it hangs. Also it would be useful to know which process, if any, eats CPU when the hang happen.
The trace that Eric attached is for a renderer, but it's not clear if that renderer process is related to the hang, and in general renderers shouldn't be able to hang the browser. So I'm sure if this bug has anything to do with extensions at all - it looks like a red herring to me. Also CRD app shouldn't be running unless you click on the icon after restarting the browser. I don't think I'm the right person to investigate this issue.

Extension renderers report error logs to the browser for the error console and developer debugging.  As such, if an extension renderer is truly misbehaving and spewing thousands/millions of logs, it could probably hang the browser.  This is bad, but, realistically, is just one of many ways an extension could hang the browser process (an extension could also hang or crash it by saying "for (i = 0; i < 100000000; ++i) { chrome.tabs.create(); }).  Ideally we'd fix all of these, but, as always, resources are a constraint.  However, that does mean that if CRD (or another extension) totally misbehaves, we could see behavior similar to that described here.

The hanging process is the browser, not the renderer. It is likely that I have provided the wrong stack trace to Erik. This time I got a sample from the Chrome process that Activity Monitor reports as "not responding". See the attached file.

Additionally, I have disabled the Chrome Remote Desktop extension and the hang still occurs. This extension is not to blame. I am now disabling hang outs for an experiment.

Please check the updated sample and let me know if it makes more sense. The experience is that Chrome hangs altogether - all browser windows, including Chrome Task manager hang with the cursor changing to the wait cursor. Next time it happens, I will also search the syslog for the processes that hang.

Deleted: Browser hang sample.txt 135 KB

Browser hang sample.txt 135 KB View Download

Attached symbolicated sample.

Main thread is waiting forever on a sync IPC, which the browser process should never be issuing. Looks like VerifySyncTokensCHROMIUM can hang.
"""
    +                                           2505 -[TabView mouseDown:]  (in Google Chrome Framework)  load address 0x103ccf000 + 0x35ebceb  [tab_view.mm:300]
    +                                             2505 -[TabStripDragController maybeStartDrag:forTab:]  (in Google Chrome Framework)  load address 0x103ccf000 + 0x35e7ad0  [tab_strip_drag_controller.mm:137]
    +                                               2505 <name omitted>  (in Google Chrome Framework)  load address 0x103ccf000 + 0x369972a  [tab_strip_model.cc:417]
    +                                                 2505 TabStripModel::SetSelection(ui::ListSelectionModel const&, TabStripModel::NotifyTypes)  (in Google Chrome Framework)  load address 0x103ccf000 + 0x3698fa9  [tab_strip_model.cc:1286]
    +                                                   2505 TabStripModel::NotifyIfActiveOrSelectionChanged(content::WebContents*, TabStripModel::NotifyTypes, ui::ListSelectionModel const&)  (in Google Chrome Framework)  load address 0x103ccf000 + 0x369cada  [tab_strip_model.h:348]
    +                                                     2505 TabStripModel::NotifyIfActiveTabChanged(content::WebContents*, TabStripModel::NotifyTypes)  (in Google Chrome Framework)  load address 0x103ccf000 + 0x36996a0  [weak_ptr.h:213]
    +                                                       2505 -[TabStripController activateTabWithContents:previousContents:atIndex:reason:]  (in Google Chrome Framework)  load address 0x103ccf000 + 0x35e3edf  [tab_strip_controller.mm:1331]
    +                                                         2505 -[TabStripController swapInTabAtIndex:]  (in Google Chrome Framework)  load address 0x103ccf000 + 0x35e0e26  [tab_strip_controller.mm:602]
    +                                                           2505 -[NSView addSubview:]  (in AppKit) + 475  [0x7fff909f182f]
    +                                                             2505 -[NSView _setWindow:]  (in AppKit) + 3009  [0x7fff909f4daa]
    +                                                               2505 __21-[NSView _setWindow:]_block_invoke649  (in AppKit) + 169  [0x7fff90a2fd6d]
    +                                                                 2505 -[__NSArrayM enumerateObjectsWithOptions:usingBlock:]  (in CoreFoundation) + 217  [0x7fff83863ac9]
    +                                                                   2505 __53-[__NSArrayM enumerateObjectsWithOptions:usingBlock:]_block_invoke  (in CoreFoundation) + 134  [0x7fff83863c36]
    +                                                                     2505 -[NSView _setWindow:]  (in AppKit) + 3179  [0x7fff909f4e54]
    +                                                                       2505 content::WebContentsImpl::WasShown()  (in Google Chrome Framework)  load address 0x103ccf000 + 0x3bcee76  [web_contents_impl.cc:1257]
    +                                                                         2505 content::RenderWidgetHostViewMac::Show()  (in Google Chrome Framework)  load address 0x103ccf000 + 0x3b0573f  [animation_utils.h:44]
    +                                                                           2505 content::RenderWidgetHostImpl::PauseForPendingResizeOrRepaints()  (in Google Chrome Framework)  load address 0x103ccf000 + 0x3afbaea  [trace_event.h:972]
    +                                                                             2505 content::RenderWidgetHostImpl::WaitForSurface()  (in Google Chrome Framework)  load address 0x103ccf000 + 0x3afbdd3  [render_widget_host_impl.cc:877]
    +                                                                               2505 ui::WindowResizeHelperMac::WaitForSingleTaskToRun(base::TimeDelta const&)  (in Google Chrome Framework)  load address 0x103ccf000 + 0x13ab797  [window_resize_helper_mac.cc:310]
    +                                                                                 2505 base::internal::Invoker<base::IndexSequence<0ul>, base::internal::BindState<base::internal::RunnableAdapter<void (base::CancelableCallback<void ()>::*)() const>, void (base::CancelableCallback<void ()> const*), base::WeakPtr<base::CancelableCallback<void ()> > >, base::internal::InvokeHelper<true, void, base::internal::RunnableAdapter<void (base::CancelableCallback<void ()>::*)() const> >, void ()>::Run(base::internal::BindStateBase*)  (in Google Chrome Framework)  load address 0x103ccf000 + 0x2a111b  [bind_internal.h:206]
    +                                                                                   2505 base::internal::Invoker<base::IndexSequence<0ul>, base::internal::BindState<base::internal::RunnableAdapter<void (cc::DisplayScheduler::*)()>, void (cc::DisplayScheduler*), base::WeakPtr<cc::DisplayScheduler> >, base::internal::InvokeHelper<true, void, base::internal::RunnableAdapter<void (cc::DisplayScheduler::*)()> >, void ()>::Run(base::internal::BindStateBase*)  (in Google Chrome Framework)  load address 0x103ccf000 + 0x3cd4eab  [bind_internal.h:181]
    +                                                                                     2505 cc::DisplayScheduler::OnBeginFrameDeadline()  (in Google Chrome Framework)  load address 0x103ccf000 + 0x3cd2fe1  [display_scheduler.cc:296]
    +                                                                                       2505 cc::DisplayScheduler::DrawAndSwap()  (in Google Chrome Framework)  load address 0x103ccf000 + 0x3cd3b31  [display_scheduler.cc:120]
    +                                                                                         2505 cc::Display::DrawAndSwap()  (in Google Chrome Framework)  load address 0x103ccf000 + 0x3cd1ce2  [memory:2737]
    +                                                                                           2505 cc::SurfaceAggregator::Aggregate(cc::SurfaceId)  (in Google Chrome Framework)  load address 0x103ccf000 + 0x3cd9e8f  [unordered_map:1109]
    +                                                                                             2505 cc::SurfaceAggregator::ProcessAddedAndRemovedSurfaces()  (in Google Chrome Framework)  load address 0x103ccf000 + 0x3cd6d7b  [__hash_table:2334]
    +                                                                                               2505 cc::ResourceProvider::DestroyChildInternal(std::__1::__hash_map_iterator<std::__1::__hash_iterator<std::__1::__hash_node<std::__1::__hash_value_type<int, cc::ResourceProvider::Child>, void*>*> >, cc::ResourceProvider::DeleteStyle)  (in Google Chrome Framework)  load address 0x103ccf000 + 0x1184f6b  [vector:450]
    +                                                                                                 2505 cc::ResourceProvider::DeleteAndReturnUnusedResourcesToChild(std::__1::__hash_map_iterator<std::__1::__hash_iterator<std::__1::__hash_node<std::__1::__hash_value_type<int, cc::ResourceProvider::Child>, void*>*> >, cc::ResourceProvider::DeleteStyle, std::__1::vector<unsigned int, std::__1::allocator<unsigned int> > const&)  (in Google Chrome Framework)  load address 0x103ccf000 + 0x1187646  [resource_provider.cc:1689]
    +                                                                                                   2505 gpu::gles2::GLES2Implementation::VerifySyncTokensCHROMIUM(signed char**, int)  (in Google Chrome Framework)  load address 0x103ccf000 + 0x1800a9b  [gles2_implementation.cc:5903]
    +                                                                                                     2505 gpu::GpuChannelHost::ValidateFlushIDReachedServer(int, bool)  (in Google Chrome Framework)  load address 0x103ccf000 + 0x138284d  [gpu_channel_host.cc:372]
    +                                                                                                       2505 gpu::GpuChannelHost::Send(IPC::Message*)  (in Google Chrome Framework)  load address 0x103ccf000 + 0x1380e57  [gpu_channel_host.cc:125]
    +                                                                                                         2505 IPC::SyncChannel::Send(IPC::Message*)  (in Google Chrome Framework)  load address 0x103ccf000 + 0x1048051  [atomic:882]
    +                                                                                                           2505 IPC::SyncChannel::WaitForReply(IPC::SyncChannel::SyncContext*, base::WaitableEvent*)  (in Google Chrome Framework)  load address 0x103ccf000 + 0x10483b3  [ipc_sync_channel.cc:522]
    +                                                                                                             2505 base::WaitableEvent::WaitMany(base::WaitableEvent**, unsigned long)  (in Google Chrome Framework)  load address 0x103ccf000 + 0x5baf28  [waitable_event_posix.cc:128]
    +                                                                                                               2505 _pthread_cond_wait  (in libsystem_pthread.dylib) + 767  [0x7fff95ffd728]
    +                                                                                                                 2505 __psynch_cvwait  (in 
"""

+piman.

The browser process should never be sending sync IPCs. Should we add a bit to GpuChannelHost that gets set by the browser process (but not by child processes) which enforces that all IPCs sent by the browser process are asynchronous?

There exist sync IPC browser -> GPU (especially when setting up a new channel to the GPU process). The direction that is not allowed is GPU -> browser.

This particular sync is from r352901, and it seems crappy to have the browser process do a sync IPC at tab-switch and tab-close (tab-close is super slow, and makes us look crappy).

That said, it may be that this will all go away when we move the display compositor to the gpu process.

This particular sync doesn't seem necessary at all since it's in cc::ResourceProvider::DeleteAndReturnUnusedResourcesToChild? Seems like the browser should send an async message, add the resource to be returned to a queue, and then check later to see if the resource has been freed, and if so, return it to the child?

The sync IPC is to the GPU IO thread so should reply very quickly. Do we have a stack trace of what's going on in the GPU process?
Note, pre r352901 there was already a sync IPC on every InsertSyncPoint, that CL made a new mechanism that requires fewer.

The sync IPC is necessary to be able to validate a security property (that a process can only wait on a token that will eventually be released, and can't cause cycles).

Doesn't ImageTransportSurfaceOverlayMac::SwapBuffersInternal get called from the IO thread?

@#14: nope, that's the main thread.

Which one is the GPU process? I can try to get a stack trace from that process the next time I encounter the issue.

By the way, I am not sure, if this is related, but the high CPU usage turned out to be due to Google Tone extension b/29443201. That said, this bug would not explain why the main UI of the browser itself was hanging.

Moving this nonessential bug to the next milestone.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

ccameron@ - what are the next steps to get this fixed? Based on the comments this seems like something we should be attending to (and it's P!)?