Stack Quality95%Show frame trust levels
0x00007f88397cdaa8	(chrome -values.cc:781 )	base::DictionaryValue::MergeDictionary(base::DictionaryValue const*)
0x00007f8838d75883	(chrome -tracing_controller_impl.cc:895 )	content::TracingControllerImpl::AddFilteredMetadata(content::TracingController::TraceDataSink*, std::unique_ptr<base::DictionaryValue, std::default_delete<base::DictionaryValue> >, base::Callback<bool (std::string const&), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&)
0x00007f8838d770d4	(chrome -tracing_controller_impl.cc:376 )	content::TracingControllerImpl::StopTracing(scoped_refptr<content::TracingController::TraceDataSink> const&)
0x00007f8838d7bafc	(chrome -tracing_ui.cc:190 )	OnBeginJSONRequest
0x00007f8838d7c15a	(chrome -tracing_ui.cc:200 )	OnTracingRequest
0x00007f8838da0275	(chrome -callback.h:85 )	content::WebUIDataSourceImpl::StartDataRequest(std::string const&, base::Callback<content::WebContents* (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, base::Callback<void (scoped_refptr<base::RefCountedMemory>), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&)
0x00007f8838d9d061	(chrome -url_data_manager_backend.cc:803 )	content::URLDataManagerBackend::CallStartRequest(scoped_refptr<content::URLDataSourceImpl>, std::string const&, int, base::Callback<content::WebContents* (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, int)
0x00007f8838d9d8db	(chrome -bind_internal.h:164 )	base::internal::Invoker<base::internal::BindState<void (*)(scoped_refptr<content::URLDataSourceImpl>, std::string const&, int, base::Callback<content::WebContents* (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, int), base::internal::RetainedRefWrapper<content::URLDataSourceImpl>, std::string, int, base::Callback<content::WebContents* (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1>, int>, void ()>::Run(base::internal::BindStateBase*)
0x00007f8837e5f54c	(chrome -callback.h:68 )	base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*)
0x00007f8837e4a991	(chrome -message_loop.cc:413 )	base::MessageLoop::DoWork()
0x00007f8837e4afd2	(chrome -message_pump_libevent.cc:218 )	base::MessagePumpLibevent::Run(base::MessagePump::Delegate*)
0x00007f883979a8b7	(chrome -run_loop.cc:37 )	base::RunLoop::Run()
0x00007f8839482875	(chrome -chrome_browser_main.cc:1974 )	ChromeBrowserMainParts::MainMessageLoopRun(int*)
0x00007f8838adb98a	(chrome -browser_main_loop.cc:1086 )	content::BrowserMainLoop::RunMainMessageLoopParts()
0x00007f8838ade614	(chrome -browser_main_runner.cc:141 )	content::BrowserMainRunnerImpl::Run()
0x00007f8838ad880b	(chrome -browser_main.cc:46 )	content::BrowserMain(content::MainFunctionParams const&)
0x00007f88394233a9	(chrome -content_main_runner.cc:793 )	content::ContentMainRunnerImpl::Run()
0x00007f883942204a	(chrome -content_main.cc:20 )	content::ContentMain(content::ContentMainParams const&)
0x00007f88380ca89a	(chrome -chrome_main.cc:109 )	ChromeMain
0x00007f88356d4fb5	(libc-2.19.so -libc-start.c:292 )	__libc_start_main
0x00007f88380ca6bf	(chrome + 0x012346bf )	_start
0x00007ffd3ccaa277		

Eeeh, this is hanging while dumping metadata which is a bunch of strings and should take some millisecond at best.

Do you happen to have a trace that took time (but didn't crash) coming from CrOS?
I think should be pretty easy to spot the culprit by looking at which _metadata event has some enormous value in the trace.

Re #3: No; trying to take a trace (at least with memory-infra ticked) always crashes (see issue 643438). Perhaps the slowness is related to the memory-infra tracing brokeness; I'll try a trace with some other components ticked.

Re #3: Oh, I think there's a misunderstanding: This bug is for it taking a long time after pressing Record before the dialog to select what to record is even displayed, so there isn't any trace data at this point, surely?

(same as  issue 604885 ; we could perhaps close that since it was originally specific to performance event logging)

>  Perhaps the slowness is related to the memory-infra tracing brokeness
?? Some bug you are specifically thinking about?

memory-infra is disabled by default so doesn't easily explain why you would get into this state. Or does it happen only when ticking memory-infra?

Re memory-infra brokenness: Yes, I was thinking of issue 643438. ;)

Re the explanation: No, I agree; since this is happening before I've even had the opportunity to choose what to record, presumably it can't be related to any specific tracing type.  Based on the crash stack it looks like we do gather some metadata when the user hits Record, before we show the dialog, though?

> Re #3: Oh, I think there's a misunderstanding: This bug is for it taking a long time after pressing Record before the dialog to select what to record is even displayed, so there isn't any trace data at this point, surely?

Well the record dialog IIRC starts and stop tracing for a small interval of time to populate the categories. But still the disabled-by-default categories are disabled.
From your stack trace this is really the metadata population (which should be really dumb code) failing.
Maybe one of the strings (e.g. the gpu vendor or such) is not null terminated and we end up trying to dump all the heap when getting to the string?
If you have a debugger this would be quite easy to narrow down by just adding a breakpoint to TracingControllerImpl::AddFilteredMetadata and see which key causes the hang.

RE #7: Ah I see (yup unfortunately we have coverage in a lot of devices on the perf waterfall but not on CrOS. We are looking into adding more browsertest coverage as part of  Issue 670828 ).
Anyways, I'd rule it out in this bug. That cannot be triggered unless you explicitly tick memory-infra, which is not the case here because, as you say,this happens even before we start recording.

Re #9: Agreed; memory-infra does not seem to be the issue here. :)

Re #8: I believe that the crash only occurs if you actually kill the chrome://tracing renderer before the recording dialog comes up. I've just let the tab stay open for several minutes, and eventually the dialog did get displayed, so I think that rules out bogus metadata as the cause of the crash.

If the metadata gathering were not async-safe (i.e. couldn't cope with the target having been torn-down before it completes), then a regression in time taken to gather metadata would make that crash scenario more obvious, so perhaps we can try to repro on other platforms by introducing a delay in completion of metadata gathering.  I think it's also necessary to actually start a second Record operation having killed the first, stalled, one, to repro the crash, FWIW.

FWIW once I have allowed the tracing page to run long enough to show the dialog, opening it up again and hitting Record completes very quickly.

> Re #8: I believe that the crash only occurs if you actually kill the chrome://tracing renderer before the recording dialog comes up. 
So wait I'm missing something here. Usually the chrome://tracing dialog comes up instantaneously after you press record. I can't imagine how to kill it timely.
So is the problem here that the even first time it takes long time to come up (if the answer is "yes" then I don't understand what is the reason of killing anything to repro this bug. You should now in the state of grabbing a trace no?)

> FWIW once I have allowed the tracing page to run long enough to show the dialog, opening it up again and hitting Record completes very quickly.
Yes because we won't repeat the category warmup and re-collect metadata.

Primiano and I discussed offline and realised that the delay/hang is specific to tracing with MemoryCoordinator enabled, when MC has chosen to suspend one or more processes.  Filed  issue 675735  to track the MC problem.

Repurposing this bug for the crashiness in chrome://tracing.

Issue 678696 has been merged into this issue.

+bashi, since it appears to be necessary to have a suspended renderer to easily repro this, since that blocks the Tracing dialog popping up ( issue 675735 ).

Assigning to me for investigation and de-prioritize as renderer suspension is disabled now.

Users experienced this crash on the following builds:

Mac Canary 59.0.3047.0 -  0.76 CPM, 1 reports, 1 clients (signature base::DictionaryValue::MergeDictionary)

If this update was incorrect, please add "Fracas-Wrong" label to prevent future updates.

- Go/Fracas

We gave up suspending tabs for now. This should happen anymore.

Re #20: This bug was actually for the browser crasher, which can occur with _any_ renderer hang in-progress, not just MC, so I would suggest keeping this open.

I see. Re-opening. Not sure who is the right owner. Assigning primiano@ for triage.

kraynov can you take a look to this? I don't think this is specific to CrOS and should reproduce on any OS (just try on Linux).
See comment #10

kraynov: ping

Can not (or quite hard to) reproduce on Linux, yet.

Steps I took:

1. Launched Chrome with many different tabs
out/Release/chrome --enable-memory-coordinator --enable-remote-debugging --remote-debugging-port=9222 http://edition.cnn.com http://bbc.co.uk http://reddit.com http://news.google.com http://crbug.com http://android.com http://samsung.com http://lg.com http://o2.co.uk http://vodafone.co.uk http://gov.uk http://usa.com http://theverge.com http://hsbc.co.uk http://barclays.co.uk http://telegraph.co.uk http://spotify.com http://play.google.com http://apple.com http://nokia.com http://htc.com http://github.com http://dhl.com http://klm.com http://facebook.com http://twitter.com http://plug.google.com http://instagram.com http://youtube.com http://vimeo.com

2. Slightly hacked https://cs.chromium.org/chromium/src/third_party/catapult/tracing/bin/memory_infra_remote_dump to invoke memory pressure listener, just had to call
self.send('Memory.simulatePressureNotification', {'level': level})

3. Waited 5 mins
4. Invoked memory pressure
5. Waited a bit
6. Invoked again
7. Repeated again multiple times
8. Got tabs switching laggy (but not much)

But still tracing categories dialog shows almost instantly.
However, during my previous experiments (multiple tabs with media playback and heavy content) on the device with physically 8 gigs of RAM I've got observable tracing dialog delay. Unfortunately killing random tabs during this delay didn't crash the browser.

Do you have any recipe to strictly limit amount of RAM in order to simulate real memory pressure? Because fake memory pressure on the workstation didn't make things slow enough.

As per comment #20, Memory Coordinator no longer tries to suspend tabs, so
it's expected that the original repro steps will no longer repro this.
This bug is not really about the MC case, but about the fact that closing a
hung tab after hitting Record crashes things, so I'd expect that steps like:

1. Navigate to chrome://hang.
2. Open chrome://tracing
3. Start recording.
4. Close chrome://hang tab.

should repro the issue.

kraynov, as per comment #20 I think that if you add a sleep() in one of the renderer->browser tracing IPCs (TracingHostMsg_ChildSupportsTracing, TracingHostMsg_TraceDataCollected, TracingHostMsg_EndTracingAck) and use wez's repro in #26 the issue should show back.

So I tried my best with combinations of various approaches (IPC sleeps, memory pressure, killing pids, hang tabs, real memory pressure, etc.) and nothing led to browser crash. I don't know what kind of progress we can ever try to make with this issue.

Can not reproduce