hmm that error code looks familiar to me - https://bugs.chromium.org/p/chromium/issues/detail?id=468922#c8

forshaw might have ideas? but we are not running app container on stable. unless it's manually turned on?

54.0.2840.71 is potentially a version behind (54.0.2840.87). Do you have a new_chrome.exe? Could you check the version numbers of the various files in ...\Google\Chrome\Application?

No, I already looked to see if I had a new_chrome.exe, but when this happened I hadn't yet taken an update. There's no problem with the binary that it's trying to run.

I should also point out that this another "fork loop", in that the browser is trying repeatedly and constantly to start a new renderer process. Last time we had one of these we flooded the crash server with the dying renderers, so there's another such code path we should trace down...

The task to launch a process is being launched with the following stack trace. It looks like there's a pending navigation entry that is failing

00 (Inline Function) --------`-------- chrome_7ffb2b6c0000!mojo::edk::PlatformHandle::{ctor} [c:\b\build\slave\win64-pgo\build\src\mojo\edk\embedder\platform_handle.h @ 64]
01 (Inline Function) --------`-------- chrome_7ffb2b6c0000!mojo::edk::PlatformHandle::{ctor} [c:\b\build\slave\win64-pgo\build\src\mojo\edk\embedder\platform_handle.h @ 62]
02 (Inline Function) --------`-------- chrome_7ffb2b6c0000!mojo::edk::ScopedPlatformHandle::{ctor} [c:\b\build\slave\win64-pgo\build\src\mojo\edk\embedder\scoped_platform_handle.h @ 19]
03 000000ac`9fb2c510 00007ffb`2b95f8ad chrome_7ffb2b6c0000!content::ChildProcessLauncher::Launch+0x3a [c:\b\build\slave\win64-pgo\build\src\content\browser\child_process_launcher.cc @ 458]
04 000000ac`9fb2c730 00007ffb`2bb20fdb chrome_7ffb2b6c0000!content::ChildProcessLauncher::ChildProcessLauncher+0xb1 [c:\b\build\slave\win64-pgo\build\src\content\browser\child_process_launcher.cc @ 418]
05 000000ac`9fb2c760 00007ffb`2b9fb6f4 chrome_7ffb2b6c0000!content::RenderProcessHostImpl::Init+0x4d3 [c:\b\build\slave\win64-pgo\build\src\content\browser\renderer_host\render_process_host_impl.cc @ 926]
06 000000ac`9fb2c9c0 00007ffb`2b9fc342 chrome_7ffb2b6c0000!content::RenderFrameHostManager::InitRenderView+0x30 [c:\b\build\slave\win64-pgo\build\src\content\browser\frame_host\render_frame_host_manager.cc @ 1879]
07 000000ac`9fb2ca00 00007ffb`2b9f89b6 chrome_7ffb2b6c0000!content::RenderFrameHostManager::ReinitializeRenderFrame+0x4a [c:\b\build\slave\win64-pgo\build\src\content\browser\frame_host\render_frame_host_manager.cc @ 2056]
08 000000ac`9fb2ca30 00007ffb`2b9e5895 chrome_7ffb2b6c0000!content::RenderFrameHostManager::Navigate+0x17e [c:\b\build\slave\win64-pgo\build\src\content\browser\frame_host\render_frame_host_manager.cc @ 241]
09 000000ac`9fb2caf0 00007ffb`2b9e5cd7 chrome_7ffb2b6c0000!content::NavigatorImpl::NavigateToEntry+0x5ad [c:\b\build\slave\win64-pgo\build\src\content\browser\frame_host\navigator_impl.cc @ 358]
0a 000000ac`9fb2d530 00007ffb`2b9dbabc chrome_7ffb2b6c0000!content::NavigatorImpl::NavigateToPendingEntry+0x63 [c:\b\build\slave\win64-pgo\build\src\content\browser\frame_host\navigator_impl.cc @ 447]
0b 000000ac`9fb2d590 00007ffb`2b9db912 chrome_7ffb2b6c0000!content::NavigationControllerImpl::NavigateToPendingEntryInternal+0x18c [c:\b\build\slave\win64-pgo\build\src\content\browser\frame_host\navigation_controller_impl.cc @ 1884]
0c 000000ac`9fb2d630 00007ffb`2b9d9765 chrome_7ffb2b6c0000!content::NavigationControllerImpl::NavigateToPendingEntry+0x13a [c:\b\build\slave\win64-pgo\build\src\content\browser\frame_host\navigation_controller_impl.cc @ 1827]
0d (Inline Function) --------`-------- chrome_7ffb2b6c0000!content::NavigationControllerImpl::LoadEntry+0x20 [c:\b\build\slave\win64-pgo\build\src\content\browser\frame_host\navigation_controller_impl.cc @ 448]
0e 000000ac`9fb2d660 00007ffb`2b9d924a chrome_7ffb2b6c0000!content::NavigationControllerImpl::LoadURLWithParams+0x4f1 [c:\b\build\slave\win64-pgo\build\src\content\browser\frame_host\navigation_controller_impl.cc @ 780]
0f 000000ac`9fb2d830 00007ffb`2c4ad6f5 chrome_7ffb2b6c0000!content::NavigationControllerImpl::LoadURL+0xa2 [c:\b\build\slave\win64-pgo\build\src\content\browser\frame_host\navigation_controller_impl.cc @ 650]
10 000000ac`9fb2db90 00007ffb`2c4ad326 chrome_7ffb2b6c0000!extensions::ExtensionHost::LoadInitialURL+0xdd [c:\b\build\slave\win64-pgo\build\src\extensions\browser\extension_host.cc @ 234]
11 000000ac`9fb2dc80 00007ffb`2c4f0b0d chrome_7ffb2b6c0000!extensions::ExtensionHost::CreateRenderViewNow+0xbe [c:\b\build\slave\win64-pgo\build\src\extensions\browser\extension_host.cc @ 160]
12 000000ac`9fb2dd90 00007ffb`2b800db2 chrome_7ffb2b6c0000!extensions::SerialExtensionHostQueue::ProcessOneHost+0x25 [c:\b\build\slave\win64-pgo\build\src\extensions\browser\serial_extension_host_queue.cc @ 80]

It looks like an extension renderer is attempting to be reloaded over and over again and failing. This is potentially another fork-bomb?

00 000000ac`9fb2bc58 00007ffb`2c4e0d67 chrome_7ffb2b6c0000!extensions::SerialExtensionHostQueue::PostTask [c:\b\build\slave\win64-pgo\build\src\extensions\browser\serial_extension_host_queue.cc @ 64]
01 000000ac`9fb2bc60 00007ffb`2c4e306e chrome_7ffb2b6c0000!extensions::LoadMonitoringExtensionHostQueue::Add+0x167 [c:\b\build\slave\win64-pgo\build\src\extensions\browser\load_monitoring_extension_host_queue.cc @ 60]
02 000000ac`9fb2bd20 00007ffb`2c4df53f chrome_7ffb2b6c0000!extensions::ProcessManager::CreateBackgroundHost+0xd6 [c:\b\build\slave\win64-pgo\build\src\extensions\browser\process_manager.cc @ 388]
03 000000ac`9fb2bd60 00007ffb`2c4a3f34 chrome_7ffb2b6c0000!extensions::LazyBackgroundTaskQueue::AddPendingTask+0x17f [c:\b\build\slave\win64-pgo\build\src\extensions\browser\lazy_background_task_queue.cc @ 93]
04 000000ac`9fb2be90 00007ffb`2c4a37fd chrome_7ffb2b6c0000!extensions::EventRouter::MaybeLoadLazyBackgroundPageToDispatchEvent+0x24c [c:\b\build\slave\win64-pgo\build\src\extensions\browser\event_router.cc @ 670]
05 000000ac`9fb2bf50 00007ffb`2c4a3599 chrome_7ffb2b6c0000!extensions::EventRouter::DispatchLazyEvent+0x71 [c:\b\build\slave\win64-pgo\build\src\extensions\browser\event_router.cc @ 531]
06 000000ac`9fb2c040 00007ffb`2c4a3294 chrome_7ffb2b6c0000!extensions::EventRouter::DispatchEventImpl+0xe9 [c:\b\build\slave\win64-pgo\build\src\extensions\browser\event_router.cc @ 490]
07 000000ac`9fb2c130 00007ffb`2dc01234 chrome_7ffb2b6c0000!extensions::EventRouter::BroadcastEvent+0x54 [c:\b\build\slave\win64-pgo\build\src\extensions\browser\event_router.cc @ 451]
08 000000ac`9fb2c1a0 00007ffb`2dc384fa chrome_7ffb2b6c0000!extensions::MDnsAPI::OnDnsSdEvent+0x370 [c:\b\build\slave\win64-pgo\build\src\chrome\browser\extensions\api\mdns\mdns_api.cc @ 186]
09 000000ac`9fb2c2e0 00007ffb`2dc37c76 chrome_7ffb2b6c0000!extensions::DnsSdRegistry::DispatchApiEvent+0x22a [c:\b\build\slave\win64-pgo\build\src\chrome\browser\extensions\api\mdns\dns_sd_registry.cc @ 232]
0a 000000ac`9fb2cec0 00007ffb`2dc00dcb chrome_7ffb2b6c0000!extensions::DnsSdRegistry::RegisterDnsSdListener+0x1c6 [c:\b\build\slave\win64-pgo\build\src\chrome\browser\extensions\api\mdns\dns_sd_registry.cc @ 166]
0b 000000ac`9fb2d030 00007ffb`2c4a2641 chrome_7ffb2b6c0000!extensions::MDnsAPI::UpdateMDnsListeners+0x2bb [c:\b\build\slave\win64-pgo\build\src\chrome\browser\extensions\api\mdns\mdns_api.cc @ 123]
0c 000000ac`9fb2d1e0 00007ffb`2c4a06f6 chrome_7ffb2b6c0000!extensions::EventRouter::OnListenerAdded+0x9d [c:\b\build\slave\win64-pgo\build\src\extensions\browser\event_router.cc @ 244]
0d 000000ac`9fb2d330 00007ffb`2c4a0aa4 chrome_7ffb2b6c0000!extensions::EventListenerMap::AddListener+0x10e [c:\b\build\slave\win64-pgo\build\src\extensions\browser\event_listener_map.cc @ 113]
0e 000000ac`9fb2d3b0 00007ffb`2c4a4534 chrome_7ffb2b6c0000!extensions::EventListenerMap::LoadFilteredLazyListeners+0xb0 [c:\b\build\slave\win64-pgo\build\src\extensions\browser\event_listener_map.cc @ 221]
0f 000000ac`9fb2d410 00007ffb`2d1bef6e chrome_7ffb2b6c0000!extensions::EventRouter::OnExtensionLoaded+0x158 [c:\b\build\slave\win64-pgo\build\src\extensions\browser\event_router.cc @ 832]
10 (Inline Function) --------`-------- chrome_7ffb2b6c0000!extensions::ExtensionRegistry::TriggerOnLoaded+0x64 [c:\b\build\slave\win64-pgo\build\src\extensions\browser\extension_registry.cc @ 54]
11 000000ac`9fb2d4d0 00007ffb`2d1c0688 chrome_7ffb2b6c0000!ExtensionService::NotifyExtensionLoaded+0x2c2 [c:\b\build\slave\win64-pgo\build\src\chrome\browser\extensions\extension_service.cc @ 1053]
12 000000ac`9fb2dbb0 00007ffb`2d1de421 chrome_7ffb2b6c0000!ExtensionService::AddExtension+0x444 [c:\b\build\slave\win64-pgo\build\src\chrome\browser\extensions\extension_service.cc @ 1520]
13 000000ac`9fb2dc70 00007ffb`2d1bdcec chrome_7ffb2b6c0000!extensions::InstalledLoader::Load+0x2b9 [c:\b\build\slave\win64-pgo\build\src\chrome\browser\extensions\installed_loader.cc @ 235]
14 000000ac`9fb2dd30 00007ffb`2d709f2c chrome_7ffb2b6c0000!ExtensionService::ReloadExtensionImpl+0x328 [c:\b\build\slave\win64-pgo\build\src\chrome\browser\extensions\extension_service.cc @ 685]
15 (Inline Function) --------`-------- chrome_7ffb2b6c0000!ExtensionService::ReloadExtension+0xe [c:\b\build\slave\win64-pgo\build\src\chrome\browser\extensions\extension_service.cc @ 699]
16 000000ac`9fb2de70 00007ffb`2c6d8613 chrome_7ffb2b6c0000!`anonymous namespace'::ReloadExtension+0xa0 [c:\b\build\slave\win64-pgo\build\src\chrome\browser\background\background_contents_service.cc @ 247]

I hit this again and spent a while debugging it, made tricky by the lack of symbols for the version of ntdll.dll or KernelBase.dll that I am running. So, all I can tell is that CreateProcessAsUserW is failing with a last error code of 0x57. Changing the exe_path, cmd_line, inherit_handles flag, and lockdown_token all fail to change the results.

So far all of the people reporting this problem are Google employees, so it may be that this is being caused by some Google specific configuration. The ERROR_INVALID_PARAMETER code is, according to the Process.Sandbox.Launch.Error metrics, only the fifth most common failure code for process launch failures. This suggests that the problem is not widespread and may be triggered mostly by Google employees.

Here is the call stack where we are calling CreateProcessAsUser:

>	chrome.exe!sandbox::TargetProcess::Create
 	chrome.exe!sandbox::BrokerServicesBase::SpawnTarget
 	chrome.dll!content::StartSandboxedProcess
 	chrome.dll!content::`anonymous namespace'::LaunchOnLauncherThread
 	chrome.dll!base::internal::Invoker<>::Run
 	chrome.dll!base::debug::TaskAnnotator::RunTask
 	chrome.dll!base::MessageLoop::RunTask
 	chrome.dll!base::MessageLoop::DoWork
 	chrome.dll!base::MessagePumpDefault::Run
 	chrome.dll!base::RunLoop::Run
 	chrome.dll!base::Thread::Run
 	chrome.dll!content::BrowserThreadImpl::ProcessLauncherThreadRun
 	chrome.dll!content::BrowserThreadImpl::Run
 	chrome.dll!base::Thread::ThreadMain
 	chrome.dll!base::`anonymous namespace'::ThreadFunc
 	kernel32.dll!@BaseThreadInitThunk@12

see also  issue 645319 

what's the !gle from the createprocessasuser call?

My only thought on this is we're now inheriting a handle for mojo. Setting inherit_handles to false while specifying a list of handles in the proc/thread attribute lists causes ERROR_INVALID_PARAMETER. Adding an invalid handle or a non-inheritable handle to the list causes ERROR_INVALID_PARAMETER. This would seem to be a reasonable explanation if indeed it is related to what Chrome is passing to CreateProcessAsUser which is causing the error. Still looking at the code I'm not sure how we'd end up with a bad handle list, and definitely one which is only bad once in a while and then stays bad.

!gle relies on symbols from ntdll.dll, which Microsoft has not seen fit to provide. So, I just have the GetLastError()/@err results which are 0x57 after returning from CreateProcessAsUser.

Due to the lack of symbols I also can't step through (in any useful way) the implementation of CreateProcessAsUser, but it sounds like that wasn't very profitable anyway.