New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 627205 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Jul 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 1
Type: Bug-Regression



Sign in to add a comment

Crash: [Out of Memory] base::internal::IncomingTaskQueue::PostPendingTask

Project Member Reported by sheriffbot@chromium.org, Jul 11 2016

Issue description

Crash Signature: [Out of Memory] base::internal::IncomingTaskQueue::PostPendingTask
Process Type: Gpu
Platform: Win
Channel: Canary
Version: 54.0.2793.0
Distinct Clients: 66
CPM: 2.41
Crash Reports: 82
Median Uptime: 01m:17s
Infected Clients: 8.54%

Sample Reports:
https://crash.corp.google.com/browse?q=reportid=%2718b512b600000000%27
https://crash.corp.google.com/browse?q=reportid=%276fe2620900000000%27
https://crash.corp.google.com/browse?q=reportid=%27ba3abe0200000000%27
https://crash.corp.google.com/browse?q=reportid=%27f5acde0200000000%27
https://crash.corp.google.com/browse?q=reportid=%27fe6602b600000000%27

Crash Link:
https://crash.corp.google.com/browse?q=product.name%3D%27Chrome%27%20AND%20product.version%3D%2754.0.2793.0%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27%5BOut%20of%20Memory%5D%20base%3A%3Ainternal%3A%3AIncomingTaskQueue%3A%3APostPendingTask%27

Crash Link (with version impact distribution):
https://crash.corp.google.com/browse?q=product.name%3D%27Chrome%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27%5BOut%20of%20Memory%5D%20base%3A%3Ainternal%3A%3AIncomingTaskQueue%3A%3APostPendingTask%27

Crash Stacktrace:
BREAKPOINT (0x5d907752)
#0 0x5d907752 in logging::LogMessage::~LogMessage base/logging.cc:748
#1 0x5dfd27c8 in base::`anonymous namespace'::OnNoMemory base/process/memory_win.cc:38
#2 0x5d894c3e in malloc base/allocator/allocator_shim_win.cc:119
#3 0x5d894c1b in operator new heap/new_scalar.cpp:19
#4 0x5d8b9d87 in std::allocator<HINSTANCE__ *>::allocate c:/b/depot_tools/win_toolchain/vs_files/95ddda401ec5678f15eeed01d2bee08fcbc5ee97/vc/include/xmemory0:625
#5 0x5d8d7ad1 in std::deque<base::PendingTask,std::allocator<base::PendingTask> >::_Growmap c:/b/depot_tools/win_toolchain/vs_files/95ddda401ec5678f15eeed01d2bee08fcbc5ee97/vc/include/deque:1842
#6 0x5d8d7a24 in std::deque<base::PendingTask,std::allocator<base::PendingTask> >::push_back c:/b/depot_tools/win_toolchain/vs_files/95ddda401ec5678f15eeed01d2bee08fcbc5ee97/vc/include/deque:1181
#7 0x5d8d790a in base::internal::IncomingTaskQueue::PostPendingTask base/message_loop/incoming_task_queue.cc:173
#8 0x5d8d7253 in base::internal::IncomingTaskQueue::AddToIncomingQueue base/message_loop/incoming_task_queue.cc:82
#9 0x5d8d7175 in base::internal::MessageLoopTaskRunner::PostDelayedTask base/message_loop/message_loop_task_runner.cc:30
#10 0x5d8d7146 in base::TaskRunner::PostTask base/task_runner.cc:45
#11 0x5d8dc352 in mojo::edk::NodeController::ForwardMessage mojo/edk/system/node_controller.cc:732
#12 0x5d8e5607 in mojo::edk::ports::Node::OnObserveProxy mojo/edk/system/ports/node.cc:540
#13 0x5d8dbe00 in mojo::edk::ports::Node::AcceptMessage mojo/edk/system/ports/node.cc:337
#14 0x5d8dca54 in mojo::edk::NodeController::AcceptIncomingMessages mojo/edk/system/node_controller.cc:645
#15 0x5d8e8381 in mojo::edk::NodeController::OnPortsMessage mojo/edk/system/node_controller.cc:1004
#16 0x5d8e7452 in mojo::edk::NodeChannel::OnChannelMessage mojo/edk/system/node_channel.cc:600
#17 0x5d8e6f82 in mojo::edk::Channel::OnReadComplete mojo/edk/system/channel.cc:563
#18 0x5d8e6e05 in mojo::edk::`anonymous namespace'::ChannelWin::OnIOCompleted mojo/edk/system/channel_win.cc:231
#19 0x5d8e5cee in base::MessagePumpForIO::DoRunLoop base/message_loop/message_pump_win.cc:731
#20 0x5d8e487f in base::MessagePumpWin::Run base/message_loop/message_pump_win.cc:140
#21 0x5d8e2051 in base::RunLoop::Run base/run_loop.cc:35
#22 0x5d8e5bd4 in base::Thread::Run base/threading/thread.cc:205
#23 0x5d8e5abe in base::Thread::ThreadMain base/threading/thread.cc:256
#24 0x5d8e58bd in base::`anonymous namespace'::ThreadFunc base/threading/platform_thread_win.cc:83
#25 0x76bbef1b in BaseThreadInitThunk 
#26 0x77a23647 in __RtlUserThreadStart 
#27 0x77a2361a in _RtlUserThreadStart 


Reporter: manoranjanr

 
Components: Internals>GPU
Labels: -Type-Bug ReleaseBlock-Dev M-54 OS-Windows Type-Bug-Regression
Owner: amistry@chromium.org
Status: Assigned (was: Untriaged)
Seems like this is a recent regression and we are seeing more no. of crashes starting M54.

amistry@, could you please look into this change (https://chromium.googlesource.com/chromium/src.git/+/7fa6701bc5183bd5a73203d4fe1309f75ccfd5b4) if possible?

Thank you!
Project Member

Comment 2 by sheriffbot@chromium.org, Jul 11 2016

Labels: FoundIn-M-54
Users experienced this crash on the following builds:

Win Canary 54.0.2793.0 -  2.41 CPM, 84 reports, 68 clients (signature [Out of Memory] base::internal::IncomingTaskQueue::PostPendingTask)

If this update was incorrect, please add "Fracas-Wrong" label to prevent future updates.

- Go/Fracas
Components: Internals>Mojo
Status: Started (was: Assigned)
This is my fault, but the linked CL isn't the problem.
Cc: roc...@chromium.org
This is weird. According to WinDbg, the allocation request is for 0x8000023 bytes, which is ~128M. The incoming message queue should not be getting anywhere near that big. This is happening on shutdown, which suggests there might be a UAF on the task runner or task queue, but both are ref counted.

+Ken. Any ideas? You understand Mojo shutdown better than anyone else.

Comment 5 by roc...@chromium.org, Jul 12 2016

I don't think it's possible for this to be a UAF on the task queue. Note that NodeController is never torn down either, so its reference to io_task_runner_ is never released.

Comment 6 by roc...@chromium.org, Jul 12 2016

Also the fact that the IO message loop pump is in the call stack implies the MessageLoop and therefore the task queue still exists.
Project Member

Comment 7 by bugdroid1@chromium.org, Jul 13 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d99c540d567bc586aeb379b30ad057d5f4187bd9

commit d99c540d567bc586aeb379b30ad057d5f4187bd9
Author: amistry <amistry@chromium.org>
Date: Wed Jul 13 02:14:54 2016

[mojo-edk] Ensure there is only one ProcessIncomingMessages() task posted to the IO thread.

BUG= 627205 

Review-Url: https://codereview.chromium.org/2138343002
Cr-Commit-Position: refs/heads/master@{#404897}

[modify] https://crrev.com/d99c540d567bc586aeb379b30ad057d5f4187bd9/mojo/edk/system/node_controller.cc
[modify] https://crrev.com/d99c540d567bc586aeb379b30ad057d5f4187bd9/mojo/edk/system/node_controller.h

Labels: -Restrict-View-EditIssue
Status: Fixed (was: Started)
No crash reports seen in 54.0.2796.2

No need for RVE label.

Sign in to add a comment