New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 762723 link

Starred by 4 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Mac
Pri: 3
Type: Bug
Hotlist-MemoryInfra



Sign in to add a comment

Disabling MemoryDumpProvider "V8Isolate".

Project Member Reported by erikc...@chromium.org, Sep 6 2017

Issue description

Just noticed this error message:

[11587:775:0906/154327.060234:ERROR:memory_dump_manager.cc(635)] Disabling MemoryDumpProvider "V8Isolate". Failed to post task on the task runner provided.


 
Cc: jochen@chromium.org
Did this happen one off? Do you have any repro?

Context:
--------
We have a failsafe mechanism in the MemoryDumpManager that mutes a MDP if we fail to PostTask (which practically means that the underlying MessageLoop is gone) for 3 times in a row.
It's a protection against MDP forgetting to unregister themselves and wasting time on the dump.

Theory:
-------
Now, Given this is V8Isolate, I think that this was due to a worker that created an isolate on another thread. 

Which concretely means that very likely there is a bug somewhere that makes it so that the TaskRunner passed to the IsolateHolder goes away without destroying the IsolateHolder itself (otherwise the dtor of IsolateHolder would have unregistered the MDP)

+jochen: I am not familiar with the lifetime of IsolateHolder. Is it expected that tis taskrunner can suddenly become invalid while the IsolateHolder is still alive?

Comment 2 by shrike@chromium.org, Sep 15 2017

Labels: Needs-Feedback
[mac bug triage] Needs feedback from reporter.
Repro steps: launch Chrome Canary on macOS. Navigate to some web pages. Wait.

Comment 4 by jochen@chromium.org, Sep 16 2017

Cc: skyos...@chromium.org haraken@chromium.org
there are two kind of isolate holders: the one for the main thread which never dies, and one per worker thread. They die some time during worker thread shutdown.

I don't know the expected lifetime of the taskrunner related to worker threads.
Cc: nhiroki@chromium.org
+nhiroki

Comment 6 by tapted@chromium.org, Sep 29 2017

Owner: ssid@chromium.org
Status: Assigned (was: Untriaged)
-> ssid@ [mac triage] this doesn't belong in the triage queue.

Looks like this was added in r377632

Can we suppress the error, or WARN during shutdown so it's clear this is non-critical?
 Issue 774257  has been merged into this issue.
Labels: -Needs-Feedback OS-Linux
This is really spammy and I would like it to stop. In a Chrome instance I launched yesterday, I've received this message 53 times. It's the majority of the logging output. I haven't figured out the repro enough, but I will keep an eye on the terminal and see when it happens next.

Given this message has existed for a while, either I did not notice this whole time, or some recent change made it trigger more often.
> This is really spammy and I would like it to stop.

Yeah I didn't realize the extend of this until I saw that myself on the internal thread.
I'm going to turn this into a DLOG until we find and solve the root cause (seems pretty reproducible now).

Honestly I hate logs as well, but I don't know a way in the middle between CHECK (which is a strong hammer here for the user, but delivers actionable feedback) and LOG (where we just rely on people reporting cases like this).
If this was just a DLOG in the first place we would have never noticed about this and never tried to fix it.
And, yes, I know about DumpWithoutCrashing but that is too dangerous and invasive (causes jank, requires throttling)
Project Member

Comment 11 by bugdroid1@chromium.org, Oct 18 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/547f6f35f1a5b83adb8885bd070c281484d60d7d

commit 547f6f35f1a5b83adb8885bd070c281484d60d7d
Author: Primiano Tucci <primiano@chromium.org>
Date: Wed Oct 18 11:26:33 2017

memory-infra: turn LOG into DLOG for MDP PostTask failures

It is causing too much spam.

TBR=ssid

Bug: 762723
Change-Id: If4e57ff410d3dcef1c53192edc70536f51161618
Reviewed-on: https://chromium-review.googlesource.com/725383
Reviewed-by: Primiano Tucci <primiano@chromium.org>
Commit-Queue: Primiano Tucci <primiano@chromium.org>
Cr-Commit-Position: refs/heads/master@{#509742}
[modify] https://crrev.com/547f6f35f1a5b83adb8885bd070c281484d60d7d/base/trace_event/memory_dump_manager.cc

This is on my workstation, BTW. After comment 8, I didn't see any more yesterday. Left Chrome running overnight, checked just now, and there's ~15 more, with only 6 tabs + 3 extensions running. All my entries start with "[1:1:" so I'm guessing it's some sandboxed process.
ssid: ping

Comment 14 by ssid@chromium.org, Oct 24 2017

Sorry didn't realize this was assigned to me. I thought it was assigned to jochen.
I looked at the code, the registration happens when IsolateHolder is created and unregistered when destroyed. This error is only possible when the IsolateHolder is not destroyed before the thread, or we got the wrong thread.
Sent a cl for review to fix what looks like a problem:
https://chromium-review.googlesource.com/c/chromium/src/+/736045
Project Member

Comment 15 by bugdroid1@chromium.org, Dec 12 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/331b5f0d92a60529c7e64e3b51944cc77de5829b

commit 331b5f0d92a60529c7e64e3b51944cc77de5829b
Author: Siddhartha <ssid@chromium.org>
Date: Tue Dec 12 14:47:05 2017

V8 dump provider should be registered on isolate thread

The V8 dump provider is currently registered on the thread that
IsolateHolder is created on. The isolate should be accessed on the task
runner provided rather than the current thread.


Bug: 762723
Change-Id: I9674690362fc0287d21baa27428b5093cb5146a7
Reviewed-on: https://chromium-review.googlesource.com/736045
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Kentaro Hara <haraken@chromium.org>
Reviewed-by: Sami Kyöstilä <skyostil@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Siddhartha S <ssid@chromium.org>
Cr-Commit-Position: refs/heads/master@{#523429}
[modify] https://crrev.com/331b5f0d92a60529c7e64e3b51944cc77de5829b/gin/isolate_holder.cc
[modify] https://crrev.com/331b5f0d92a60529c7e64e3b51944cc77de5829b/gin/v8_isolate_memory_dump_provider.cc
[modify] https://crrev.com/331b5f0d92a60529c7e64e3b51944cc77de5829b/gin/v8_isolate_memory_dump_provider.h

Project Member

Comment 16 by bugdroid1@chromium.org, Jan 3 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5ac59e663d764a3257c8a038a364d4cd578b4ef4

commit 5ac59e663d764a3257c8a038a364d4cd578b4ef4
Author: Siddhartha <ssid@chromium.org>
Date: Wed Jan 03 09:02:07 2018

Make WebThread::GetSingleThreadTaskRunner virtual

gin/per_isolate_data expects task runner to be provided when
initializing isolate. Implementing GetWebTaskRunner seems unnecessary.
So, make WebThread::GetSingleThreadTaskRunner() virtual and implement it
in utility thread.

BUG=762723

Change-Id: Ib8aefb5ceca077166d361cc5be9396de170d9453
Reviewed-on: https://chromium-review.googlesource.com/819110
Commit-Queue: Siddhartha S <ssid@chromium.org>
Reviewed-by: Kentaro Hara <haraken@chromium.org>
Reviewed-by: Alexander Timin <altimin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#526653}
[modify] https://crrev.com/5ac59e663d764a3257c8a038a364d4cd578b4ef4/content/child/blink_platform_impl.cc
[modify] https://crrev.com/5ac59e663d764a3257c8a038a364d4cd578b4ef4/content/renderer/render_thread_impl.cc
[modify] https://crrev.com/5ac59e663d764a3257c8a038a364d4cd578b4ef4/third_party/WebKit/Source/bindings/core/v8/V8Initializer.cpp
[modify] https://crrev.com/5ac59e663d764a3257c8a038a364d4cd578b4ef4/third_party/WebKit/Source/platform/WebThread.cpp
[modify] https://crrev.com/5ac59e663d764a3257c8a038a364d4cd578b4ef4/third_party/WebKit/Source/platform/bindings/V8PerIsolateData.cpp
[modify] https://crrev.com/5ac59e663d764a3257c8a038a364d4cd578b4ef4/third_party/WebKit/Source/platform/bindings/V8PerIsolateData.h
[modify] https://crrev.com/5ac59e663d764a3257c8a038a364d4cd578b4ef4/third_party/WebKit/Source/platform/scheduler/child/webthread_base.cc
[modify] https://crrev.com/5ac59e663d764a3257c8a038a364d4cd578b4ef4/third_party/WebKit/Source/platform/scheduler/child/webthread_impl_for_worker_scheduler.cc
[modify] https://crrev.com/5ac59e663d764a3257c8a038a364d4cd578b4ef4/third_party/WebKit/Source/platform/scheduler/child/webthread_impl_for_worker_scheduler.h
[modify] https://crrev.com/5ac59e663d764a3257c8a038a364d4cd578b4ef4/third_party/WebKit/Source/platform/scheduler/child/webthread_impl_for_worker_scheduler_unittest.cc
[modify] https://crrev.com/5ac59e663d764a3257c8a038a364d4cd578b4ef4/third_party/WebKit/Source/platform/scheduler/renderer/webthread_impl_for_renderer_scheduler.cc
[modify] https://crrev.com/5ac59e663d764a3257c8a038a364d4cd578b4ef4/third_party/WebKit/Source/platform/scheduler/renderer/webthread_impl_for_renderer_scheduler.h
[modify] https://crrev.com/5ac59e663d764a3257c8a038a364d4cd578b4ef4/third_party/WebKit/Source/platform/scheduler/utility/webthread_impl_for_utility_thread.cc
[modify] https://crrev.com/5ac59e663d764a3257c8a038a364d4cd578b4ef4/third_party/WebKit/Source/platform/scheduler/utility/webthread_impl_for_utility_thread.h
[modify] https://crrev.com/5ac59e663d764a3257c8a038a364d4cd578b4ef4/third_party/WebKit/Source/platform/testing/TestingPlatformSupportWithMockScheduler.cpp
[modify] https://crrev.com/5ac59e663d764a3257c8a038a364d4cd578b4ef4/third_party/WebKit/public/platform/WebThread.h
[modify] https://crrev.com/5ac59e663d764a3257c8a038a364d4cd578b4ef4/third_party/WebKit/public/platform/scheduler/child/webthread_base.h
[modify] https://crrev.com/5ac59e663d764a3257c8a038a364d4cd578b4ef4/tools/v8_context_snapshot/v8_context_snapshot_generator.cc

Sign in to add a comment