New issue
Advanced search Search tips

Issue 594230 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Mar 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Mac
Pri: 2
Type: Bug



Sign in to add a comment

m_workerGlobalScope->hasOneRef() ASSERT is hit in some layout tests on WebKit Linux non-Oilpan dbg

Project Member Reported by alex...@chromium.org, Mar 11 2016

Issue description

These tests have been crashing pretty consistently on "WebKit Linux non-Oilpan dbg":

fast/files/workers/worker-read-blob-async-crash.html
http/tests/fetch/serviceworker/fetch-base-https-other-https.html
http/tests/fetch/serviceworker/fetch.html

Example build:
https://build.chromium.org/p/chromium.webkit/builders/WebKit%20Linux%20non-Oilpan%20%28dbg%29/builds/300

Crash output for fast/files/workers/worker-read-blob-async-crash.html:

crash log for renderer (pid <unknown>):
STDOUT: PASS successfullyParsed is true
STDOUT: 
STDOUT: TEST COMPLETE
STDOUT: Test for crash bug in WorkerThreadableLoader::MainThreadBridge::mainThreadCreateLoader when FileReader is runing in the worker thread and the page location is changed.
STDOUT: 
STDOUT: This test PASSED if it did not crash.
STDOUT: 
STDOUT: 
STDERR: ASSERTION FAILED: m_workerGlobalScope->hasOneRef()
STDERR: ../../third_party/WebKit/Source/core/workers/WorkerThread.cpp(271) : void blink::WorkerThread::performShutdownTask()
STDERR: 1   0x7fa1aba06d60 blink::WorkerThread::performShutdownTask()
STDERR: 2   0x7fa1aaa0c4c9
STDERR: 3   0x7fa1aba0b3cd
STDERR: 4   0x7fa1aba0b379
STDERR: 5   0x7fa1aafe0b79
STDERR: 6   0x7fa1aba05bbe blink::WorkerThread::performTask(WTF::PassOwnPtr<blink::ExecutionContextTask>, bool)
STDERR: 7   0x7fa1aba0a593
STDERR: 8   0x7fa1aba0a4dd
STDERR: 9   0x7fa1aba0a389
STDERR: 10  0x7fa1b014affa
STDERR: 11  0x7fa1adde3d3e scheduler::WebTaskRunnerImpl::runTask(std::unique_ptr<blink::WebTaskRunner::Task, std::default_delete<blink::WebTaskRunner::Task> >)
STDERR: 12  0x7fa1adde4dda
STDERR: 13  0x7fa1adde4d45
STDERR: 14  0x7fa1adde4d06
STDERR: 15  0x7fa1b410f07e
STDERR: 16  0x7fa1b413514e base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask const&)
STDERR: 17  0x7fa1addbba32 scheduler::TaskQueueManager::ProcessTaskFromWorkQueue(scheduler::internal::WorkQueue*, scheduler::internal::TaskQueueImpl::Task*)
STDERR: 18  0x7fa1addb99b2 scheduler::TaskQueueManager::DoWork(base::TimeTicks, bool)
STDERR: 19  0x7fa1addc117e
STDERR: 20  0x7fa1addc107a
STDERR: 21  0x7fa1addc0ff8
STDERR: 22  0x7fa1b410f07e
STDERR: 23  0x7fa1b413514e base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask const&)
STDERR: 24  0x7fa1b41acaff base::MessageLoop::RunTask(base::PendingTask const&)
STDERR: 25  0x7fa1b41acd88 base::MessageLoop::DeferOrRunPendingTask(base::PendingTask const&)
STDERR: 26  0x7fa1b41acf52 base::MessageLoop::DoWork()
STDERR: 27  0x7fa1b41be723 base::MessagePumpDefault::Run(base::MessagePump::Delegate*)
STDERR: 28  0x7fa1b41ac52f base::MessageLoop::RunHandler()
STDERR: 29  0x7fa1b4242594 base::RunLoop::Run()
STDERR: 30  0x7fa1b41ab5d4 base::MessageLoop::Run()
STDERR: 31  0x7fa1b42b6559 base::Thread::Run(base::MessageLoop*)

The m_workerGlobalScope->hasOneRef() ASSERT is getting hit for all three tests.

It seems this has been happening for a long time, so I don't know what the culprit is:
https://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=http%2Ftests%2Ffetch%2Fserviceworker%2Ffetch.html&testType=webkit_tests

Adding test authors and a couple of other people who might be able to triage this.
 
Cc: haraken@chromium.org
Note that these tests work fine with Oilpan and only fail without Oilpan.  +haraken, who might be able to comment on whether Oilpan just fixes this altogether.
Project Member

Comment 2 by bugdroid1@chromium.org, Mar 11 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c9a69c7f52ec395da04ba82c919b618ba9ffdf95

commit c9a69c7f52ec395da04ba82c919b618ba9ffdf95
Author: alexmos <alexmos@chromium.org>
Date: Fri Mar 11 22:43:22 2016

Add expectations for three crashing layout tests on Linux non-Oilpan debug.

BUG= 594230 
NOTRY=true
TBR=haraken@chromium.org

Review URL: https://codereview.chromium.org/1786663004

Cr-Commit-Position: refs/heads/master@{#380766}

[modify] https://crrev.com/c9a69c7f52ec395da04ba82c919b618ba9ffdf95/third_party/WebKit/LayoutTests/TestExpectations

Comment 3 by falken@chromium.org, Mar 18 2016

Components: Blink>ServiceWorker Blink>Workers
Note: This is happening often on linux_blink_oilpan_rel tryserver bot for http/tests/serviceworker/fetch-event-network-error.html, but that bot is actually non-Oilpan per blink-dev mail "PSA: Enable Oilpan on ToT".

Comment 4 by kinuko@chromium.org, Mar 22 2016

 Issue 596703  has been merged into this issue.

Comment 5 by kinuko@chromium.org, Mar 22 2016

 Issue 596915  has been merged into this issue.

Comment 6 by kinuko@chromium.org, Mar 22 2016

Cc: -falken@chromium.org nhiroki@chromium.org
Owner: falken@chromium.org
Could someone have bandwidth to take a look into this? (it's a little hard for me to squeeze out time this week) From the dashboard it looks it's kinda reproducible on Linux/Mac with non-oilpan build.

https://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=http%2Ftests%2Ffetch%2Fserviceworker%2Ffetch.html&testType=webkit_tests

STDERR: ASSERTION FAILED: m_workerGlobalScope->hasOneRef()
STDERR: ../../third_party/WebKit/Source/core/workers/WorkerThread.cpp(271) : void blink::WorkerThread::performShutdownTask()
STDERR: 1   0x122bd59be blink::WorkerThread::performShutdownTask()
STDERR: 2   0x122bddc97 void WTF::FunctionWrapper<void (blink::WorkerThread::*)()>::operator()<>(blink::WorkerThread*)
STDERR: 3   0x122bddbf6 void WTF::PartBoundFunctionImpl<(WTF::FunctionThreadAffinity)1, std::__1::tuple<blink::WorkerThread* const&>, WTF::FunctionWrapper<void (blink::WorkerThread::*)()> >::callInternal<0ul>(base::IndexSequence<0ul> const&)
STDERR: 4   0x122bddb17 WTF::PartBoundFunctionImpl<(WTF::FunctionThreadAffinity)1, std::__1::tuple<blink::WorkerThread* const&>, WTF::FunctionWrapper<void (blink::WorkerThread::*)()> >::operator()()
STDERR: 5   0x122661257 blink::internal::CallClosureTask<(WTF::FunctionThreadAffinity)1>::performTask(blink::ExecutionContext*)
STDERR: 6   0x122bd36a2 blink::WorkerThread::performTask(WTF::PassOwnPtr<blink::ExecutionContextTask>, bool)
STDERR: 7   0x122bdaf94 void WTF::FunctionWrapper<void (blink::WorkerThread::*)(WTF::PassOwnPtr<blink::ExecutionContextTask>, bool)>::operator()<WTF::PassOwnPtr<blink::ExecutionContextTask>&, bool&>(blink::WorkerThread*, WTF::PassOwnPtr<blink::ExecutionContextTask>&&&, bool&&&)
STDERR: 8   0x122bdaeb8 void WTF::PartBoundFunctionImpl<(WTF::FunctionThreadAffinity)0, std::__1::tuple<blink::WorkerThread*&&, WTF::PassOwnPtr<blink::ExecutionContextTask>&&, bool&&>, WTF::FunctionWrapper<void (blink::WorkerThread::*)(WTF::PassOwnPtr<blink::ExecutionContextTask>, bool)> >::callInternal<0ul, 1ul, 2ul>(base::IndexSequence<0ul, 1ul, 2ul> const&)
STDERR: 9   0x122bdabd7 WTF::PartBoundFunctionImpl<(WTF::FunctionThreadAffinity)0, std::__1::tuple<blink::WorkerThread*&&, WTF::PassOwnPtr<blink::ExecutionContextTask>&&, bool&&>, WTF::FunctionWrapper<void (blink::WorkerThread::*)(WTF::PassOwnPtr<blink::ExecutionContextTask>, bool)> >::operator()()
STDERR: 10  0x10fc82d6b blink::CrossThreadTask::run()
STDERR: 11  0x11e980332 scheduler::WebTaskRunnerImpl::runTask(std::__1::unique_ptr<blink::WebTaskRunner::Task, std::__1::default_delete<blink::WebTaskRunner::Task> >)
STDERR: 12  0x11e981b1d void base::internal::RunnableAdapter<void (*)(std::__1::unique_ptr<blink::WebTaskRunner::Task, std::__1::default_delete<blink::WebTaskRunner::Task> >)>::Run<std::__1::unique_ptr<blink::WebTaskRunner::Task, std::__1::default_delete<blink::WebTaskRunner::Task> > >(std::__1::unique_ptr<blink::WebTaskRunner::Task, std::__1::default_delete<blink::WebTaskRunner::Task> >&&)
STDERR: 13  0x11e981936 void base::internal::InvokeHelper<false, void, base::internal::RunnableAdapter<void (*)(std::__1::unique_ptr<blink::WebTaskRunner::Task, std::__1::default_delete<blink::WebTaskRunner::Task> >)> >::MakeItSo<std::__1::unique_ptr<blink::WebTaskRunner::Task, std::__1::default_delete<blink::WebTaskRunner::Task> > >(base::internal::RunnableAdapter<void (*)(std::__1::unique_ptr<blink::WebTaskRunner::Task, std::__1::default_delete<blink::WebTaskRunner::Task> >)>, std::__1::unique_ptr<blink::WebTaskRunner::Task, std::__1::default_delete<blink::WebTaskRunner::Task> >&&)
STDERR: 14  0x11e98182d base::internal::Invoker<base::IndexSequence<0ul>, base::internal::BindState<base::internal::RunnableAdapter<void (*)(std::__1::unique_ptr<blink::WebTaskRunner::Task, std::__1::default_delete<blink::WebTaskRunner::Task> >)>, void (std::__1::unique_ptr<blink::WebTaskRunner::Task, std::__1::default_delete<blink::WebTaskRunner::Task> >), base::internal::PassedWrapper<std::__1::unique_ptr<blink::WebTaskRunner::Task, std::__1::default_delete<blink::WebTaskRunner::Task> > > >, base::internal::InvokeHelper<false, void, base::internal::RunnableAdapter<void (*)(std::__1::unique_ptr<blink::WebTaskRunner::Task, std::__1::default_delete<blink::WebTaskRunner::Task> >)> >, void ()>::Run(base::internal::BindStateBase*)
STDERR: 15  0x10f66331f base::Callback<void (), (base::internal::CopyMode)1>::Run() const
STDERR: 16  0x10f68ecfe base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask const&)
STDERR: 17  0x11e9494d3 scheduler::TaskQueueManager::ProcessTaskFromWorkQueue(scheduler::internal::WorkQueue*, scheduler::internal::TaskQueueImpl::Task*)
STDERR: 18  0x11e945da0 scheduler::TaskQueueManager::DoWork(base::TimeTicks, bool)
STDERR: 19  0x11e94f6aa void base::internal::RunnableAdapter<void (scheduler::TaskQueueManager::*)(base::TimeTicks, bool)>::Run<base::TimeTicks const&, bool const&>(scheduler::TaskQueueManager*, base::TimeTicks const&&&, bool const&&&)
STDERR: 20  0x11e94f44e void base::internal::InvokeHelper<true, void, base::internal::RunnableAdapter<void (scheduler::TaskQueueManager::*)(base::TimeTicks, bool)> >::MakeItSo<base::WeakPtr<scheduler::TaskQueueManager>, base::TimeTicks const&, bool const&>(base::internal::RunnableAdapter<void (scheduler::TaskQueueManager::*)(base::TimeTicks, bool)>, base::WeakPtr<scheduler::TaskQueueManager>, base::TimeTicks const&&&, bool const&&&)
STDERR: 21  0x11e94f3ad base::internal::Invoker<base::IndexSequence<0ul, 1ul, 2ul>, base::internal::BindState<base::internal::RunnableAdapter<void (scheduler::TaskQueueManager::*)(base::TimeTicks, bool)>, void (scheduler::TaskQueueManager*, base::TimeTicks, bool), base::WeakPtr<scheduler::TaskQueueManager>, base::TimeTicks, bool>, base::internal::InvokeHelper<true, void, base::internal::RunnableAdapter<void (scheduler::TaskQueueManager::*)(base::TimeTicks, bool)> >, void ()>::Run(base::internal::BindStateBase*)
STDERR: 22  0x10f66331f base::Callback<void (), (base::internal::CopyMode)1>::Run() const
STDERR: 23  0x10f68ecfe base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask const&)
STDERR: 24  0x10f765970 base::MessageLoop::RunTask(base::PendingTask const&)
STDERR: 25  0x10f765fa6 base::MessageLoop::DeferOrRunPendingTask(base::PendingTask const&)
STDERR: 26  0x10f766658 base::MessageLoop::DoWork()
STDERR: 27  0x10f6384d8 base::MessagePumpCFRunLoopBase::RunWork()
STDERR: 28  0x10f63844a invocation function for block in base::MessagePumpCFRunLoopBase::RunWorkSource(void*)
STDERR: 29  0x10f71d16a base::mac::CallWithEHFrame(void () block_pointer)
STDERR: 30  0x10f6377e5 base::MessagePumpCFRunLoopBase::RunWorkSource(void*)
STDERR: 31  0x7fff8ed325b1 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__
STDERR: Received signal 11 SEGV_MAPERR 0000fbadbeef
Project Member

Comment 7 by chromium...@appspot.gserviceaccount.com, Mar 22 2016

Labels: Sheriff-Chromium
Detected 4 new flakes for test/step "http/tests/serviceworker/fetch-event-network-error.html". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyQgsSBUZsYWtlIjdodHRwL3Rlc3RzL3NlcnZpY2V3b3JrZXIvZmV0Y2gtZXZlbnQtbmV0d29yay1lcnJvci5odG1sDA. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
Cc: -nhiroki@chromium.org falken@chromium.org
Owner: nhiroki@chromium.org
Status: Assigned (was: Available)
I'll take a look.
Status: Started (was: Assigned)
According to bisection, this assertion failure started at master@{#378969}:

"Remove StopDetector" / https://codereview.chromium.org/1749633002
I may understand the reason of this failure. I'm now making a patch...
Labels: -Sheriff-Chromium
Bug has an owner. Removing from Sheriff queue. Thanks a lot for looking into this!
Project Member

Comment 13 by bugdroid1@chromium.org, Mar 24 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0fdc4685d3b7e5134109b814eb08d8c265cdb84c

commit 0fdc4685d3b7e5134109b814eb08d8c265cdb84c
Author: nhiroki <nhiroki@chromium.org>
Date: Thu Mar 24 10:23:41 2016

Worker: Move an assertion to a correct place

This CL fixes an assertion failure happening only on non-oilpan builds.

WorkerThread::performShutdownTask() assumes that the count of references to
WorkerGlobalScope is only one. However, WorkerThreadableLoader that has a
reference to the context is still alive at the point and the assertion fails.
This CL moves the assertion to after the line to dispose the loader.

<More details>

The timing to release a reference to the context owned by the loader was changed
by this patch: https://codereview.chromium.org/1749633002

Before the patch, GlobalFetchImpl::StopDetector was an ActiveDOMObject and was
supposed to release the reference before performShutdownTask() as follows:

  WorkerThread::shutdown()
    WorkerGlobalScope::dispose()
      stopActiveDOMObjects();
        StopDetector::stop()
          FetchManager::stop()
            FetchManager::Loader::dispose()
              // Release WorkerThreadableLoader.
  WorkerThread::performShutdownTask()
    // The assertion can pass.

The patch removed StopDetector and instead made FetchManager::Loader to observe
lifecycle of the context to release the reference. The context destruction is
notified after the assertion as follows:

  WorkerThread::shutdown()
  WorkerThread::performShutdownTask()
    // The assertion cannot pass.
    WorkerGlobalScope::notifyContextDestroyed()
      LifecycleNotifier::notifyContextDestroyed()
        FetchManager::contextDestroyed()
          FetchManager::Loader::dispose()
            // Release WorkerThreadableLoader.

BUG= 594230 

Review URL: https://codereview.chromium.org/1831743002

Cr-Commit-Position: refs/heads/master@{#383048}

[modify] https://crrev.com/0fdc4685d3b7e5134109b814eb08d8c265cdb84c/third_party/WebKit/LayoutTests/TestExpectations
[modify] https://crrev.com/0fdc4685d3b7e5134109b814eb08d8c265cdb84c/third_party/WebKit/Source/core/workers/WorkerThread.cpp

Labels: M-51 OS-Mac
Status: Fixed (was: Started)
Project Member

Comment 15 by bugdroid1@chromium.org, Mar 25 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/355fabfffabf57f1f49808ddfa485347dc8695f6

commit 355fabfffabf57f1f49808ddfa485347dc8695f6
Author: nhiroki <nhiroki@chromium.org>
Date: Fri Mar 25 02:25:53 2016

Remove "Crash" mark for http/tests/serviceworker/fetch-event-network-error.html

This crash was fixed by https://codereview.chromium.org/1831743002

BUG= 594230 ,  596703 
TBR=nhiroki@chromium.org

Review URL: https://codereview.chromium.org/1827243004

Cr-Commit-Position: refs/heads/master@{#383228}

[modify] https://crrev.com/355fabfffabf57f1f49808ddfa485347dc8695f6/third_party/WebKit/LayoutTests/TestExpectations

Labels: Test-Layout

Sign in to add a comment