New issue
Advanced search Search tips

Issue 808164 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Closed: Apr 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 1
Type: Bug



Sign in to add a comment

Lock-order-inversion in blink::MediaElementAudioSourceHandler::SetFormat

Project Member Reported by ClusterFuzz, Feb 1 2018

Issue description

Detailed report: https://clusterfuzz.com/testcase?key=6213636226547712

Fuzzer: ochang_domfuzzer
Job Type: linux_tsan_chrome_mp
Platform Id: linux

Crash Type: Lock-order-inversion
Crash Address: 
Crash State:
  blink::MediaElementAudioSourceHandler::SetFormat
  blink::MediaElementAudioSourceNode::SetFormat
  blink::HTMLMediaElement::AudioClientImpl::SetFormat
  
Sanitizer: thread (TSAN)

Regressed: https://clusterfuzz.com/revisions?job=linux_tsan_chrome_mp&range=514498:517698

Reproducer Testcase: https://clusterfuzz.com/download?testcase_id=6213636226547712

Additional requirements: Requires HTTP

Issue filed automatically.

See https://github.com/google/clusterfuzz-tools for more information.
 
Project Member

Comment 1 by ClusterFuzz, Feb 1 2018

Components: Blink>Internals>WTF Blink>WebAudio
Labels: Test-Predator-Auto-Components
Automatically applying components based on crash stacktrace and information from OWNERS files.

If this is incorrect, please apply the Test-Predator-Wrong-Components label.
Project Member

Comment 2 by ClusterFuzz, Feb 1 2018

Cc: vkuzkokov@chromium.org lucmult@chromium.org gyuyoung...@lge.com
Labels: Test-Predator-Auto-CC
Automatically adding ccs based on suspected regression changelists:

Pass scalar types to callback by value. by vkuzkokov@chromium.org - https://chromium.googlesource.com/chromium/src/+/6d208e1cee4179c87775f26a7971d3f7f466fce7

Replace WTF:MakeUnique by std::make_unique in modules/indexeddb/ and by lucmult@chromium.org - https://chromium.googlesource.com/chromium/src/+/0908d7c68c20dc3614dcfcf75a68c13286b16421

Replace WTF::AdoptRef with base::AdoptRef in /modules of blink by gyuyoung.kim@lge.com - https://chromium.googlesource.com/chromium/src/+/f6287b3c44a53e496b480b93137c4d159b8464b4

If this is incorrect, please let us know why and apply the Test-Predator-Wrong-CLs label.
Cc: dalecur...@chromium.org
Components: Internals>Media
dalecurtis@

This part of code predates me, so could you help me to understand?

1. WebAudioSourceProviderImpl::SetClient() holds the lock:
https://cs.chromium.org/chromium/src/media/blink/webaudiosourceprovider_impl.cc?q=webaudiosourceprovider&sq=package:chromium&l=114

2. In the same lock scope, WebAudioSourceProviderImpl::OnSetFormat() tries to acquire the same lock:
https://cs.chromium.org/chromium/src/media/blink/webaudiosourceprovider_impl.cc?sq=package:chromium&l=302

I don't understand why BindToCurrentLoop() and base::ResetAndReturn() need to be done (perhaps to preserve the life of the object?), perhaps only one of them needs to hold the lock somehow?

The stack trace from CF suggests that this is a lock-order-inversion in 3 locks, but I think it might be wrong.

WDYT?
2 is not in the same lock scope. 2 is a posted task that runs after SetClient() completes. If initialize hasn't completed we need to run the set_format_Cb_ later. It's essentially just a base::OnceCallback, ResetAndReturn is the old way of handling that.
Owner: hongchan@chromium.org
Status: Assigned (was: Untriaged)
Thanks! Yes, that was my guess but wanted to double check by you.

I believe this is same with the  issue 786739 , but the offending case was actually verified by CF. I guess I'll have to look into this issue further.

Comment 6 by yutak@chromium.org, Feb 2 2018

Components: -Blink>Internals>WTF
Project Member

Comment 7 by ClusterFuzz, Feb 16 2018

Status: WontFix (was: Assigned)
ClusterFuzz testcase 6213636226547712 is flaky and no longer crashes, so closing issue.

If this is incorrect, please add ClusterFuzz-Wrong label and re-open the issue.
Cc: xhw...@chromium.org
 Issue 818122  has been merged into this issue.
Status: Started (was: WontFix)
This issue came back up in the same spot. I am reopening this.
I tried to reproduce the cf test case locally, but I couldn't do it. Perhaps this is super flaky or needs a low-end machine to reproduce.
The most recent CF repro case:
https://clusterfuzz.com/v2/testcase-detail/5383623981924352?noredirect=1

In the stack trace in the test case above, I found a weird deadlock. Here's the summary:

M16805: WebAudioSourceProviderImpl::sink_lock (while M16645 locked)
media::WebAudioSourceProviderImpl::SetClient()
blink::HTMLMediaElement::StartPlayerLoad()
blink::HTMLMediaElement::LoadResource()
blink::HTMLMediaElement::LoadSourceFromAttribute()
blink::HTMLMediaElement::SelectMediaResource()
blink::HTMLMediaElement::LoadInternal()
blink::HTMLMediaElement::LoadTimerFired()

M16645: WebAudioSourceProviderImpl::sink_lock
media::WebAudioSourceProviderImpl::SetClient()
blink::HTMLMediaElement::StartPlayerLoad()
blink::HTMLMediaElement::LoadResource()
blink::HTMLMediaElement::LoadSourceFromAttribute()
blink::HTMLMediaElement::SelectMediaResource()
blink::HTMLMediaElement::LoadInternal()
blink::HTMLMediaElement::LoadTimerFired()

(code location: https://cs.chromium.org/chromium/src/media/blink/webaudiosourceprovider_impl.cc)

Apparently SetClient() function got called twice while the first mutex is still locked. My guess is the second call is the result of a posted task after the first call failed. But the first call's mutex should be release at the end of the function scope.

dalecurtis@ WDYT?
Cc: -xhw...@chromium.org
Project Member

Comment 13 by ClusterFuzz, Apr 24 2018

Labels: ClusterFuzz-Verified
Status: Verified (was: Started)
ClusterFuzz testcase 5383623981924352 is verified as fixed, so closing issue as verified.

If this is incorrect, please add ClusterFuzz-Wrong label and re-open the issue.

Sign in to add a comment