New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 760928 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner:
Closed: Sep 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Mac
Pri: 2
Type: Bug



Sign in to add a comment

Out-of-memory in mediasource_WEBM_VP9_pipeline_integration_fuzzer

Project Member Reported by ClusterFuzz, Aug 31 2017

Issue description

Detailed report: https://clusterfuzz.com/testcase?key=5541056488931328

Fuzzer: libFuzzer_mediasource_WEBM_VP9_pipeline_integration_fuzzer
Job Type: libfuzzer_chrome_msan
Platform Id: linux

Crash Type: Out-of-memory (exceeds 2048 MB)
Crash Address: 
Crash State:
  mediasource_WEBM_VP9_pipeline_integration_fuzzer
  
Sanitizer: memory (MSAN)

Regressed: https://clusterfuzz.com/revisions?job=libfuzzer_chrome_msan&range=497057:497112

Reproducer Testcase: https://clusterfuzz.com/download?testcase_id=5541056488931328

Issue filed automatically.

See https://chromium.googlesource.com/chromium/src/+/master/testing/libfuzzer/reproducing.md for more information.
 
Cc: msrchandra@chromium.org
Labels: M-62 Test-Predator-Wrong
Owner: wolenetz@chromium.org
Status: Assigned (was: Untriaged)
Predator could not provide any possible suspects.
Assigning to the concern owner from CL --
https://chromium.googlesource.com/chromium/src/+log/0a707c2a1200365fa32192a8728c81b10a86824d..163cd2ca687a58b9918a7e5b4f696322a2286f79?pretty=fuller&n=10000

Suspecting Commit#
https://chromium.googlesource.com/chromium/src/+/0b378819e3b3925e0eabd34e73d3972a19b4a46d

@wolenetz -- Could you please look into the issue, kindly re-assign if this is not related to your changes.
Thank You.
Cc: mmoroz@chromium.org wolenetz@chromium.org
Components: Internals>Media
Labels: -Pri-1 Pri-2
Owner: dalecur...@chromium.org
I have a local repro.
With dcheck_always_on = true, and loglevel in the fuzzer changed from FATAL to VERBOSE-10, repro shows the following before a few seconds lapse followed by OOM:

...
VERBOSE1:decoder_stream.cc(277)] OnDecoderSelected<video>: VpxVideoDecoder
VERBOSE2:decoder_selector.cc(56)] ~DecoderSelector
VERBOSE1:renderer_impl.cc(422)] OnVideoRendererInitializeDone: 0
VERBOSE1:renderer_impl.cc(220)] StartPlayingFrom
VERBOSE1:wall_clock_time_source.cc(49)] SetMediaTime(0)
VERBOSE1:video_renderer_impl.cc(208)] StartPlayingFrom(0)
VERBOSE3:decoder_stream.cc(124)] Read<video>
VERBOSE3:decoder_stream.cc(538)] ReadFromDemuxerStream<video>
VERBOSE2:source_buffer_stream.cc(1327)] GetNextBuffer VIDEO
VERBOSE2:source_buffer_stream.cc(1332)] GetNextBuffer VIDEO: no pending buffer, returning status 0
VERBOSE2:chunk_demuxer.cc(383)] CompletePendingReadIfPossible_Locked: returning kOk, type 2, dts 0.062095, pts 0.062095, dur 0.063, key 1
VERBOSE1:renderer_impl.cc(238)] SetPlaybackRate(0)
VERBOSE1:wall_clock_time_source.cc(36)] SetPlaybackRate(0)
VERBOSE1:renderer_impl.cc(259)] SetVolume
VERBOSE1:pipeline_impl.cc(814)] kStarting -> kPlaying
VERBOSE3:decoder_stream.cc(565)] OnBufferReady<video>: 0, timestamp=62095 duration=63000 size=373 side_data_size=0 is_key_frame=1 encrypted=0 discard_padding (ms)=(0, 0)
VERBOSE3:decoder_stream.cc(352)] Decode<video>
VERBOSE3:decoder_stream.cc(377)] DecodeInternal<video>

==15898== ERROR: libFuzzer: out-of-memory (used: 2193Mb; limit: 2048Mb)

On a debug asan rebuild, no OOM, but multiple corrupt VPx frames are reported by the decoder and it appears there may also be some decoder fallback attempted by the pipeline.

tl;dr: It looks like VpxVideoDecoder can gobble lots of memory and hit OOM on some inputs.

=> dalecurtis@ to reassign in case this might be something that the pipeline or vpx decoder folks could mitigate.

cc+=mmoroz@ w.r.t. recent discussion about other OOMs on issue 759277 and  issue 758806 .

(Also, lowering priority. Doesn't appear to be P1 based on those recent discussions.)
Cc: johannko...@google.com fgalligan@chromium.org
It's possible this is expected, just constructing massive frames can lead to issues like this. +fgalligan, johannkoenig
Project Member

Comment 4 by ClusterFuzz, Sep 2 2017

Labels: OS-Mac
In chromium libvpx restricts the frame size to 16k by 16k[0], but fuzzed streams could still specify a large number of reference frames and make it to 2gb pretty easily.

[0] https://chromium.googlesource.com/chromium/src/+/master/third_party/libvpx/generate_gni.sh#338
"--size-limit=16384x16384"

Not sure how much more mitigation we want to add - fuzzed/broken files OOM'ing seems like a fairly reasonable response.
Status: WontFix (was: Assigned)
We should just bump the -rss_limit_mb for all msan failures. None of these seem to repro outside of msan, so the primary issue is overhead from msan. 4096 was sufficient for this to pass fine.

Comment 7 by mmoroz@chromium.org, Sep 30 2017

Cc: och...@chromium.org mbarbe...@chromium.org infe...@chromium.org kcc@chromium.org metzman@chromium.org
Yeah, we had a discussion some time ago about OOMs with MSan, but haven't found a good solution. The issue with rss_limit_mb=4096 is that our VMs have only 3.5G RAM and don't have swap. I filed a separate issue to discuss that: https://bugs.chromium.org/p/chromium/issues/detail?id=770430

Sign in to add a comment