New issue
Advanced search Search tips

Issue 904091 link

Starred by 1 user

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 2
Type: Bug

Blocking:
issue 760264
issue 842639



Sign in to add a comment

CHECK failure: buffers_.empty() || CanAppendBuffersToEnd(new_buffers, new_buffers_group_start_p

Project Member Reported by ClusterFuzz, Nov 10

Issue description

Detailed report: https://clusterfuzz.com/testcase?key=5719252826587136

Fuzzer: afl_mediasource_MP4_AACLC_AVC_pipeline_integration_fuzzer
Job Type: afl_chrome_asan
Platform Id: linux

Crash Type: CHECK failure
Crash Address: 
Crash State:
  buffers_.empty() || CanAppendBuffersToEnd(new_buffers, new_buffers_group_start_p
  media::SourceBufferRangeByPts::AppendBuffersToEnd
  media::SourceBufferStream<media::SourceBufferRangeByPts>::Append
  
Sanitizer: address (ASAN)

Regressed: https://clusterfuzz.com/revisions?job=afl_chrome_asan&range=532267:532325

Reproducer Testcase: https://clusterfuzz.com/download?testcase_id=5719252826587136

Issue filed automatically.

See https://chromium.googlesource.com/chromium/src/+/master/testing/libfuzzer/reference.md for more information.
 
Project Member

Comment 1 by ClusterFuzz, Nov 10

Components: Internals>Media
Labels: Test-Predator-Auto-Components
Automatically applying components based on crash stacktrace and information from OWNERS files.

If this is incorrect, please apply the Test-Predator-Wrong-Components label.
Project Member

Comment 2 by ClusterFuzz, Nov 10

Cc: xhw...@chromium.org
Labels: ClusterFuzz-Auto-CC
Automatically adding ccs based on OWNERS file / target commit history.

If this is incorrect, please add ClusterFuzz-Wrong label.
Project Member

Comment 3 by ClusterFuzz, Nov 10

Labels: Test-Predator-Auto-Owner
Owner: wolenetz@chromium.org
Status: Assigned (was: Untriaged)
Automatically assigning owner based on suspected regression changelist https://chromium.googlesource.com/chromium/src/+/fdeaf6e25bb2d2596de09eca9a902ea98e70d84c (MSE: Use |highest_timestamp_in_append_sequence_| in PotentialNextAppendTimestamp()).

If this is incorrect, please let us know why and apply the Test-Predator-Wrong-CLs label. If you aren't the correct owner for this issue, please unassign yourself as soon as possible so it can be re-triaged.
Components: -Internals>Media Internals>Media>Source
Excellent. This is correctly triaged. I'll take a look.
I have a local repro.
Blocking: 760264
It'll take a bit of log-digging to understand the repro. On the surface, it looks like there might be an issue with splicing SAP-Type-2 GOP sequence (KF + 1 non-KF w/TS<KF + 1 non-KF w/TS following KF} into the middle of an existing range (or the end of one), possibly involving an off-by-1-microsecond in the tracking of next_gop_timestamp versus the actual keyframe timestamp(higher by 1 us). 
Failed while trying to append:
Previous state: (excluding audio track/stream) video has 2 ranges, each 15 buffers beginning with 1 keyframe:
[1113/134129.886249:VERBOSE1:source_buffer_stream.cc(500)] Append VIDEO: done. ranges_=[567233us;1034366us(1067732us)] [1159000us;2237262433us(2237295799us)]

Then:

[1113/134130.035685:VERBOSE3:frame_processor.cc(671)] ProcessFrame: Processing frame Type=2, TrackID=1, PTS=1067733us, DTS=1001000us, DUR=33366us, RAP=1
[1113/134130.035793:VERBOSE2:frame_processor.cc(527)] FlushProcessedFrames()
[1113/134130.035854:VERBOSE2:chunk_demuxer.cc(258)] ChunkDemuxerStream::OnStartOfCodedFrameGroup(dts 0.967633, pts 1.06773)
[1113/134130.035938:VERBOSE1:source_buffer_stream.cc(254)] OnStartOfCodedFrameGroup VIDEO (dts 967633us, pts 1067732us)
[1113/134130.036008:VERBOSE1:source_buffer_range_by_pts.cc(517)] BelongsToRange
[1113/134130.036061:VERBOSE4:source_buffer_range_by_pts.cc(518)] keyframe_map_index_base_=0, buffers.size()=15, keyframe_map_.size()=1, keyframe_map_:
	 pts 567233, unadjusted idx = 0

[1113/134130.036165:VERBOSE4:source_buffer_stream.cc(1497)] IsNextGopAdjacentToEndOfCurrentAppendSequence VIDEO next_gop_timestamp=1067732us, highest_timestamp_in_append_sequence_=1034366us, upper_bound=1101100us
[1113/134130.036241:VERBOSE3:source_buffer_stream.cc(302)] OnStartOfCodedFrameGroupInternal next appended buffers will continue range unless intervening remove makes discontinuity
[1113/134130.036338:VERBOSE3:frame_processor.cc(996)] ProcessFrame: Sending processed frame to stream, PTS=1067733us, DTS=1001000us
[1113/134130.036419:VERBOSE3:frame_processor.cc(671)] ProcessFrame: Processing frame Type=2, TrackID=1, PTS=1034366us, DTS=1034366us, DUR=33366us, RAP=0
[1113/134130.036514:VERBOSE3:frame_processor.cc(996)] ProcessFrame: Sending processed frame to stream, PTS=1034366us, DTS=1034366us
...
[1113/134130.036775:VERBOSE3:frame_processor.cc(671)] ProcessFrame: Processing frame Type=2, TrackID=1, PTS=1101100us, DTS=1067733us, DUR=33366us, RAP=0
[1113/134130.036868:VERBOSE3:frame_processor.cc(996)] ProcessFrame: Sending processed frame to stream, PTS=1101100us, DTS=1067733us
...
[1113/134130.037054:VERBOSE2:frame_processor.cc(527)] FlushProcessedFrames()
[1113/134130.037116:VERBOSE1:source_buffer_stream.cc(317)] Append VIDEO: buffers dts=[1001000us;1067733us(last frame dur=33366us)], pts interval=[1034366us,1134466us)
[1113/134130.037233:VERBOSE4:source_buffer_stream.cc(319)] Buffers:
	dts=1001000 timestamp=1067733 duration=33366 size=8512 side_data_size=0 is_key_frame=1 encrypted=0 discard_padding (us)=(0, 0), is_duration_estimated=0
	dts=1034366 timestamp=1034366 duration=33366 size=1687 side_data_size=0 is_key_frame=0 encrypted=0 discard_padding (us)=(0, 0), is_duration_estimated=0
	dts=1067733 timestamp=1101100 duration=33366 size=523 side_data_size=0 is_key_frame=0 encrypted=0 discard_padding (us)=(0, 0), is_duration_estimated=0

--> I'll see if I can put together a minimized unit test repro for this and go from there...
Labels: -Pri-1 Pri-2
Scenario is understood. Minimized unit test repros it.
Essentially, a SAP-Type-2 GOP overlapping the end of an out-of-order decode previously buffered GOP could cause removal during splice of enough dependent frames on the overlapped frame such that the new GOP is no longer adjacent within fudge room of the selected range.

This is a fairly complicated scenario which I'll need to consider. In self-contained continuous streams, it shouldn't really occur. But at splice points involving SAP-Type-2 overlapping some previous out-of-order GOP, the implementation can hit this CHECK currently.
Labels: M-72
WIP CL (just unit test repro'ing the issue at the moment): https://chromium-review.googlesource.com/c/chromium/src/+/1334833
If there is a decode-time discontinuity between the two GOPs, crash is avoided (in my unit test).

Or if the second GOP is appended frame-by-frame, crash is avoided (in my unit test), regardless of whether or not the two GOPs are continuous in DTS.

Fix might be to trigger discontinuity recovery logic somehow, when this scenario is detected. Still investigating...
This is complicated :/

Either full-on fixing of SBRByPts to understand SAP-Type-2 presentation intervals at start of range, or a more targeted fix that perhaps:

1) (not performant, as audio byPts perf issue has shown): flush frame processor on each keyframe that started a new coded frame group (essentially trigger "working" frame-by-frame behavior for at least keyframes), or

2) Detect when such a flush might be needed (e.g., on next frame for that track buffer, iff current logic currently doesn't indicate to flush but operating in ByPts RangeAPI and PTS decreases (for non-keyframe)) (still not great, but more targeted to flushing only when needed).

Of these, I'll try approach (2) first.
@#13: (2) might cause stall at playback across such overlaps due to lack of (1). However, this is like (but not precisely the same as) other known stalls that can be introduced when a new GOP overlaps the end of a previous out-of-order GOP sufficiently enough to trigger removal of dependencies and introduce a gap. Upcoming MSE vNext work to allow apps to have more control over what size gap might be a stall (or what to do when playback reaches a gap) can help resolve such issues.
@#14: Actually, existing byPts logic can be leveraged to help introduction of these gaps in this particular case. I think I have an approach like #2 which might work:

If not otherwise already signalling a new CFG (and flushing processed frames) when processing a frame, before enqueuing it, do a flush first IF:
1) buffering byPts, AND
2) frame is not a keyframe, AND
3) frame's PTS is below the PTS of its GOP, AND
4) there has not already been a flush for a previous nonkeyframe in this same GOP

This lets the initial SAP-type-1 portion of a SAP-type-2 GOP become buffered as a new GOP in a SBRByPts, then subsequent (continuous in DTS, since not otherwise dropping/signalling a CFG). Any further nonkeyframes after the flush will continue to be buffered into the same GOP.

I've some partial unit tests that repro the issue and that, when adjusted to append the SAP-Type-2 GOP frame-by-frame, no longer repros the issue. The approach suggested in previous paragraph will involve at most one additional flush per SAP-Type-2 GOP.
Blocking: 842639

Sign in to add a comment