CHECK failure: buffers_.empty() || CanAppendBuffersToEnd(new_buffers, new_buffers_group_start_p |
||||||||
Issue descriptionDetailed report: https://clusterfuzz.com/testcase?key=5719252826587136 Fuzzer: afl_mediasource_MP4_AACLC_AVC_pipeline_integration_fuzzer Job Type: afl_chrome_asan Platform Id: linux Crash Type: CHECK failure Crash Address: Crash State: buffers_.empty() || CanAppendBuffersToEnd(new_buffers, new_buffers_group_start_p media::SourceBufferRangeByPts::AppendBuffersToEnd media::SourceBufferStream<media::SourceBufferRangeByPts>::Append Sanitizer: address (ASAN) Regressed: https://clusterfuzz.com/revisions?job=afl_chrome_asan&range=532267:532325 Reproducer Testcase: https://clusterfuzz.com/download?testcase_id=5719252826587136 Issue filed automatically. See https://chromium.googlesource.com/chromium/src/+/master/testing/libfuzzer/reference.md for more information.
,
Nov 10
Automatically adding ccs based on OWNERS file / target commit history. If this is incorrect, please add ClusterFuzz-Wrong label.
,
Nov 10
Automatically assigning owner based on suspected regression changelist https://chromium.googlesource.com/chromium/src/+/fdeaf6e25bb2d2596de09eca9a902ea98e70d84c (MSE: Use |highest_timestamp_in_append_sequence_| in PotentialNextAppendTimestamp()). If this is incorrect, please let us know why and apply the Test-Predator-Wrong-CLs label. If you aren't the correct owner for this issue, please unassign yourself as soon as possible so it can be re-triaged.
,
Nov 12
Excellent. This is correctly triaged. I'll take a look.
,
Nov 13
I have a local repro.
,
Nov 13
,
Nov 13
It'll take a bit of log-digging to understand the repro. On the surface, it looks like there might be an issue with splicing SAP-Type-2 GOP sequence (KF + 1 non-KF w/TS<KF + 1 non-KF w/TS following KF} into the middle of an existing range (or the end of one), possibly involving an off-by-1-microsecond in the tracking of next_gop_timestamp versus the actual keyframe timestamp(higher by 1 us).
,
Nov 13
Failed while trying to append: Previous state: (excluding audio track/stream) video has 2 ranges, each 15 buffers beginning with 1 keyframe: [1113/134129.886249:VERBOSE1:source_buffer_stream.cc(500)] Append VIDEO: done. ranges_=[567233us;1034366us(1067732us)] [1159000us;2237262433us(2237295799us)] Then: [1113/134130.035685:VERBOSE3:frame_processor.cc(671)] ProcessFrame: Processing frame Type=2, TrackID=1, PTS=1067733us, DTS=1001000us, DUR=33366us, RAP=1 [1113/134130.035793:VERBOSE2:frame_processor.cc(527)] FlushProcessedFrames() [1113/134130.035854:VERBOSE2:chunk_demuxer.cc(258)] ChunkDemuxerStream::OnStartOfCodedFrameGroup(dts 0.967633, pts 1.06773) [1113/134130.035938:VERBOSE1:source_buffer_stream.cc(254)] OnStartOfCodedFrameGroup VIDEO (dts 967633us, pts 1067732us) [1113/134130.036008:VERBOSE1:source_buffer_range_by_pts.cc(517)] BelongsToRange [1113/134130.036061:VERBOSE4:source_buffer_range_by_pts.cc(518)] keyframe_map_index_base_=0, buffers.size()=15, keyframe_map_.size()=1, keyframe_map_: pts 567233, unadjusted idx = 0 [1113/134130.036165:VERBOSE4:source_buffer_stream.cc(1497)] IsNextGopAdjacentToEndOfCurrentAppendSequence VIDEO next_gop_timestamp=1067732us, highest_timestamp_in_append_sequence_=1034366us, upper_bound=1101100us [1113/134130.036241:VERBOSE3:source_buffer_stream.cc(302)] OnStartOfCodedFrameGroupInternal next appended buffers will continue range unless intervening remove makes discontinuity [1113/134130.036338:VERBOSE3:frame_processor.cc(996)] ProcessFrame: Sending processed frame to stream, PTS=1067733us, DTS=1001000us [1113/134130.036419:VERBOSE3:frame_processor.cc(671)] ProcessFrame: Processing frame Type=2, TrackID=1, PTS=1034366us, DTS=1034366us, DUR=33366us, RAP=0 [1113/134130.036514:VERBOSE3:frame_processor.cc(996)] ProcessFrame: Sending processed frame to stream, PTS=1034366us, DTS=1034366us ... [1113/134130.036775:VERBOSE3:frame_processor.cc(671)] ProcessFrame: Processing frame Type=2, TrackID=1, PTS=1101100us, DTS=1067733us, DUR=33366us, RAP=0 [1113/134130.036868:VERBOSE3:frame_processor.cc(996)] ProcessFrame: Sending processed frame to stream, PTS=1101100us, DTS=1067733us ... [1113/134130.037054:VERBOSE2:frame_processor.cc(527)] FlushProcessedFrames() [1113/134130.037116:VERBOSE1:source_buffer_stream.cc(317)] Append VIDEO: buffers dts=[1001000us;1067733us(last frame dur=33366us)], pts interval=[1034366us,1134466us) [1113/134130.037233:VERBOSE4:source_buffer_stream.cc(319)] Buffers: dts=1001000 timestamp=1067733 duration=33366 size=8512 side_data_size=0 is_key_frame=1 encrypted=0 discard_padding (us)=(0, 0), is_duration_estimated=0 dts=1034366 timestamp=1034366 duration=33366 size=1687 side_data_size=0 is_key_frame=0 encrypted=0 discard_padding (us)=(0, 0), is_duration_estimated=0 dts=1067733 timestamp=1101100 duration=33366 size=523 side_data_size=0 is_key_frame=0 encrypted=0 discard_padding (us)=(0, 0), is_duration_estimated=0 --> I'll see if I can put together a minimized unit test repro for this and go from there...
,
Nov 14
Scenario is understood. Minimized unit test repros it. Essentially, a SAP-Type-2 GOP overlapping the end of an out-of-order decode previously buffered GOP could cause removal during splice of enough dependent frames on the overlapped frame such that the new GOP is no longer adjacent within fudge room of the selected range. This is a fairly complicated scenario which I'll need to consider. In self-contained continuous streams, it shouldn't really occur. But at splice points involving SAP-Type-2 overlapping some previous out-of-order GOP, the implementation can hit this CHECK currently.
,
Nov 14
,
Nov 14
WIP CL (just unit test repro'ing the issue at the moment): https://chromium-review.googlesource.com/c/chromium/src/+/1334833
,
Nov 20
If there is a decode-time discontinuity between the two GOPs, crash is avoided (in my unit test). Or if the second GOP is appended frame-by-frame, crash is avoided (in my unit test), regardless of whether or not the two GOPs are continuous in DTS. Fix might be to trigger discontinuity recovery logic somehow, when this scenario is detected. Still investigating...
,
Nov 20
This is complicated :/ Either full-on fixing of SBRByPts to understand SAP-Type-2 presentation intervals at start of range, or a more targeted fix that perhaps: 1) (not performant, as audio byPts perf issue has shown): flush frame processor on each keyframe that started a new coded frame group (essentially trigger "working" frame-by-frame behavior for at least keyframes), or 2) Detect when such a flush might be needed (e.g., on next frame for that track buffer, iff current logic currently doesn't indicate to flush but operating in ByPts RangeAPI and PTS decreases (for non-keyframe)) (still not great, but more targeted to flushing only when needed). Of these, I'll try approach (2) first.
,
Dec 1
@#13: (2) might cause stall at playback across such overlaps due to lack of (1). However, this is like (but not precisely the same as) other known stalls that can be introduced when a new GOP overlaps the end of a previous out-of-order GOP sufficiently enough to trigger removal of dependencies and introduce a gap. Upcoming MSE vNext work to allow apps to have more control over what size gap might be a stall (or what to do when playback reaches a gap) can help resolve such issues.
,
Dec 1
@#14: Actually, existing byPts logic can be leveraged to help introduction of these gaps in this particular case. I think I have an approach like #2 which might work: If not otherwise already signalling a new CFG (and flushing processed frames) when processing a frame, before enqueuing it, do a flush first IF: 1) buffering byPts, AND 2) frame is not a keyframe, AND 3) frame's PTS is below the PTS of its GOP, AND 4) there has not already been a flush for a previous nonkeyframe in this same GOP This lets the initial SAP-type-1 portion of a SAP-type-2 GOP become buffered as a new GOP in a SBRByPts, then subsequent (continuous in DTS, since not otherwise dropping/signalling a CFG). Any further nonkeyframes after the flush will continue to be buffered into the same GOP. I've some partial unit tests that repro the issue and that, when adjusted to append the SAP-Type-2 GOP frame-by-frame, no longer repros the issue. The approach suggested in previous paragraph will involve at most one additional flush per SAP-Type-2 GOP.
,
Dec 6
|
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by ClusterFuzz
, Nov 10Labels: Test-Predator-Auto-Components