New issue
Advanced search Search tips

Issue 593560 link

Starred by 15 users

Issue metadata

Status: Verified
Owner:
Closed: Mar 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug



Sign in to add a comment

MediaRecorder pause/resume doesn't stop timeStamp

Reported by rst...@usertesting.com, Mar 10 2016

Issue description

UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.87 Safari/537.36

Example URL:
https://codepen.io/rstormsf/pen/mPPvXP

Steps to reproduce the problem:
1. Start MediaRecorder
2. Pause MediaRecorder
3. Resume MediaRecorder
4. Stop MediaRecorder

What is the expected behavior?
It should not create any empty gaps when whole array of blobs is saved to FileSystem. It is not supposed to append any data in paused state. 

What went wrong?
when I record just video via getUserMedia {video :true}, it appends empty time to the final video because timeStamp is still counting.
When recording both audio and video, audio is still recorded regardless.

Did this work before? No 

Is it a problem with Flash or HTML5? HTML5

Does this work in other browsers? Yes 

Chrome version: 49.0.2623.87  Channel: stable
OS Version: OS X 10.11.3
Flash Version: Shockwave Flash 21.0 r0

https://codepen.io/rstormsf/pen/mPPvXP - to reproduce the issue
https://gist.github.com/rstormsf/e00345e302f442721249 - filesystem helper function.
 
rstorm@- I do not see any data appended in paused state using your example URL. I am using Version 51.0.2672.0 on linux. When recorder is resumed, the timestamps progress as they should every 1000ms in your case. 
What kind of gap do you see? Do you mean that you get an empty Blob when the video is paused? I have not been able to reproduce this.
Try to record the video and watch it as a whole. It adds extra time which shouldn't be there. Lets say you record for 10 seconds, pause for 5 seconds and record for 10 seconds more. Total video length should be 20 seconds. But we get 25 with 5 seconds from paused state. How do I fix it? 

You are right, it doesn't append any new data in Paused state, but it does contain metadata from timestamps, that's why final video has extra N paused seconds. It seems like video timestamps have to be re-written. I haven't tested it with audio streams yet.
another codepen.io with multiple files. Seems like it's not happy with timestamps. https://codepen.io/rstormsf/pen/yOJMyy
cpaulin@ running ffmpeg:
ffmpeg -i video-2.webm -c:v copy out.webm
[webm @ 0x7f803100b000] Non-monotonous DTS in output stream 0:0; previous: 22403, current: 6010; changing to 22403. This may result in incorrect timestamps in the output file.
bunch of errors about wrong timeStamps

Owner: mcasas@chromium.org
Status: Assigned (was: Unconfirmed)
rstorm@- I confirm the issue, an example video attached in issue.
I recorded video for 10 seconds, then paused for 5 seconds and resumed for 10 seconds. Total video length should be 20 seconds, but the recorder added 5 seconds of pause in video. The Video is of a stop watch that is started roughly at the same time as the recorder start time.
timestampskipping.webm
2.5 MB Download
Components: Blink>MediaStreamRecording

Comment 8 by rstor...@gmail.com, Mar 17 2016

I would love to know if it's planned to be fixed soon. Any ETA? 

Comment 9 by mcasas@chromium.org, Mar 17 2016

Labels: M-51
rstormfs@ - probably during the next week. 

The issue is that the webm timestamp is derived directly
from the AudioBus/VideoFrames timestamp [1] and that keeps
running while the Audio/Video recorders are in pause. Will
need some piping.

[1] https://code.google.com/p/chromium/codesearch#chromium/src/media/muxers/webm_muxer.cc&sq=package:chromium&type=cs&q=webmmuxer&l=272

Comment 10 by rstor...@gmail.com, Mar 18 2016

mcasas@ Awesome! I hope this claim can be proven next week. 

FYI seems like for Audio stream, it doesn't work at all. I mean it contains whole audio data. 
mcasas@ it seems like the code only fixes audiostream. Will there be another PR for timeStamps issue in VideoStream? 
Status: Started (was: Assigned)
rstorm@ you just beat me to it :)

I think the CL in #12 solves part of the problem, but might
not be enough: it adds Pause/Resume for AudioTrackRecorder,
so we won't keep recording Audio while on Pause(), which is
a good thing and addresses your comment #11. But it won't fix
the timestamps completely; now, a well behaved player should 
present black frames + silence while the playback clock advances
through recorded periods of "pause" - however, not all players
follow the rules of logic. I would be interested in seeing
what VLC and/or Chrome does when presented with a video with
a pause now. 

In // I'll work on the real pause thing.
VLC just jumps back and forth depending on the timestamp. Chrome Web Video Browser player just shows the last frame until the next frame is present while the time is ticking. 
How do I know the build number for the AudioStream code that is going to be released? Is it in Canary dev channel? Where can I see the version of submitted bug fix? 
#15: go to https://omahaproxy.appspot.com/
and check the branch_base_position column
versus the landed version of the CL
(line Cr-Commit-Position: refs/heads/master@{#382668}),
that'll tell when Canary (or any other channel)
gets the CL.
Which means this bug fix for AudioStream could be verified in Mac Canary channel if I read it correctly
Project Member

Comment 18 by bugdroid1@chromium.org, Mar 28 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/768a9b746767f4a4b64a199fd68f9985c7da7ba3

commit 768a9b746767f4a4b64a199fd68f9985c7da7ba3
Author: mcasas <mcasas@chromium.org>
Date: Mon Mar 28 06:30:33 2016

WebmMuxer: add Pause()/Resume() methods and internal time-in-pause

See bug for more details, but basically, since WebmMuxer relies on the
incoming video/audio packet timestamp, pausing the recorder (not any
associated MediaStream{Video,Audio}Track), leaves empty gaps in the
output .webm file.

This CL solves this by measuring the elapsed time(s) between Pause()
and Resume(), accumulating the elapsed time(s), then subtracting that
from the output webm file timestamp.

BUG= 593560 
TEST= manual:
- head for QA page [1] (with --enable-blink-features=GetUserMedia)
- click on "GetUSerMedia"
- select "Stream To Record" "local"
- click "CreateRecorder" then "Start".
- Leave it recording for a A seconds.
- Click "Pause" and wait a B seconds.
- Click "Resume" and wait C seconds.
- Click "Stop" and "Download".
- Play back the generated file and verify that there is
no substantial freezing in the recorded sequence and that
the length of the output file is estimated in A+C seconds.

[1] https://cdn.rawgit.com/cricdecyan/mediarecorder/master/manualtest/index.html

Review URL: https://codereview.chromium.org/1830383003

Cr-Commit-Position: refs/heads/master@{#383481}

[modify] https://crrev.com/768a9b746767f4a4b64a199fd68f9985c7da7ba3/content/renderer/media/audio_track_recorder.cc
[modify] https://crrev.com/768a9b746767f4a4b64a199fd68f9985c7da7ba3/content/renderer/media/media_recorder_handler.cc
[modify] https://crrev.com/768a9b746767f4a4b64a199fd68f9985c7da7ba3/media/muxers/webm_muxer.cc
[modify] https://crrev.com/768a9b746767f4a4b64a199fd68f9985c7da7ba3/media/muxers/webm_muxer.h

Status: Fixed (was: Started)
rstorm@ could you please verify?
(with the appropriate Canary)

Comment 20 Deleted

mcasas@
looks like Mac canary branch_base_position is 383454 while your Cr is 383481
How do I verify it against your branch? 

#21: seems indeed like yesterday's Canary was cut before
#18 landed, so I'd say wait for the next cut and try. You
could always use a ToT Chromium build if you want, but 
I'd also verify with Canary -- controlled build etc.
mcasas@
it seems like it's working now in canary channel. Thank you very much for your fix. I really appreciate it. All I can add is that it would be great if someone fixes https://bugs.chromium.org/p/chromium/issues/detail?id=223639 for OSX (ability to use getUserMedia with both streams - audio and video using chrome.desktopCapture API) 
Can't wait to see it in stable channel.
I was able to verify the fix using the same test case from https://codepen.io/rstormsf/pen/mPPvXP
Verified in Version 51.0.2693.2 Mac Canary
Labels: -OS-Mac OS-All
Status: Verified (was: Fixed)
rstorm@ thanks for verifying.

From what I can gather, Desktop audio capture works on
Win and CrOS but nowhere else due to lack of OS support,
correct?
mcasas@
The problem is in getUserMedia call.
What we can:
1){video: true, audio: true} - grabs webcam and microphone
What we cannot:
2) navigator.webkitGetUserMedia({
      video: {
        mandatory: {
            chromeMediaSourceId: id, 
            chromeMediaSource:'desktop'
        }
      },
      audio: {
          mandatory: {
            chromeMediaSource: 'system',
            chromeMediaSourceId: id, 
          }
      }
- doesn't work. Honestly, I don't know all the reasons why it is not possible to implement it this way.

rstorm@: perhaps is best to make a bug with the precise
description so we can investigate and/or at least trace 
the issue to the root causes. Please add me as cc: and 
I'll pull in some other fellows. Cheers!
Components: -Blink>MediaStreamRecording Blink>MediaStream>Recording
Renamed component Blink>MediaStreamRecording to Blink>MediaStream>Recording. Moving issues to the new component. 
Components: Blink>MediaRecording
Components: -Blink>MediaStream>Recording

Sign in to add a comment