New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 173 users
Status: Assigned
Owner:
Last visit > 30 days ago
Cc:
Components:
EstimatedDays: ----
NextAction: 2017-07-01
OS: Linux
Pri: 2
Type: Feature

Blocked on:
issue webrtc:8183

Blocking:
issue webrtc:5187
issue webrtc:8530


Show other hotlists

Hotlists containing this issue:
test
Top-Starred-Bugs


Sign in to add a comment
Need to implement WebRTC "Unified Plan" for multistream
Reported by randell....@gmail.com, Mar 9 2015 Back to list
UserAgent: Mozilla/5.0 (X11; Linux x86_64; rv:39.0) Gecko/20100101 Firefox/39.0

Steps to reproduce the problem:
1. Attempt to use multiple streams in a WebRTC RTCPeerConnection

What is the expected behavior?
Generated SDP should follow the "Unified Plan" spec, instead of the older "Plan B"

What went wrong?
Generates Plan B SDP (see below)

Did this work before? No 

Chrome version: Version 42.0.2311.22 dev (64-bit)  (Really all versions)  Channel: canary
OS Version: Fedora 21 (really all OSes)
Flash Version: 

Earlier this year, Firefox landed a new implementation of our JSEP code, which deals with a number of long-missing features. Most notably, we now support the Unified Plan approach for multiple media streams in a connection. Since landing, we've been seeing an increasing number of developers running into issues with Chrome's "Plan B"-style handling of SDP.

By way of example; we just had an implementer come to us with an issue in which he adds one audio stream and two media streams to a PC in Chrome (M40), and gets only two m-sections (rather than the three that Unified would call for). Since the video section contains multiple different MSIDs, our code chokes on the resulting body (see https://dxr.mozilla.org/mozilla-central/source/media/webrtc/signaling/src/jsep/JsepSessionImpl.cpp#564).

The implementer reports that Canary has the same behavior.
 
Here's the SDP generated by Chrome:
chrome.sdp
3.1 KB Download
Comment 2 by sv...@appear.in, Mar 9 2015
We would like to implement interoperable screen sharing between Chrome and Firefox in appear.in without having to switch to a different PeerConnection for the shared screen. Hence the need for interoperable support for multiple streams.
I realize it's suboptimal, but a possible temporary solution would be to munge the SDP in JS, and give different SDP to Firefox and Chrome in the calls to SetLocalDescription and SetRemoteDescription.  It might even be possible to do so via polyfill.
There's a gotcha there. If you're munging statelessly, and a plan-b stream is removed, you'll end up removing an m-section.
Cc: tommi@chromium.org satyavat...@chromium.org
Labels: TE-NeedsFurtherTriage M-42 Cr-Blink-WebRTC
Labels: -Type-Bug Type-Feature
Owner: juberti@chromium.org
Justin, please comment on time frame.
Labels: WebrtcTriaged
Status: Assigned
Summary: Need to implement WebRTC "Unified Plan" for multistream (was: Need to implement WebRTC "Unified Plan" for multistream/BUNDLE)
yes, planB->planA munging will require some state.

We have some legacy stuff to clean up before this can work, so it will be a few months.
I'm concerned about the complexity of munging Unified<->PlanB, since they have very different behaviors when adding and removing streams. In fact, this was the core objection to using Plan B in the spec: it doesn't play well with existing devices (which largely exhibit behavior in line with Plan A and, to a slightly more limited degree, Unified Plan), and mapping between the two schemes is intractably difficult. I suspect that any SDP munging that one might design here would be very brittle.

If I were to implement this as a polyfill, I would mock up a single API surface that looks like a PC, but which internally spins up a new PC for each media stream. The remote/local subscriptions would be collections of SDP bodies rather than a single SDP, but the behavior would otherwise match PC behavior as speced. But that seems like a lot of extra work when applications can simply spin up their own PCs on the fly.

I could be missing something, though: no one has ever adequately explained the sentiment expressed by statements like "without having to switch to a different PeerConnection for the shared screen" (this isn't the first time I've heard that kind of objection). Other than aesthetic purity, what is the real cost of using exactly that solution?

P.S. Justin, Surely you mean "Unified" rather than "Plan A" in comment #8, right? I mean, if you want to push for Plan A rather than Unified Plan, I'm ready to have that conversation. :)
Here's our take on a polyfill https://www.npmjs.com/package/sdp-interop.
what is the status of the bug?
What are the plans to implement the unified plan? Any updates regarding time frame to expect these changes?
Comment 13 by sojh...@gmail.com, Aug 9 2015
Should we expect this issue to be resolved in next 1 or 2 months?
Comment 14 by juberti@webrtc.org, Aug 13 2015
We will reevaluate this issue at the start of Q4. For now, the recommendation is to do JS munging as indicated in #9.
Comment 15 by hta@chromium.org, Aug 25 2015
Cc: hta@chromium.org
Labels: Hotlist-Recharge
This issue likely requires triage.  The current issue owner maybe inactive (i.e. hasn't fixed an issue in the last 30 days).  Thanks for helping out!

-Anthony
Cc: juberti@chromium.org deadbeef@chromium.org
Labels: -TE-NeedsFurtherTriage -M-42 -Hotlist-Recharge M-49
Owner: pthatcher@chromium.org
Will plan B be deprecated once Chrome supports unified plan?
How do you plan to make the transition?
Will Chrome support both plans at some point?
Good question. I think we will need to support receiving plan B offers/answers for a short time while we have clients with/without unified in the field. But eventually, expect plan B to be deprecated.
Yes, there will definitely have to be a period where we support both.  And yes, Plan B will go away shortly after that. 
Hi all,

Do you guys have a version of Chrome when this unified plan would be implemented and released?
We will reevaluate this issue in Q1 2016.
Comment 23 by ibc@aliax.net, Feb 28 2016
Hi, we are in Q1 2016. Is there any news regarding an ETA for unified plan in Chrome?
We've begun work on it, and hope to have something available in the first half of 2016.  
Regarding the transition plans, Chrome being able to accept both plan-B and Unified plan SDP makes sense, however I'm concerned about the opposite direction: Chrome generates a unified plan SDP but the remote side only accepts plan-B (old Chrome or libwebrtc peer)
 
Currently the sdp-interop package can be used to help unified plan -> plan B interop, but it only works when the Offerer is plan-B. (https://github.com/jitsi/sdp-interop/issues/4)
 
This might be problematic when we don't know ahead of time what the other peer might be, or if the app signaling is incapable of switching who the offerer/answerer is.
 
I see 2 approaches possible:
- Constraint or other mechanism to force Chrome to produce Plan B SDP Offers during the transition period. I'm assuming here that Chrome would automatically produce Plan B Answers if it sees a Plan B Offer.
- Have shim support unified plan -> Plan B, even for offerer scenarios, which might be the same as creating a generic plan b <-> unified plan shim? I've heard concerns this might not be feasible.
 
Both of those require app code changes, I don’t see any magic way around that. Maybe Unified plan could be opt-in at first, the same way the SDES->DTLS transition was done?
 
Out of curiosity, how is Hangouts planning on dealing with the transition? My understanding is that Hangouts is always the offerer and would be subject to this issue unless the infrastructure side is capable of handling unified plan.

You have exactly the same thinking as we do. Currently the plan is to do this in three stages:

-Stage 1-
Plan B is the default, and Unified Plan is opt-in through an RTCConfiguration parameter.

-Stage 2-
Unified Plan is the default, and Plan B is opt-in through an RTCConfiguration parameter.

-Stage 3-
Option to use Plan B is completely removed.


Does this approach sound reasonable to you?
do you have an ETA for each stage?
Comment 28 by hta@webrtc.org, Mar 10 2016
Of course any app that works with Firefox as the initiator will also work with an Unified-Plan-generating Chrome, since Firefox already generates Unified Plan.

In response to #27: We don't have a definite ETA for each stage, but I think the most important step will be giving developers some way to generate (and accept) Unified Plan offers. As pthatcher mentioned, we hope to have something available in the first half of 2016.
Comment 30 by pbos@webrtc.org, Mar 22 2016
Blocking: webrtc:5187
Hello, we are in the second half of 2016. Is there any news about unified plans? Thanks!
Comment 32 by yijin...@gmail.com, Jul 29 2016
We are also waiting for this feature. It's sad that Chrome team doesn't seem to give it high priority.
This work is underway and we are aiming to complete it by EOY.
Hey Justin,

  Is end of year still a reasonable completion date?
  Would that mean it would be shipping in the first chromium release after January?

  Thanks so much!

-- Samson

Comment 35 by ibc@aliax.net, Oct 28 2016
SFU developers are waiting for this more than Christmas.
We are waiting for this feature too..
Comment 37 by arung...@cisco.com, Nov 17 2016
Cisco developer team is also waiting for this feature to make conference feature much better on chrome browser.
Comment 38 by oprof...@cisco.com, Nov 19 2016
Huge demand from enterprise market, please highly prioritise it 
Thanks 
Need it for my sons chromebook. May have to get a cheap wintel machine instead. 
Comment 40 by xahem...@cisco.com, Nov 20 2016
Highly expected
VisionsConnected development team wants this feature for Cisco integration.
Comment 42 by jship...@cisco.com, Nov 28 2016
Cisco enterprise customers want to use chromium.   Without this we must point them to other OS.
Comment 43 by ibc@aliax.net, Nov 28 2016
> This work is underway and we are aiming to complete it by EOY.

Justin, may you please clarify which year you mean?
> SFU developers are waiting for this more than Christmas.

No they are not ;). At least not all of them.
> > SFU developers are waiting for this more than Christmas.

>No they are not ;). At least not all of them.

Well, Christmas is arround the corner, so maybe that is the case.
Comment 46 by ibc@aliax.net, Dec 19 2016
> No they are not ;). At least not all of them.

Well, they should IMHO. Unified-Plan is the multi stream mechanism standardized for WebRTC. Plan-B is not. This is not about personal preferences but about conforming to approved standards.
Comment 47 by ish...@zingaya.com, Dec 21 2016
10 days and "EOY"
Comment 48 by phistuck@gmail.com, Dec 21 2016
People, please, stop. Star the issue and wait for updates. "aiming for" is not a commitment. We do not need a countdown. They will get to it when they do.
OK, it's clear that some Cisco employees and partners would like Unified Plan :).  

Can you (or one of you) be more specific about what part of unified plan SDP you really need, and why?   

- Do you just need support for more than one m= section of the same media type?  Or is it more than that (like support for MID header extension, RIDs, RtpTranceivers, etc)?
- Is there something that you are unable to do with the current API, or are you simply avoiding the work/complexity of writing Chrome-specific code and converting between SDP formats?
- Could you accomplish the same think you are seeking with the ORTC API (possibly with a JS library on top), the same way you would need to support Edge currently?


@pthatcher Not from CISCO, but we're keen to get Unified Plan working also. For us the use case is that we have several media streams running in parallel between two peers, e.g. the frontal face camera plus a document camera, or a screenshare. The key issue we have is interoperability with Firefox on this (and later other browsers of course).
Comment 51 by ibc@aliax.net, Dec 23 2016
Hi Peter:

> Do you just need support for more than one m= section of the same media type?  Or is it more than that (like support for MID header extension, RIDs, RtpTranceivers, etc)?

Everything is welcome, but I mainly mean multiple m= sections. Dealing with multiple streams (tracks) in a SFU means that a SDP with multiple m= sections must be produced for Firefox (Unified Plan) and a SDP with just two m= sections (audio and video) and multiple a=ssrc lines must be produced for Chrome. I do both, and I know there are some JS libs doing such a task. But Chrome's Plan-B imposes some undesirable constraints and limitations (cannot indicate media direction per track, etc).

> - Is there something that you are unable to do with the current API, or are you simply avoiding the work/complexity of writing Chrome-specific code and converting between SDP formats?

This is not just about browser side API.

> - Could you accomplish the same think you are seeking with the ORTC API (possibly with a JS library on top), the same way you would need to support Edge currently?

Not all of us want to spend the required time to support current Edge. That's another story.

Anyhow, and please don't take me wrong, you cannot just tell us "hey, we don't implement the standard but you should not need it anyway". We just want a proper response to our question: When will Google implement the WebRTC standardized SDP? You may prefer not to reply to this question, but please, don't try to make our question irrelevant. Having to maintaining a specific shim for Chrome (in client and/or server side) is relevant for, at least, 145 developers.
Comment 52 Deleted
In case anyone is wondering what comment 52 was: it was spam.  
In response to comment 50 and 51:

It's good to know what exactly your pain points are.  Thanks for the input.  It helps us know what to prioritize.  

We can't give a firm time at which it will be available.  We can't even give a firm time at which the standard will be complete.  Be we are working hard on getting the WebRTC, JSEP, and all of the dependent documents (BUNDLE, trickle ICE, etc) complete.  And we're also making progress on our implementation to make it more compliant with those documents.  It may not be going as fast as you like, but we're making progress and we'll get there in time.  


 
Comment 55 by ibc@aliax.net, Dec 28 2016
Thanks Peter.
may we see this in  this EOY?
Hello. we have a customer use case and project on hold until this is resolved and available. any roadmaps / timelines for availability.?
Comment 58 by ibc@aliax.net, Mar 5 2017
The lack of Unified-Plan and the lack of pc.getSenders() makes it hard to implement proper simulcast.

Yes, simulcast can be currently achieved by mangling the SDP (adding a=ssrc and a=ssrc-group:SIM) before calling setLocalDescription(), but it's 100% undocumented.

For example, how to manage two local video streams having both simulcast? We really need Chrome moving forward to the new specs.
Changed my mind, please DO NOT implement this.. lets's go for the RTP Media API instead as top prio
Comment 60 by ibc@aliax.net, Mar 8 2017
> please DO NOT implement this.. lets's go for the RTP Media API instead as top prio

You may prefer to open a new issue to express your personal feeling (or give +1 somewhere else) rather than nullify the desire of so many in this issue/feature.

There is never a good reason to not implement a standard, no matter how bad it is.
If it draws resources for implementing more important APIs, yes it is
Comment 62 by ibc@aliax.net, Mar 8 2017
Well, unvote here and vote somewhere else. And let the webrtc project managers decide.
Sorry moderator, but I can't unstar the issue and keep be notified about changes on this issue
#63 - you actually can, using the saved queries feature.
> #63 - you actually can, using the saved queries feature.

Thanxs! Very useful indeed
@ comment #59: When you say "RTP Media API", are you referring to this: https://www.w3.org/TR/webrtc/#rtp-media-api?

Because this API is very closely tied to the unified plan SDP model (at least, the "RtpTransceiver" part of it).
Comment 67 by huib@chromium.org, Apr 3 2017
Cc: huib@chromium.org
Labels: Hotlist-Interop
This was linked from https://twitter.com/wanderview/status/859422708181987328

Adding Hotlist-Interop since it sounds like it has something to do with Hangouts working best (only?) on Chrome. Is the current status accurate, assigned but not being worked on yet?
Labels: -M-49
It is being (and has been) actively worked on, but it is a large, multi-quarter project, and the exact release milestone is still unclear. We will provide another update by EOQ.
> We will provide another update by EOQ.

I may have missed it. Which was the previous update?
Loop detected :)
BTW, is there any commit, branch, blog post, tweet, or whatever describing the work done on Unified-Plan stuff? I just cannot find anything (sure I'm wrong).
so i wrote a very simple web-platform-test @ https://github.com/w3c/web-platform-tests/pull/5858

Interestingly, Chrome calls the setRemoteDescription success callback even though it does not understand the SDP. While I have seen this before (IIRC it errors in createAnswer) this seems like a particularly poor failure mode.
Thanks Justin, happy to see that.
NextAction: 2017-07-01
Owner: juberti@chromium.org
Adding a next action date and assigning to juberti@ based on comment #69. @Juberti to provide an update on status at the beginning of next quarter!
Owner: pthatcher@chromium.org
Re-assigning to Peter.
The NextAction date has arrived: 2017-07-01
Project Member Comment 79 by bugdroid1@chromium.org, Aug 15
The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/a77e6bbd30276bdc5b30f2cbc1e92ca181ae76f0

commit a77e6bbd30276bdc5b30f2cbc1e92ca181ae76f0
Author: zhihuang <zhihuang@webrtc.org>
Date: Tue Aug 15 01:17:48 2017

Adding support for Unified Plan offer/answer negotiation to the mediasession layer.

This layer takes in a simplified "options" struct and the current local description,
and generates a new offer/answer. Previously the options struct assumed there would
only be one media description per media type (audio/video), but it now supports
N number of audio/video descriptions.

The |add_legacy_stream| options is removed from the mediasession.cc/.h
in this CL.

The next step is to add the ability for PeerConnection/WebRtcSession to create
"options" to represent multiple RtpTransceivers, and apply the Unified Plan
descriptions correctly. Right now, only Plan B descriptions will be
generated in unit tests.

BUG=chromium:465349

Review-Url: https://codereview.webrtc.org/2991693002
Cr-Commit-Position: refs/heads/master@{#19343}

[modify] https://crrev.com/a77e6bbd30276bdc5b30f2cbc1e92ca181ae76f0/webrtc/pc/mediasession.cc
[modify] https://crrev.com/a77e6bbd30276bdc5b30f2cbc1e92ca181ae76f0/webrtc/pc/mediasession.h
[modify] https://crrev.com/a77e6bbd30276bdc5b30f2cbc1e92ca181ae76f0/webrtc/pc/mediasession_unittest.cc
[modify] https://crrev.com/a77e6bbd30276bdc5b30f2cbc1e92ca181ae76f0/webrtc/pc/peerconnection.cc
[modify] https://crrev.com/a77e6bbd30276bdc5b30f2cbc1e92ca181ae76f0/webrtc/pc/peerconnection.h
[modify] https://crrev.com/a77e6bbd30276bdc5b30f2cbc1e92ca181ae76f0/webrtc/pc/peerconnectioninterface_unittest.cc
[modify] https://crrev.com/a77e6bbd30276bdc5b30f2cbc1e92ca181ae76f0/webrtc/pc/webrtcsession_unittest.cc
[modify] https://crrev.com/a77e6bbd30276bdc5b30f2cbc1e92ca181ae76f0/webrtc/pc/webrtcsessiondescriptionfactory.cc

Project Member Comment 80 by bugdroid1@chromium.org, Aug 17
The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/3c747665d06f6db9e0e570fbb43f122f228419d7

commit 3c747665d06f6db9e0e570fbb43f122f228419d7
Author: olka <olka@webrtc.org>
Date: Thu Aug 17 13:50:32 2017

Revert of Adding support for Unified Plan offer/answer negotiation. (patchset #9 id:500001 of https://codereview.webrtc.org/2991693002/ )

Reason for revert:
BUG=webrtc:8108: breaks Clang build.

Original issue's description:
> Adding support for Unified Plan offer/answer negotiation to the mediasession layer.
>
> This layer takes in a simplified "options" struct and the current local description,
> and generates a new offer/answer. Previously the options struct assumed there would
> only be one media description per media type (audio/video), but it now supports
> N number of audio/video descriptions.
>
> The |add_legacy_stream| options is removed from the mediasession.cc/.h
> in this CL.
>
> The next step is to add the ability for PeerConnection/WebRtcSession to create
> "options" to represent multiple RtpTransceivers, and apply the Unified Plan
> descriptions correctly. Right now, only Plan B descriptions will be
> generated in unit tests.
>
> BUG=chromium:465349
>
> Review-Url: https://codereview.webrtc.org/2991693002
> Cr-Commit-Position: refs/heads/master@{#19343}
> Committed: https://chromium.googlesource.com/external/webrtc/+/a77e6bbd30276bdc5b30f2cbc1e92ca181ae76f0

TBR=deadbeef@webrtc.org,zhihuang@webrtc.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=chromium:465349

Review-Url: https://codereview.webrtc.org/3001083002
Cr-Commit-Position: refs/heads/master@{#19384}

[modify] https://crrev.com/3c747665d06f6db9e0e570fbb43f122f228419d7/webrtc/pc/mediasession.cc
[modify] https://crrev.com/3c747665d06f6db9e0e570fbb43f122f228419d7/webrtc/pc/mediasession.h
[modify] https://crrev.com/3c747665d06f6db9e0e570fbb43f122f228419d7/webrtc/pc/mediasession_unittest.cc
[modify] https://crrev.com/3c747665d06f6db9e0e570fbb43f122f228419d7/webrtc/pc/peerconnection.cc
[modify] https://crrev.com/3c747665d06f6db9e0e570fbb43f122f228419d7/webrtc/pc/peerconnection.h
[modify] https://crrev.com/3c747665d06f6db9e0e570fbb43f122f228419d7/webrtc/pc/peerconnectioninterface_unittest.cc
[modify] https://crrev.com/3c747665d06f6db9e0e570fbb43f122f228419d7/webrtc/pc/webrtcsession_unittest.cc
[modify] https://crrev.com/3c747665d06f6db9e0e570fbb43f122f228419d7/webrtc/pc/webrtcsessiondescriptionfactory.cc

Project Member Comment 81 by bugdroid1@chromium.org, Aug 17
The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/1c378ed83b5b32a00368bbfc3ce2ee7687691abe

commit 1c378ed83b5b32a00368bbfc3ce2ee7687691abe
Author: zhihuang <zhihuang@webrtc.org>
Date: Thu Aug 17 21:10:50 2017

Relanding: Adding support for Unified Plan offer/answer negotiation to the mediasession layer.

This layer takes in a simplified "options" struct and the current local description,
and generates a new offer/answer. Previously the options struct assumed there would
only be one media description per media type (audio/video), but it now supports
N number of audio/video descriptions.

The |add_legacy_stream| options is removed from the mediasession.cc/.h
in this CL.

The next step is to add the ability for PeerConnection/WebRtcSession to create
"options" to represent multiple RtpTransceivers, and apply the Unified Plan
descriptions correctly. Right now, only Plan B descriptions will be
generated in unit tests.

BUG=chromium:465349

Review-Url: https://codereview.webrtc.org/2991693002
Cr-Original-Commit-Position: refs/heads/master@{#19343}
Committed: https://chromium.googlesource.com/external/webrtc/+/a77e6bbd30276bdc5b30f2cbc1e92ca181ae76f0
Review-Url: https://codereview.webrtc.org/2991693002
Cr-Commit-Position: refs/heads/master@{#19394}

[modify] https://crrev.com/1c378ed83b5b32a00368bbfc3ce2ee7687691abe/webrtc/pc/mediasession.cc
[modify] https://crrev.com/1c378ed83b5b32a00368bbfc3ce2ee7687691abe/webrtc/pc/mediasession.h
[modify] https://crrev.com/1c378ed83b5b32a00368bbfc3ce2ee7687691abe/webrtc/pc/mediasession_unittest.cc
[modify] https://crrev.com/1c378ed83b5b32a00368bbfc3ce2ee7687691abe/webrtc/pc/peerconnection.cc
[modify] https://crrev.com/1c378ed83b5b32a00368bbfc3ce2ee7687691abe/webrtc/pc/peerconnection.h
[modify] https://crrev.com/1c378ed83b5b32a00368bbfc3ce2ee7687691abe/webrtc/pc/peerconnectioninterface_unittest.cc
[modify] https://crrev.com/1c378ed83b5b32a00368bbfc3ce2ee7687691abe/webrtc/pc/webrtcsession_unittest.cc
[modify] https://crrev.com/1c378ed83b5b32a00368bbfc3ce2ee7687691abe/webrtc/pc/webrtcsessiondescriptionfactory.cc

Blockedon: webrtc:8183
Hi bugdroid1@chromium.org Can we get more info/instructions to run on the first layer where it converts local description to unified plan. ?
Not sure I fully understand the question. But this first CL is just adding the support to MediaSessionDescriptionFactory to generate multiple m= sections of the same media type, given a MediaDescriptionOptions structure. It's still not possible to do this through PeerConnection yet.
Thanks you . I thought we have an adapter to do that on the client side for now . 
we are currently facing an issue where our server handles multiple m lines for multi stream.  Is there any workaround to convert the sdp generated  from chrome for multistream to multiple m line . 
I may not be the best person to ask, but a quick search revealed this:
https://github.com/jitsi/sdp-interop

For other options, I'd recommend asking on discuss-webrtc (or searching through the archive).
Thanks will will a try .
Comment 88 by steveanton@webrtc.org, Nov 15 (3 days ago)
Blocking: webrtc:8530
Sign in to add a comment