Project: chromium Issues People Development process History Sign in
New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Issue 465349 Need to implement WebRTC "Unified Plan" for multistream
Starred by 155 users Reported by randell....@gmail.com, Mar 9 2015 Back to list
Status: Assigned
Owner:
Last visit 16 days ago
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 2
Type: Feature

Blocking:
issue webrtc:5187



Sign in to add a comment
UserAgent: Mozilla/5.0 (X11; Linux x86_64; rv:39.0) Gecko/20100101 Firefox/39.0

Steps to reproduce the problem:
1. Attempt to use multiple streams in a WebRTC RTCPeerConnection

What is the expected behavior?
Generated SDP should follow the "Unified Plan" spec, instead of the older "Plan B"

What went wrong?
Generates Plan B SDP (see below)

Did this work before? No 

Chrome version: Version 42.0.2311.22 dev (64-bit)  (Really all versions)  Channel: canary
OS Version: Fedora 21 (really all OSes)
Flash Version: 

Earlier this year, Firefox landed a new implementation of our JSEP code, which deals with a number of long-missing features. Most notably, we now support the Unified Plan approach for multiple media streams in a connection. Since landing, we've been seeing an increasing number of developers running into issues with Chrome's "Plan B"-style handling of SDP.

By way of example; we just had an implementer come to us with an issue in which he adds one audio stream and two media streams to a PC in Chrome (M40), and gets only two m-sections (rather than the three that Unified would call for). Since the video section contains multiple different MSIDs, our code chokes on the resulting body (see https://dxr.mozilla.org/mozilla-central/source/media/webrtc/signaling/src/jsep/JsepSessionImpl.cpp#564).

The implementer reports that Canary has the same behavior.
 
Here's the SDP generated by Chrome:
chrome.sdp
3.1 KB Download
Comment 2 by sv...@appear.in, Mar 9 2015
We would like to implement interoperable screen sharing between Chrome and Firefox in appear.in without having to switch to a different PeerConnection for the shared screen. Hence the need for interoperable support for multiple streams.
I realize it's suboptimal, but a possible temporary solution would be to munge the SDP in JS, and give different SDP to Firefox and Chrome in the calls to SetLocalDescription and SetRemoteDescription.  It might even be possible to do so via polyfill.
There's a gotcha there. If you're munging statelessly, and a plan-b stream is removed, you'll end up removing an m-section.
Cc: tommi@chromium.org satyavat...@chromium.org
Labels: TE-NeedsFurtherTriage M-42 Cr-Blink-WebRTC
Labels: -Type-Bug Type-Feature
Owner: juberti@chromium.org
Justin, please comment on time frame.
Labels: WebrtcTriaged
Status: Assigned
Summary: Need to implement WebRTC "Unified Plan" for multistream (was: Need to implement WebRTC "Unified Plan" for multistream/BUNDLE)
yes, planB->planA munging will require some state.

We have some legacy stuff to clean up before this can work, so it will be a few months.
I'm concerned about the complexity of munging Unified<->PlanB, since they have very different behaviors when adding and removing streams. In fact, this was the core objection to using Plan B in the spec: it doesn't play well with existing devices (which largely exhibit behavior in line with Plan A and, to a slightly more limited degree, Unified Plan), and mapping between the two schemes is intractably difficult. I suspect that any SDP munging that one might design here would be very brittle.

If I were to implement this as a polyfill, I would mock up a single API surface that looks like a PC, but which internally spins up a new PC for each media stream. The remote/local subscriptions would be collections of SDP bodies rather than a single SDP, but the behavior would otherwise match PC behavior as speced. But that seems like a lot of extra work when applications can simply spin up their own PCs on the fly.

I could be missing something, though: no one has ever adequately explained the sentiment expressed by statements like "without having to switch to a different PeerConnection for the shared screen" (this isn't the first time I've heard that kind of objection). Other than aesthetic purity, what is the real cost of using exactly that solution?

P.S. Justin, Surely you mean "Unified" rather than "Plan A" in comment #8, right? I mean, if you want to push for Plan A rather than Unified Plan, I'm ready to have that conversation. :)
Here's our take on a polyfill https://www.npmjs.com/package/sdp-interop.
what is the status of the bug?
What are the plans to implement the unified plan? Any updates regarding time frame to expect these changes?
Comment 13 by sojh...@gmail.com, Aug 9 2015
Should we expect this issue to be resolved in next 1 or 2 months?
Comment 14 by juberti@webrtc.org, Aug 13 2015
We will reevaluate this issue at the start of Q4. For now, the recommendation is to do JS munging as indicated in #9.
Comment 15 by hta@chromium.org, Aug 25 2015
Cc: hta@chromium.org
Labels: Hotlist-Recharge
This issue likely requires triage.  The current issue owner maybe inactive (i.e. hasn't fixed an issue in the last 30 days).  Thanks for helping out!

-Anthony
Cc: juberti@chromium.org deadbeef@chromium.org
Labels: -TE-NeedsFurtherTriage -M-42 -Hotlist-Recharge M-49
Owner: pthatcher@chromium.org
Will plan B be deprecated once Chrome supports unified plan?
How do you plan to make the transition?
Will Chrome support both plans at some point?
Good question. I think we will need to support receiving plan B offers/answers for a short time while we have clients with/without unified in the field. But eventually, expect plan B to be deprecated.
Yes, there will definitely have to be a period where we support both.  And yes, Plan B will go away shortly after that. 
Hi all,

Do you guys have a version of Chrome when this unified plan would be implemented and released?
We will reevaluate this issue in Q1 2016.
Comment 23 by i...@aliax.net, Feb 28 2016
Hi, we are in Q1 2016. Is there any news regarding an ETA for unified plan in Chrome?
We've begun work on it, and hope to have something available in the first half of 2016.  
Regarding the transition plans, Chrome being able to accept both plan-B and Unified plan SDP makes sense, however I'm concerned about the opposite direction: Chrome generates a unified plan SDP but the remote side only accepts plan-B (old Chrome or libwebrtc peer)
 
Currently the sdp-interop package can be used to help unified plan -> plan B interop, but it only works when the Offerer is plan-B. (https://github.com/jitsi/sdp-interop/issues/4)
 
This might be problematic when we don't know ahead of time what the other peer might be, or if the app signaling is incapable of switching who the offerer/answerer is.
 
I see 2 approaches possible:
- Constraint or other mechanism to force Chrome to produce Plan B SDP Offers during the transition period. I'm assuming here that Chrome would automatically produce Plan B Answers if it sees a Plan B Offer.
- Have shim support unified plan -> Plan B, even for offerer scenarios, which might be the same as creating a generic plan b <-> unified plan shim? I've heard concerns this might not be feasible.
 
Both of those require app code changes, I don’t see any magic way around that. Maybe Unified plan could be opt-in at first, the same way the SDES->DTLS transition was done?
 
Out of curiosity, how is Hangouts planning on dealing with the transition? My understanding is that Hangouts is always the offerer and would be subject to this issue unless the infrastructure side is capable of handling unified plan.

You have exactly the same thinking as we do. Currently the plan is to do this in three stages:

-Stage 1-
Plan B is the default, and Unified Plan is opt-in through an RTCConfiguration parameter.

-Stage 2-
Unified Plan is the default, and Plan B is opt-in through an RTCConfiguration parameter.

-Stage 3-
Option to use Plan B is completely removed.


Does this approach sound reasonable to you?
do you have an ETA for each stage?
Comment 28 by hta@webrtc.org, Mar 10 2016
Of course any app that works with Firefox as the initiator will also work with an Unified-Plan-generating Chrome, since Firefox already generates Unified Plan.

In response to #27: We don't have a definite ETA for each stage, but I think the most important step will be giving developers some way to generate (and accept) Unified Plan offers. As pthatcher mentioned, we hope to have something available in the first half of 2016.
Comment 30 by pbos@webrtc.org, Mar 22 2016
Blocking: webrtc:5187
Hello, we are in the second half of 2016. Is there any news about unified plans? Thanks!
Comment 32 by yijin...@gmail.com, Jul 29 2016
We are also waiting for this feature. It's sad that Chrome team doesn't seem to give it high priority.
This work is underway and we are aiming to complete it by EOY.
Hey Justin,

  Is end of year still a reasonable completion date?
  Would that mean it would be shipping in the first chromium release after January?

  Thanks so much!

-- Samson

SFU developers are waiting for this more than Christmas.
We are waiting for this feature too..
Cisco developer team is also waiting for this feature to make conference feature much better on chrome browser.
Huge demand from enterprise market, please highly prioritise it 
Thanks 
Need it for my sons chromebook. May have to get a cheap wintel machine instead. 
Highly expected
VisionsConnected development team wants this feature for Cisco integration.
Cisco enterprise customers want to use chromium.   Without this we must point them to other OS.
> This work is underway and we are aiming to complete it by EOY.

Justin, may you please clarify which year you mean?
> SFU developers are waiting for this more than Christmas.

No they are not ;). At least not all of them.
> > SFU developers are waiting for this more than Christmas.

>No they are not ;). At least not all of them.

Well, Christmas is arround the corner, so maybe that is the case.
> No they are not ;). At least not all of them.

Well, they should IMHO. Unified-Plan is the multi stream mechanism standardized for WebRTC. Plan-B is not. This is not about personal preferences but about conforming to approved standards.
10 days and "EOY"
People, please, stop. Star the issue and wait for updates. "aiming for" is not a commitment. We do not need a countdown. They will get to it when they do.
OK, it's clear that some Cisco employees and partners would like Unified Plan :).  

Can you (or one of you) be more specific about what part of unified plan SDP you really need, and why?   

- Do you just need support for more than one m= section of the same media type?  Or is it more than that (like support for MID header extension, RIDs, RtpTranceivers, etc)?
- Is there something that you are unable to do with the current API, or are you simply avoiding the work/complexity of writing Chrome-specific code and converting between SDP formats?
- Could you accomplish the same think you are seeking with the ORTC API (possibly with a JS library on top), the same way you would need to support Edge currently?


@pthatcher Not from CISCO, but we're keen to get Unified Plan working also. For us the use case is that we have several media streams running in parallel between two peers, e.g. the frontal face camera plus a document camera, or a screenshare. The key issue we have is interoperability with Firefox on this (and later other browsers of course).
Hi Peter:

> Do you just need support for more than one m= section of the same media type?  Or is it more than that (like support for MID header extension, RIDs, RtpTranceivers, etc)?

Everything is welcome, but I mainly mean multiple m= sections. Dealing with multiple streams (tracks) in a SFU means that a SDP with multiple m= sections must be produced for Firefox (Unified Plan) and a SDP with just two m= sections (audio and video) and multiple a=ssrc lines must be produced for Chrome. I do both, and I know there are some JS libs doing such a task. But Chrome's Plan-B imposes some undesirable constraints and limitations (cannot indicate media direction per track, etc).

> - Is there something that you are unable to do with the current API, or are you simply avoiding the work/complexity of writing Chrome-specific code and converting between SDP formats?

This is not just about browser side API.

> - Could you accomplish the same think you are seeking with the ORTC API (possibly with a JS library on top), the same way you would need to support Edge currently?

Not all of us want to spend the required time to support current Edge. That's another story.

Anyhow, and please don't take me wrong, you cannot just tell us "hey, we don't implement the standard but you should not need it anyway". We just want a proper response to our question: When will Google implement the WebRTC standardized SDP? You may prefer not to reply to this question, but please, don't try to make our question irrelevant. Having to maintaining a specific shim for Chrome (in client and/or server side) is relevant for, at least, 145 developers.
Comment 52 Deleted
In case anyone is wondering what comment 52 was: it was spam.  
In response to comment 50 and 51:

It's good to know what exactly your pain points are.  Thanks for the input.  It helps us know what to prioritize.  

We can't give a firm time at which it will be available.  We can't even give a firm time at which the standard will be complete.  Be we are working hard on getting the WebRTC, JSEP, and all of the dependent documents (BUNDLE, trickle ICE, etc) complete.  And we're also making progress on our implementation to make it more compliant with those documents.  It may not be going as fast as you like, but we're making progress and we'll get there in time.  


 
Thanks Peter.
may we see this in  this EOY?
Hello. we have a customer use case and project on hold until this is resolved and available. any roadmaps / timelines for availability.?
The lack of Unified-Plan and the lack of pc.getSenders() makes it hard to implement proper simulcast.

Yes, simulcast can be currently achieved by mangling the SDP (adding a=ssrc and a=ssrc-group:SIM) before calling setLocalDescription(), but it's 100% undocumented.

For example, how to manage two local video streams having both simulcast? We really need Chrome moving forward to the new specs.
Changed my mind, please DO NOT implement this.. lets's go for the RTP Media API instead as top prio
> please DO NOT implement this.. lets's go for the RTP Media API instead as top prio

You may prefer to open a new issue to express your personal feeling (or give +1 somewhere else) rather than nullify the desire of so many in this issue/feature.

There is never a good reason to not implement a standard, no matter how bad it is.
If it draws resources for implementing more important APIs, yes it is
Well, unvote here and vote somewhere else. And let the webrtc project managers decide.
Sorry moderator, but I can't unstar the issue and keep be notified about changes on this issue
#63 - you actually can, using the saved queries feature.
> #63 - you actually can, using the saved queries feature.

Thanxs! Very useful indeed
@ comment #59: When you say "RTP Media API", are you referring to this: https://www.w3.org/TR/webrtc/#rtp-media-api?

Because this API is very closely tied to the unified plan SDP model (at least, the "RtpTransceiver" part of it).
Sign in to add a comment