New issue
Advanced search Search tips

Issue 21318 link

Starred by 118 users

Comments by non-members will not trigger notification emails to users who starred this issue.

Issue metadata

Status: WontFix
Owner: ----
Closed: Nov 2015
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 3
Type: Feature

Blocking:
issue 28287
issue 158955


Participants' hotlists:
Chromium-Packagers


Sign in to add a comment

Query FFmpeg libraries for codec support

Project Member Reported by scherkus@chromium.org, Sep 8 2009

Issue description

Instead of hard coding the values for canPlayType(), it would be great if we 
could dynamically query FFmpeg for container/codec support.

This would also fix compatibility issues on systems where regular Chromium 
builds are distributed with non-free builds of FFmpeg.
 
Showing comments 2 - 101 of 101 Older
 Issue 18809  has been merged into this issue.
Labels: -Size-Medium
Labels: -Pri-2 Pri-1
Bumping to Pri-1 cause it gets in the way of different linux distributions using their 
own FFMpeg versions.
The only official way to do this is with avcodec_find_decoder_by_name() and av_find_input_format(). They only need to be fed static strings not, data from the internet. If you do decide to go with a symbol matching approach, it should be noted that only symbols 
prefixed with "av" are considered public API. 

If you absolutely can't do it that way, I would recommend symbols like aac_decoder and mov_demuxer but this may break in future 
FFmpeg revisions. Probing symbols like aac_decoder and mov_demuxer might be ok for practical purposes but somebody might come along 
and rename mov_demuxer to isom_demuxer and that wouldn't be considered a break. 
For reference, here is a list of mp4 codecs
http://mp4ra.org/codecs.html

Looking at avformat.dll with depends.exe on windows, I found the following 3 symbols 
that might be used to enable our mimetypes - mov_demuxer, mp3_demuxer, ogg_demuxer.
These could be probed with GetProcAddress() on windows.  I'm sure Linux has a similar 
function.
Here is a proof of concept that works on Linux, is untested on Mac, and not 
implemented for Windows. 
media-probe.diff
4.5 KB View Download
Thanks for the proof of concept alex!

I was also considering calling av_codec_next(NULL) to get the first AVCodec* then
walking the list inspecting the CodecId enums.  Similar approach with
av_iformat_next(NULL).
Labels: -Mstone-4 Mstone-5
As long as only a handful of deocders/demuxers are being supported, probing with avcodec_find_decoder_by_name is probably better than av_codec_next. (Though av_codec_next would 
still be considered an officially supported way of doing this). 

Comment 11 by oritm@chromium.org, Dec 17 2009

Labels: -Area-BrowserBackend Area-Internals
Replacing labels:
   Area-BrowserBackend by Area-Internals

Labels: Internals-Video
Labels: -Video

Comment 14 by karen@chromium.org, Jan 11 2010

Labels: -HTML5 Mstone-X

Comment 15 by f...@sofaraway.org, Jan 21 2010

This prevents the HTML5 TestTube feature to work with Chromium, even when the right 
codecs are installed.
Yep.. they use canPlayType() which requires a change to the hardcoded MIME type list :(
Fta, I'm not sure how willing you are to deviate from upstream but my patch from above works on ubuntu 
last I checked. 
Alex, thanks for the patch.  I did similar code, but for Windows, and to address 
chromium builds that don't include ffmpeg at all.  There were concerns with security 
and startup time (100ms).
Maybe we should try submitting it for review.
Chrome4/5 use libffmpegsumo.so for linux, chromeos, and mac.  But the old names are 
used for windows and still work for linux/mac.
I can confirm Alex's proof of concept patch allows me to watch h.264 and wmv (chose 
random format from http://samples.mplayerhq.hu/) videos on my gentoo amd64. The patch 
had to be modified (read: new format added upstream). I'm not familiar with ffmpeg 
library nor chromium source code so I cannot attest to it doing it The Right Way(tm) 
or not since I didn't actually read it... at all...

Attached is the (very minorly) modified version of Alex's patch.

That said, there were videos I could not play for whatever reason. The videos would 
load correct size but not actually _play_. An example of this is atlantis405-test.mkv 
(from the above link). This is probably a different issue though. Just saying for 
completeness...
chromium-media-probe.patch
4.5 KB View Download

Comment 20 by tow...@gmail.com, Jan 22 2010

With this patch I get an empty string for canPlayType() for both H264 and Theora.

This page agrees: http://diveintohtml5.org/detect.html
@abhishek.mukher.g, @towolf

I think that code is executed in the renderer/sandboxed process, which would mean the 
patch won't work if the sandbox is enabled since dynamically loading libraries is a 
no-no :)

Comment 22 by koo...@gmail.com, Jan 24 2010

Is there any chance that the patch (or a better way to solve the problem) would be 
added to the source code by default ? Or does it creates legal issues ?

Comment 23 by darin@chromium.org, Jan 25 2010

Perhaps as a temporary solution, we could add a command line flag to Chrome that would 
change the behavior of canPlayType() to reveal support for H264 and Theora.
I could live with that... distros can set up their app launcher to use the flag

--chrome-media-mime-types or --extended-media-mime-types or something like that

could even go further with

--media-mime-types="video/ogg;audio/ogg;application/ogg;etc..."

Comment 25 by f...@sofaraway.org, Jan 25 2010

it won't work for ubuntu.
I have chromium-browser depending on either chromium-codecs-ffmpeg or chromium-codecs-
ffmpeg-nonfree (but not both, they conflict).

the launcher is in the main package.

My workaround so far was to drop the ifdef GOOGLE_CHROME_BUILD in 
src/net/base/mime_util.cc
Ugly but that was the only way to give -nonfree users access to the new Youtube & 
vimeo HTML5 videos.
@fta,

Could you have the codecs file provide flags in /etc/chromium-browser/codecs and have 
that file be sourced by /etc/chromium-browser/default?

Comment 27 by f...@sofaraway.org, Jan 25 2010

@alex, could work but is it worth it? i mean, this is supposed to be a temporary 
solution, right?
@fta, I think it is worth it.  Likelihood, that this will be resolved in the really 
short-term (eg., < a month) is pretty low.  Beyond 1 month, my ability to see into the 
future becomes unusably cloudy.
Labels: -Pri-1 -Mstone-X Pri-2 Mstone-5
I'm going to move this into Mstone-5 but lower the priority as we have some nasty bugs 
to fix first :\
Labels: -Pri-2
Labels: Pri-3
 Issue 32801  has been merged into this issue.
Would it help to use ffmpeg_branding instead of branding?  That way chromium could be built with fully enabled 
mime types?

Labels: -Mstone-5 Mstone-6
P3 == Punt
Status: Available
Labels: -Pri-3 -Mstone-6 Pri-2 Mstone-X
Labels: -Area-Internals -Internals-Video Area-WebKit Feature-Media
Labels: ffmpeg
As all of the proposed solutions/patches have been shot down, should we mark this as wont fix?
Does any other browser with dynamic formats (ie9/safari) do it better?

Yeah I just don't think we'll ever need this.

It's a nice to have -- but at the same time hard coding + the USE_PROPRIETARY_CODECS define works for nearly all cases for Chrome/Chromium
@scherkus "but at the same time hard coding + the USE_PROPRIETARY_CODECS define works for nearly all cases for Chrome/Chromium"

I beg to differ. I'm certain that there are plenty of people like myself that install a standard chromium, but with proprietary codecs. It's not even a little hard on Ubuntu, and I expect, many other Linux distributions.

This is very critical for many people.
@good.evil.genius, 

The Ubuntu situation always seemed kind of silly. MP3, AAC, and H.264 decoders are all in Ubuntu's system version of libavcodec in main. Since they don't seem to have a prohibition about shipping such formats in main, there is no need for them to support a configuration of chromium without these formats.
@alex.converse,
Whether or not you think the situation is silly is moot. It is the situation, and with so many people using Ubuntu, what's silly is Chromium developers trying to ignore it.

Also, I very much disagree with you about the situation being silly. Many people (not myself, but others) are very serious about keeping their system completely free of any potentially proprietary code. That's not a difficult thing to do with Ubuntu, but if Ubuntu devs only supported a version of Chromium that required MP3, AAC, and H.264, these people would be unable to use Chromium from the Ubuntu repos at all. That's not very fair to them.

Comment 43 by spo...@gmail.com, Aug 18 2010

FWIW, for distributions which will likely never be able to bundle ffmpeg (Fedora, Open Suse, for example), this feature will be pivotal for chromium to appropriately know what it can support, if it dlopen's ffmpeg libs that the user has placed there.
@spotrh,

If Fedora and openSuSe are still concerned about the parts of dsputil that are build unconditionally, you probably could find someone on the consultants list who could separate it out in a few weeks. Alternatively you could do it with your own personnel. Red Hat had plenty of money to fund Thusnelda. If neither of those are options you could probably just hack the offending chunks out in your builds in a matter of hours. 

Comment 45 by spo...@gmail.com, Aug 18 2010

Even if ffmpeg was separated out into "clean" and "functionality plugins", similar to gstreamer, it would just further reinforce the value of having chromium be able to query the loaded ffmpeg libraries to determine which codecs it is possible to support.
@good.evil.genius,
There is no proprietary code in FFmpeg in *any* configuration.  All of it is LGPL or GPL, except for a few files with more permissive (BSD-style) licences.

Comment 47 by spo...@gmail.com, Aug 18 2010

Proprietary is not the appropriate word there. Potentially (for percentages approaching 100) patented is the the appropriate wording. :)
@spotrh:

Consider attached patch. It makes Chromium runtime detects any and all of the codecs that Chrome supports from a libffmpegsumo.so. 
ffmpeg_mime.diff
5.4 KB View Download

Comment 49 by spo...@gmail.com, Aug 28 2010

Alex, thanks for the patch. Unfortunately, it doesn't seem to do anything. With that patch applied, and use_system_ffmpeg=1 && proprietary_codecs=1, html5test.com reports the same results whether or not ffmpeg-libs are present on the system.

Which is to say, it says there is no support for <video> or <audio> or any of their codecs. In addition, the browser does not know how to handle any of the mimetypes. :(
Voting up for this issue, Chromium should be able to detect ffmpeg support on runtime, so casual users could just grab libffmpegsumo.so and be happy ever after.
If runtime detection in ffmpeg is not easy, another option could be to provide a file that chromium will read to detect ffmpeg codec support (and do something by default if the file is missing). The file should then be manually created by package maintainers reflecting actually compiled codecs in ffmpeg.

BTW, the Ubuntu bug is here:
https://bugs.launchpad.net/ubuntu/+source/chromium-browser/+bug/646930
@alex 2 things missing from this patch:
1. windows
2. wav and chromeos formats
@fbachard

PCM/WAV and the chromeos formats should be trivial to add. 

As far as windows goes yes windows support is missing, because of my new job I don't have the time I once had to work on open source projects nor do I have a windows development setup for chromium.

@spotrh

Sorry the patch only was for loading from FFmpegsumo. System FFmprg support could probably be hacked into the loader but I do think using Chromium with system FFmpeg is a risky maneuver. http://spectralhole.blogspot.com/2010/08/why-you-dont-want-to-build-you-chromium.html

Comment 54 by spo...@gmail.com, Nov 9 2010

Alex, we cannot include ffmpeg in Fedora. There are third party repositories which provide it those Fedora users who choose to install it, and for them, dlopen'ing ffmpeg is the only way that they can get HTML5 video support in Chromium.

Google's decision to aggressively fork ffmpeg makes that impossible.
@spotrh: What portions of Chromium's FFmpeg fork are deemed unacceptable by Fedora?

As configured by Chromium, all of the MPEG and reverse engineered codecs (with the exception of Theora) are turned off.

Comment 56 by spo...@gmail.com, Nov 10 2010

Alex, we've been advised that carrying even that source code, wholly disabled, is a risk. And since ffmpeg isn't modular, like gstreamer, it is an all or nothing proposition.

Comment 57 by Deleted ...@, Nov 10 2010

Then I assume you would have no problem including ffmpeg if the source was preprocessed with a 1-line perl script that applied the C preprocessor to HAVE_* #defines, thus removing all sections of the code that would be disabled by the appropriate ./configure line?  Then that source code would no longer be part of the version of ffmpeg you carry.

Comment 58 Deleted

Comment 59 by spo...@gmail.com, Nov 10 2010

No, because we still have to distribute that source code (before it was preprocessed), whether it is clearly disabled or not.

Now, if Google were to apply this preprocessing in the source included in Chromium, maybe, but I doubt they're likely to do this (then again, if you're already forking...)

Comment 60 Deleted

Comment 61 Deleted

Comment 62 Deleted

Comment 63 Deleted

Comment 64 by Deleted ...@, Nov 10 2010

>No, because we still have to distribute that source code (before it was preprocessed)

Er, what?  Let's go through this step by step.

1.  I download the source code from ffmpeg's site, or from Chromium.  I haven't distributed anything yet.

2.  I preprocess the source.  I haven't distributed anything yet.

3.  I give the source to Fedora.  I still haven't distributed any of the code Fedora doesn't like.

Where, exactly, do I have to distribute the code that the preprocessor removes?  Tell me which step.

Comment 65 by spo...@gmail.com, Nov 10 2010

In the general case, yes, it would be hypothetically possible to preprocess away 95% of ffmpeg's functionality and provide it in Fedora... but why? It would simply conflict with the fully functional ffmpeg available from third party repositories (and confuse users when they expect ffmpeg to support MPEG). The correct approach (IMHO) is for ffmpeg to adopt a modular strategy, and support codecs on a plugin basis. Then, it would be trivial to remove the plugin source that a distribution is unable to ship, while still providing the ffmpeg framework.

In the chromium specific case, the problem is that Chromium is aggressively forking ffmpeg, so anyone who attempted to make an out of tree, preprocessed copy available for external loading would be facing an uphill battle of forking a fork.

Comment 66 by Deleted ...@, Nov 10 2010

I love answering people with their own quotes!

>The correct approach (IMHO) is for ffmpeg to adopt a modular strategy, and support codecs on a plugin basis. Then, it would be trivial to remove the plugin source that a distribution is unable to ship, while still providing the ffmpeg framework.

This is a bad idea because...

>It would simply conflict with the fully functional ffmpeg available from third party repositories

Comment 67 by spo...@gmail.com, Nov 10 2010

Ah, I see what you're saying, darkshikari. I would think the only difficulty in that approach would be in auditing the post-processed ffmpeg code to ensure that all of the questionable bits are gone. (Also, I'm not entirely convinced that MPEG is actually disabled in the chromium ffmpeg fork, last time I looked, they were still enabling it for HTML5.)

Comment 68 by spo...@gmail.com, Nov 10 2010

Re Comment 66:

No, you misunderstand me. I am proposing that ffmpeg should adopt a gstreamer style framework, where all of the codecs are abstracted into independent components, and there is a centralized framework that each codec compiles against. The framework should be able to trivially avoid legal issues (gstreamer does), and the distributions can choose which codec sources to build as plugins. Chromium could then build against the framework and ask the framework for specific codec support. If the codec is present, it is loaded by the framework, if not, it returns that failure to the application.

Again, this is not new. This is how gstreamer works, and it is why we're trivially able to handle codec enablement via third-party repositories.

Comment 69 by Deleted ...@, Nov 10 2010

>No, you misunderstand me. I am proposing that ffmpeg should adopt a gstreamer style framework, where all of the codecs are abstracted into independent components, and there is a centralized framework that each codec compiles against.

>Chromium could then build against the framework and ask the framework for specific codec support.

We could have a function that returns a codec if ffmpeg has one available.  We could call it "avcodec_find_encoder".

Oh wait.

Comment 70 by spo...@gmail.com, Nov 10 2010

Re: Comment 69

Unfortunately, Google has managed to fork that function such that it no longer works with the standard (stable) ffmpeg.

Also, that function is not independent from the rest of ffmpeg. The bit of contention here is not "how do we figure out which codecs are available" but rather "why must the framework code always come intertwined with the souce for legally troublesome codecs" ?

Comment 71 by vlado...@gmail.com, Nov 10 2010

since in ffmpeg H264 and vp8 share a lot of codec code, how can a modular ffmpeg ship a vp8 plugin and at the same time not ship *any* h264 code at all? Or can fedora supply an exhaustive list of shared codec code that is "safe" to have?
  

Comment 72 by Deleted ...@, Nov 10 2010

>Also, that function is not independent from the rest of ffmpeg. The bit of contention here is not "how do we figure out which codecs are available" but rather "why must the framework code always come intertwined with the souce for legally troublesome codecs" ?

It doesn't seem intertwined to me; utils.c (the framework code) compiles just fine without, say, h264.c.

Comment 73 by spo...@gmail.com, Nov 10 2010

Re: Comment 71 (Man, it would be nice if this interface supported sane threading and replies)

vladoman, all I can say is that gstreamer manages to accomplish this. Arguably, much of the commonality between the vp8 and h264 codec is probably code that could be safely considered "framework", as it is likely not the code which does the actual encode/decode of the specific codec formats.

I can say that Fedora has no issue with shipping vp8 (thanks to Google's patent grant) and Theora.

Comment 74 by spo...@gmail.com, Nov 10 2010

Re: Comment 72

If that is indeed the case, then it seems that the amount of change required for ffmpeg to provide a "clean" tarball is minimal. One wonders why the codec support is not modularized out by default.

Or to be blunt: If upstream ffmpeg provided a source tarball that contained no codecs, just the framework that could load separately built modular codec objects (for ffmpeg), Fedora would almost certainly provide and include that framework code (and would be able to pick and include codecs which are determined to be legally acceptable).

But, we're digressing from this specific issue, which is that Google is forking ffmpeg within Chromium.

Comment 75 by Deleted ...@, Nov 10 2010

>I can say that Fedora has no issue with shipping vp8 (thanks to Google's patent grant)

The patent grant doesn't actually cover any patents of note (On2 had no useful patents), nor does it guarantee or even begin to suggest that the patents it covers are sufficient.  I think what you really mean is "thanks to our blind trust in Google".

Comment 76 by vlado...@gmail.com, Nov 10 2010

Re: 73

Of course they "manage" that, they compile the code once to decode h264 and the other time to decode vp8 and produce 2 different "plugins". The fact that vp8 and h264 share a lot of code remains unchanged. 
  
> If that is indeed the case, then it seems that the amount of change required for ffmpeg to provide a "clean" tarball is minimal. One wonders why the codec support is not modularized out by default.

Most people MPEG-LA and Google (based on the way FFmpeg is hooked into Chromium) included don't seem to care about source only distributions.

> Or to be blunt: If upstream ffmpeg provided a source tarball that contained no codecs, just the framework that could load separately built modular codec objects (for ffmpeg), Fedora would almost certainly provide and include that framework code (and would be able to pick and include codecs which are determined to be legally acceptable).

In that case you would get a lot of object code duplication

Consider a simplified version of FFmpeg that contains only H.264, VP8, and vorbis.

There is common code shared between H.264 and VP8 that is only built if one of them is enabled. It isn't built if just Vorbis is enabled. 

If you configure:
VP8 only -> vp8 decoder, video common code
H.264 only -> h.264 decoder, video common code
Vorbis only -> Vorbis only
VP8+H.264 -> h.264 decoder, vp8 decoder, video common code

Furthermore this common code isn't API stable. It is the magic smoke that makes FFmpeg go fast <http://blogs.gnome.org/rbultje/2010/06/27/googles-vp8-video-codec/>. For all it's hype liboil failed to go anywhere. 

> But, we're digressing from this specific issue, which is that Google is forking ffmpeg within Chromium.

So use Google's fork.
Re: 67

> Ah, I see what you're saying, darkshikari. I would think the only difficulty in that approach would be in auditing the post-processed ffmpeg code to ensure that all of the questionable bits are gone. (Also, I'm not entirely convinced that MPEG is actually disabled in the chromium ffmpeg fork, last time I looked, they were still enabling it for HTML5.)

Please check again, Chrome != Chromium
RE 70: 

>Unfortunately, Google has managed to fork that function such that it no longer works with the standard (stable) ffmpeg.
>

The patch attached to this thread reenables that function. 

As far as the fork goes for reasons outlined in the blog entry upstream FFmpeg is inappropriate for chromium. 

To summarize:

1. Chromium's internal FFmpeg copy is based on FFmpeg-mt. 

2. Chromium uses a checked get_bits()

3. Chromium makes use of new FFmpeg features very quickly. 

Please please please read the blog entry if you are curious as to why any of these are important.

Comment 80 by spo...@gmail.com, Nov 10 2010

Re: Comment 79

And, as pointed out in that thread, the patch doesn't really resolve the issue, unless ffmpegsumo is on the system, as opposed to stable ffmpeg.

Chromium has made the choice to use and fork off the experimental branch of ffmpeg, which just means that it is impractical for distributions like Fedora to provide (even via third-party repositories). I think I've explained why this is to the best of my (legally constrained) ability.
Did version 10 drop support for H.264? So is this still an issue?

Comment 82 by spo...@gmail.com, Mar 9 2011

Re: Comment 81

This is still an issue, because chromium is still using a bundled copy of ffmpeg. The fact that they do not compile in the H264 stupport in chromium (they still do in chrome), only resolves one of the many patent risks in ffmpeg. The only way this can be cleanly resolved is if chromium stops bundling a copy of ffmpeg and uses a system copy, if found, (or moves to a saner framework, like gstreamer) but Google clearly has no intention of doing this.
GStreamer is more modular, but it's not necessarily saner. Last I'd heard, the opinion of people experienced in the art of audio-visual application development was that GSt still had a long way to go.
Status: WontFix
this is never going to get done
New year, new life :)

I'm new to this topic but I think it would be very good to have full support of system-installed codecs instead of supporting only one or two.. I think it would be the best for all of us!

Now I'm developing a media album, and if I want the videos to show I've to convert them to another codec, or use an external plugin.. isn't that why video tag came to live? 

I'm using HTML5, and now it's very sad to use some "workaround" to support the different video codecs of my media archive.. 

I'm a programmer, what can I do to help?
Comment 85:

I agree. Chrome should support system-installed codecs! On Windows, there is an API called "Microsoft Media Foundation" for this. It's great and give you full hardware-accelerated video playback.

http://msdn.microsoft.com/en-us/library/windows/desktop/ms694197(v=VS.85).aspx
Project Member

Comment 87 by bugdroid1@chromium.org, Oct 13 2012

Labels: Restrict-AddIssueComment-Commit
This issue has been closed for some time. No one will pay attention to new comments.
If you are seeing this bug or have new data, please click New Issue to start a new bug.
Cc: -f...@sofaraway.org
Labels: -Pri-2 -Mstone-X -ffmpeg -Restrict-AddIssueComment-Commit Pri-3
Status: Available
You know what? Times have changed. We actually might end up implementing this.

I chatted with willchan@ and darin@ about how we should make the MIME map (which powers canPlayType()) support getting types added by higher levels such as content/ and webkit/.

During process launch we then:
  1) Set up a mapping of codec+container to MIME types
  2) Load libffmpegsumo.so
  3) Iterate over which codecs and containers are supported
  4) Stick the mapped MIME types into the net/ module
  5) Profit!
Blocking: chromium:158955
Labels: Hotlist-GoodFirstBug
Project Member

Comment 91 by bugdroid1@chromium.org, Mar 10 2013

Labels: -Area-WebKit -Feature-Media Cr-Content Cr-Internals-Media
Blocking: chromium:224793
Cc: phajdan.jr@chromium.org
Blocking: chromium:28287
Project Member

Comment 95 by bugdroid1@chromium.org, Apr 6 2013

Labels: -Cr-Content Cr-Blink
Blocking: -chromium:224793
Labels: GlutenFree
Got a few users noticing this as we prepare for HTML5ing.

Frankly I'd even be cool if we removed codecs from isTypeSupported/canPlayType as we discovered that they were unplayable ;)
Labels: -Cr-Internals-Media Cr-Internals-Media-FFmpeg Cr-Internals-Media-Codecs
Labels: Hotlist-Fixit
Labels: StaleAvailable
Labels: StaleClosed
Status: WontFix
these bugs are stale for > 6 month. After 3 rounds of call, the bug owners still haven't marked them as StaleKeep. I assume the bug owner has no intention to keep them. Close these bug, and add label StaleClosed, so it's possible to bring them back in the future.
Showing comments 2 - 101 of 101 Older

Sign in to add a comment