Is the goal to provide the FLAC codec in the FLAC format or in MP4/WebM? The latter is more straightforward (no need for a new bytestream spec or a new ChunkDemuxer parser). Such streams could be used with other features, such as EME, that focus on the more common containers.

We should find out how authors intend to use FLAC with MSE.

+johnpallett@

(FLAC for src= is now in M56, as of  bug 93887  being fixed earlier this week).

Only just seen this issue after requesting this feature in https://bugs.chromium.org/p/chromium/issues/detail?id=93887#c59.

We have implemented both ondemand and live FLAC MP4 streams according to https://github.com/xiph/flac/blob/master/doc/isoflac.txt and had great success playing them using MSE in Firefox (>51).

An on-demand test asset can be provided privately on request.

We would really like to see Chrome support this feature, preferably asap.

Today we've launched a public trial simulcast of BBC Radio 3 using FLAC in MP4.

The trial is hosted at http://www.bbc.co.uk/taster/projects/radio-3-concert-sound, and will be available for the next four weeks.

(gentle ping)

We'll be simulcasting Radio 3 for the duration of the Proms season this year, launching next week.

We will be streaming using the encapsulation spec linked above and have had great success patching Chromium in the lab, which was fairly straightforward since only the MP4 parser requires changes.

It would be great for our audiences who choose Chrome to be able to enjoy lossless streaming.

Rescoping this bug to be to support FLAC in MP4 for Chrome MSE.

See also:
https://github.com/w3c/media-source/issues/188
https://bugzilla.mozilla.org/show_bug.cgi?id=1286097

and especially:
https://github.com/xiph/flac/blob/master/doc/isoflac.txt

@ #11, can you describe if EME / CENC is part of your use-case, and if so, how is EME / CENC signaled using the isoflac.txt requirements of a distinct "fLaC" AudioSampleEntry (extended to contain a 'dfla' box): where would the 'dfla' box exist in the CENC version of such an mp4?

@wolenetz: CENC isn't currently part of our use case, but I think it would be generally useful to have this capability.

As far as I can see the standard pattern in 14496-12 would apply i.e the dfLa would remain in its current position as a child of the AudioSampleEntry. The AudioSampleEntry fourcc becomes enca. A sinf box with child frma box containing data_format fLaC would be appended to the AudioSampleEntry, along with other relevant boxes.

@#15 Thank you - that makes sense to me (though there might be other, more EME/DRM/CDM-related, issues in the path of FLAC-in-MP4 + MSE + EME). I'll continue working on the MSE part :)

Note that EME + FLAC on Android is unlikely to be uniformly supported by Chrome in the near future (see bug 747050). IIUC, Firefox already disabled their support for MSE+FLAC on Android (https://bugzilla.mozilla.org/show_bug.cgi?format=default&id=1295886) for similar reasons. Chrome has FFmpeg on Android for unencrypted FLAC decode, so should be able to do MSE FLAC-in-MP4 once  bug 666000  is fixed.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b6be2ef08a6cc5ae86dcc4a9cd2b3e3066353726

commit b6be2ef08a6cc5ae86dcc4a9cd2b3e3066353726
Author: Matt Wolenetz <wolenetz@chromium.org>
Date: Fri Jul 21 18:00:35 2017

Enable src= support for FLAC-in-MP4

Since M56, all Chrome build types that include proprietary codecs and
use the unified media pipeline have been able to demux and decode
FLAC-in-MP4, but canPlayType() hasn't been updated yet.

Further motivation to update that, a prerequisite for supporting
FLAC-in-MP4 with MSE in Chrome is that canPlayType('{audio,video}/mp4,
"flac"') needs to return "probably".

This change updates canPlayType() to indicate FLAC-in-MP4 is supported,
and includes new tests for src= FLAC-in-MP4.

A further complication is that MP4's AudioSampleEntry samplerate field
is fixed point 16.16 bits, so lacks range to handle bitrates above
65535Hz. The FLAC-in-MP4 spec [1] guides how to encode the correct
sample rate when it is outside this scale, and this change includes test
files at both 44.1kHz and 192kHz to verify FFmpegDemuxer extracts the
correct samplerate information.

Also, based on investigation leading to https://crbug.com/747050#c3,
this change includes a comment indicating lack of decode support for
*encrypted* FLAC (in any container) on Android (which would need FLAC
decode support directly in MediaCodec, rather than in MediaExtractor as
some devices appear to do). In the future, perhaps more refined logic
can be used to identify cases where encrypted FLAC-in-MP4 on Android is
supported by the device.

BUG= 666000 ,747050
TEST=Added sfx-flac.mp4 FFmpeg AudioDecoderTest cases, basic media
playback browsertests for bear-flac.mp4 and bear-flac-192kHz.mp4,
FFmpegDemuxerTests, PipelineIntegrationPerfTests, and a
PipelineIntegrationTest; Updated MediaCanPlayTypeTests and
MimeUtilTest.ParseAudioCodecString;

Change-Id: I04df11c34aa539c53dd2c8868257f7ef9608ddc7
Reviewed-on: https://chromium-review.googlesource.com/580575
Reviewed-by: Xiaohan Wang <xhwang@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#488703}
[modify] https://crrev.com/b6be2ef08a6cc5ae86dcc4a9cd2b3e3066353726/components/cdm/browser/cdm_message_filter_android.cc
[modify] https://crrev.com/b6be2ef08a6cc5ae86dcc4a9cd2b3e3066353726/content/browser/media/media_browsertest.cc
[modify] https://crrev.com/b6be2ef08a6cc5ae86dcc4a9cd2b3e3066353726/content/browser/media/media_canplaytype_browsertest.cc
[modify] https://crrev.com/b6be2ef08a6cc5ae86dcc4a9cd2b3e3066353726/media/base/mime_util_internal.cc
[modify] https://crrev.com/b6be2ef08a6cc5ae86dcc4a9cd2b3e3066353726/media/base/mime_util_unittest.cc
[modify] https://crrev.com/b6be2ef08a6cc5ae86dcc4a9cd2b3e3066353726/media/filters/audio_decoder_unittest.cc
[modify] https://crrev.com/b6be2ef08a6cc5ae86dcc4a9cd2b3e3066353726/media/filters/ffmpeg_demuxer_unittest.cc
[modify] https://crrev.com/b6be2ef08a6cc5ae86dcc4a9cd2b3e3066353726/media/test/data/README
[add] https://crrev.com/b6be2ef08a6cc5ae86dcc4a9cd2b3e3066353726/media/test/data/bear-flac-192kHz.mp4
[add] https://crrev.com/b6be2ef08a6cc5ae86dcc4a9cd2b3e3066353726/media/test/data/bear-flac.mp4
[add] https://crrev.com/b6be2ef08a6cc5ae86dcc4a9cd2b3e3066353726/media/test/data/sfx-flac.mp4
[modify] https://crrev.com/b6be2ef08a6cc5ae86dcc4a9cd2b3e3066353726/media/test/pipeline_integration_perftest.cc
[modify] https://crrev.com/b6be2ef08a6cc5ae86dcc4a9cd2b3e3066353726/media/test/pipeline_integration_test.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/446e5b893436ceab5dae61e87d5df95467348975

commit 446e5b893436ceab5dae61e87d5df95467348975
Author: Matt Wolenetz <wolenetz@chromium.org>
Date: Fri Jul 21 21:36:32 2017

Add test cases to verify extracted metadata for .flac and flac.mp4

Since M56, we've had .flac and flac-in-mp4 demux and decode support in
Chrome, but we haven't included AudioVideoMetadataExtractorTests to
confirm expectations. This change adds those.

It also calls out expected "double" values for duration metadata that
might be needed when  bug 747480  is fixed.

BUG= 666000 , 747480 

Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel
Change-Id: I072ba8561d0912169848e96deaf87dfc636ee54b
Reviewed-on: https://chromium-review.googlesource.com/581937
Reviewed-by: Tommy Li <tommycli@chromium.org>
Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#488754}
[modify] https://crrev.com/446e5b893436ceab5dae61e87d5df95467348975/media/filters/audio_video_metadata_extractor_unittest.cc

Update: on a local in-progress CL to enable FLAC-in-ISOBMFF via MSE in Chrome, I have confirmed successful play of http://www.bbc.co.uk/taster/projects/radio-3-concert-sound2 (along with some other tests).

More info/PSA's/feature launching updates to come soon :)

While I get the process pieces in place, I finished off the initial version of implementation of this feature. It's in review at https://chromium-review.googlesource.com/c/588249

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/cc79060bcce11b0cb6fafa673a2a20dcb12bd077

commit cc79060bcce11b0cb6fafa673a2a20dcb12bd077
Author: Matt Wolenetz <wolenetz@chromium.org>
Date: Wed Aug 02 02:07:37 2017

MSE: Add support for FLAC-in-MP4

Before this change, Chrome (with proprietary codecs enabled) supported
FLAC-in-MP4 only in the regular src= media pipeline configuration.

This change adds support for FLAC-in-MP4 in the MSE media pipeline to
Chrome (with proprietary codecs enabled). The gating on proprietary
codecs is not new (all of our existing src= and MSE demuxing of MP4
currently requires proprietary codecs in the build flags).

This new FLAC-in-MP4 with MSE support is gated by a new kMseFlacInIsobmff
base::Feature.

Specifically, this change:
* Adds MseFlacInIsobmff feature, disabled by default (it can be enabled
  by cmdline --enable-features=MseFlacInIsobmff).
  This flag controls MediaSource.isTypeSupported(..) and
  mediaSource.addSourceBuffer(..) for FLAC-in-MP4 mimetype+codec
  strings.
* Adds MSE codec histogram tag to count successful addSourceBuffer()
  occurrences containing a mime type indicating FLAC codec. In Chrome
  MSE, no other bytestream besides MP4 supports addSourceBuffer with
  FLAC codec (and that support is new in this CL).
* Updates StreamParserFactory to support (conditioned by feature flag)
  "flac" codec for each of audio/mp4 and video/mp4 mime types.
* Adds |has_flac| parameter to MP4StreamParser ctor because FLAC-in-ISO
  has no standardized audio object type.
* Adds support to MP4StreamParser for parsing 'fLaC' AudioSampleEntry
  and 'dfLa' FLACSpecificBox information to populate an
  AudioDecoderConfig.
* Adds support to MP4StreamParser for higher-than-64k samplerate
  FLAC-in-ISO by parsing the FLACSpecificBox 'dfLa' containing the
  METADATA_BLOCK_STREAMINFO and using its sample rate instead of the one
  in the AudioSampleEntry.
* Adds support to MP4StreamParser for 24-bit SampleFormat
  (the bear-flac*_frag.mp4 test files are 24-bit).
* Enables partial append window trimming in MSE for FLAC-in-MP4
  (conditioned by feature flag). This supports gapless FLAC playback
  using MSE.
* Verifies the following information in 'dfLa' matches the info in the
  container and in the addSourceBuffer() call, giving parse error on
  failure:
 * fLaC AudioSampleEntry (or fLaC original format with cenc/sinf
   signalling) must only occur for SourceBuffers created with a
   FLAC-in-MP4 mimetype+codec.
 * dfLa must (only) occur for streams with fLaC AudioSampleEntry (or
   fLaC original format with cenc/sinf signalling).
 * First metadata block in dfLa must be METADATA_BLOCK_STREAMINFO
 * Only version 0 of dfLa is allowed
 * Only 0 dfLa Flags field is allowed
 * Channel count in METADATA_BLOCK_STREAMINFO must match that in the
   AudioSampleEntry.
 * Sample size in METADATA_BLOCK_STREAMINFO must match that in the
   AudioSampleEntry.

* Includes new unit and browser tests:
 * content_browsertests:
    MediaSourceTest.Playback_AudioOnly_FLAC_MP4_Unsupported
     (the feature is forced off for this test, playback failure is verified)
    MediaSourceFlacInIsobmffTest.Playback_AudioOnly_FLAC_MP4_Supported
     (the feature is forced on for this test, playback success is verified)
 * media_unittests (with feature forced on):
    MP4StreamParserTest.Flac and Flac192kHz
    PipelineIntegrationTest.MediaSource_FlacInMp4_Hashed

* Also includes audio/mp4 FLAC media capabilities browsertest case,
  though underlying support for CanDecode does not currently
  differentiate kFile vs kMediaSource (TODO added).

Note, like other MSE ISO streams, the coded frame (eg, FLAC sample)
duration is determined from the ISO boxes, not from inspecting (a
possibly encrypted) coded frame header. This has similar consequence as
other codecs in ISO of potential overlap-splicing or worse if the ISO
metadata is incorrect with respect to the decode output of the coded
frames.

See also [1] for the FLAC-in-MP4 encapsulation specification (version
0.0.4 (draft) was current when this change was authored).
Note, this change ignores the following metadata block types (see [2])
other than METADATA_BLOCK_STREAMINFO in the FLACSpecificBox (similar to
current ffmpeg demuxing of FLAC-in-MP4):
* Padding (this is stream, not timeline, padding)
* Application
* SeekTable
* Vorbis Comment (informally, "FLAC tags")
* CueSheet
* Picture

[1] https://github.com/xiph/flac/blob/master/doc/isoflac.txt
[2] https://xiph.org/flac/format.html

Intent to implement and ship:
https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/ntoLfR7rbmE

BUG= 666000 
TEST=Multiple new tests; also manual success with http://www.bbc.co.uk/taster/projects/radio-3-concert-sound2

Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel
Change-Id: I70bbe30d78d4769243d886f4a521a6b03e08a890
Reviewed-on: https://chromium-review.googlesource.com/588249
Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org>
Reviewed-by: Chrome Cunningham <chcunningham@chromium.org>
Cr-Commit-Position: refs/heads/master@{#491218}
[modify] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/content/browser/media/media_capabilities_browsertest.cc
[modify] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/content/browser/media/media_source_browsertest.cc
[modify] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/media/base/media_switches.cc
[modify] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/media/base/media_switches.h
[modify] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/media/base/test_helpers.h
[modify] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/media/filters/chunk_demuxer.cc
[modify] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/media/filters/stream_parser_factory.cc
[modify] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/media/formats/mp4/box_definitions.cc
[modify] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/media/formats/mp4/box_definitions.h
[modify] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/media/formats/mp4/fourccs.h
[modify] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/media/formats/mp4/mp4_stream_parser.cc
[modify] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/media/formats/mp4/mp4_stream_parser.h
[modify] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/media/formats/mp4/mp4_stream_parser_unittest.cc
[modify] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/media/test/data/README
[add] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/media/test/data/bear-flac-192kHz_frag.mp4
[add] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/media/test/data/bear-flac_frag.mp4
[add] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/media/test/data/sfx-flac_frag.mp4
[modify] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/media/test/pipeline_integration_test.cc
[modify] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/media/test/pipeline_integration_test_base.h
[modify] https://crrev.com/cc79060bcce11b0cb6fafa673a2a20dcb12bd077/tools/metrics/histograms/enums.xml

Intent to implement and ship: https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/ntoLfR7rbmE

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a39a49a3eaf3c4418348ccd35951837421b77777

commit a39a49a3eaf3c4418348ccd35951837421b77777
Author: Matt Wolenetz <wolenetz@chromium.org>
Date: Tue Aug 08 17:57:41 2017

MSE: Enable-by-Default FLAC-in-ISOBMFF support

BUG= 666000 ,750868

Change-Id: I90203ab4947da199b3f53ac33c99dca8e67eee21
Reviewed-on: https://chromium-review.googlesource.com/601289
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#492696}
[modify] https://crrev.com/a39a49a3eaf3c4418348ccd35951837421b77777/media/base/media_switches.cc

#25 enables this feature in M62 (Canary/Dev).
M62 Beta/Stable approvals via launch bug 750868 will cancel unshipping this feature on Beta/Stable, respectively.

FYI - Caleb, MSE fragmented FLAC in MP4 support is in M62 Canary/Dev as of #25.

I created issue 753451 to make sure test coverage is there. I'm assuming you already wrote some automated tests, and that is probably all that is needed here.

+bbcrddave@gmail.com can you test this out in Canary?

I can confirm that, with Canary 62.0.3180.0, I can successfully play the live stream at http://www.bbc.co.uk/taster/projects/radio-3-concert-sound2, as well as a number of internal test streams. Awesome, thanks!