Audio/Video Updates in Chrome 70

• Cross-codec and cross-bytestream buffering and playback is possible in MSE.
• Chrome now supports Opus in MP4 with MSE.
• Protected content playback is allowed by default on Android.

Support for codec and container switching in MSE

Chrome is adding support for improved cross-codec or cross-bytestream transitions in Media Source Extensions (MSE) playback using a new changeType() method on SourceBuffer. It allows the type of media bytes appended to the SourceBuffer to be changed afterwards.

The current version of MSE (W3C Recommendation 17 November 2016) supports adaptive playback of media; however adaptation requires that any media appended to a SourceBuffer must conform to the MIME type provided when initially creating the SourceBuffer via MediaSource.addSourceBuffer(type). Codecs from that type and any previously parsed initialization segments must remain the same throughout. This means the website has to take explicit steps to accomplish codec or bytestream switching (by using multiple media elements or SourceBuffer tracks and switching among those), increasing application complexity and user-visible latency. (Such transitions require the web app to take synchronous action on the renderer main thread). This transition latency impairs the smoothness of media playback across transitions.

With its new changeType() method, a SourceBuffer can buffer and support playback across different bytestream formats and codecs. This new method retains previously buffered media, modulo future MSE coded frame eviction or removal, and leverages the splicing and buffering logic in the existing MSE coded frame processing algorithm.

Here's how to use the changeType() method:

const sourceBuffer = myMediaSource.addSourceBuffer('video/webm; codecs="opus, vp09.00.10.08"');
sourceBuffer.appendBuffer(someWebmOpusVP9Data);

// Later on...
if ('changeType' in sourceBuffer) {
  // Change source buffer type and append new data.
  sourceBuffer.changeType('video/mp4; codecs="mp4a.40.5, avc1.4d001e"');
  sourceBuffer.appendBuffer(someMp4AacAvcData);
}

As expected, if the passed type is not supported by the browser, this method throws a NotSupportedError exception.

Check out the sample to play with cross-codec and cross-bytestream buffering and playback of an audio element.

Intent to Ship | Chromestatus Tracker | Chromium Bug

Opus in MP4 for MSE

The open and highly versatile audio codec Opus has been supported in the <audio> and <video> elements since Chrome 33. Opus in ISO-BMFF support (aka Opus in MP4) was added after. And now Opus in MP4 is available in Chrome 70 for Media Source Extensions (MSE).

Here's how you can detect if Opus in MP4 is supported for MSE:

if (MediaSource.isTypeSupported('audio/mp4; codecs="opus"')) {
  // TODO: Fetch data and feed it to a media source.
}

If you want to see a full example, check out our official sample.

Due to lack of tools to mux Opus in MP4 with correct end trimming and preskip values, if such precision is important to you, you'll need to use SourceBuffer.appendWindow{Start,End} and SourceBuffer.timestampOffset in Chrome to obtain sample-accurate playback.

Warning: Chrome for Android does not support encrypted Opus content on Android versions prior to Lollipop.

Intent to Ship | Chromestatus Tracker | Chromium Bug

Allow protected content playback by default on Android

In Chrome 70 for Android, the default value of the “protected content” site setting changes from “Ask first” to “Allowed”, lowering the friction associated with playback of such media. This change is possible, in part, because of additional steps taken to clear media licenses alongside cookies and site data, ensuring that media licenses are not used by sites to track users who have cleared browsing data.