WWDC16: HLS supports Fragmented MP4 – and gets MPEG-DASH compatible!

Not only since our most recent player release, HTTP Live Streaming, or short HLS, is an important part of our services. So we have been even more excited than Apple while they announced at WWDC16 that the HLS protocol will be extended with the support for fragmented MP4.

Although Apple still forces content providers to make use of HLS streaming on iOS for assets longer than 10 minutes, the step to introduce fragmented MP4 can be a huge advance from an encoding perspective. Up to now, it has been necessary to encode content in different formats, to maximize browser coverage and reach most of today’s end-user devices. This increases the storage footprint of your content by 2x and reduces also CDN efficiency as content cannot be effectively reused across devices.

In more detail, to enable playback on e.g,. iOS devices, you need to multiplex your content into MPEG-2 Transport Stream, which is (was) required by HLS. In addition you need to multiplex your content also into fragmented MP4 format to enable native HTML5 playback with MPEG-DASH on e.g., Google Chrome, Firefox, Android and other devices. But with the step towards the usage of fragmented MP4s in combination with HLS, it will be sufficient to multiplex your encoding once to fragmented MP4 and simply use the produced segments for both technologies, HLS and MPEG-DASH – only the manifest files will be different.

So, how will a HLS m3u8 will look like with fMP4 segments, as they are used by MPEG-DASH as well. Here is the example directly from the WWDC16 keynote, pretty straight forward:

Apple already published a HLS test asset, using fragmented MP4 segments, which is available at their website, where also additional guides and documentation about HLS can be found.

MPEG-CENC with HLS

But this is not all! Apple goes even further and will also support DRM-protected fMP4 segments using MPEG-CENC (Common Encryption) in HLS, which is used by MPEG-DASH as well, and supported within all major browsers. Here you can find a recent introduction to DRM on the web, and here you can find a compatibility overview for DRM-systems and browsers, seems we have to update them soon with some more possibilities.

Offline HLS Playback & Offline FairPlay

Furthermore, HLS supports now offline playback which can be used to persist videos on the user device that can be watched later when no internet connection is available, e.g., in the airplane, train, car, etc. The application has full control which renditions will be downloaded. This is especially useful for media files that contain different audio tracks and subtitles as the application could download only the tracks that are important for the user. Moreover, offline playback could also be used with FairPlay content that can now be played back without a connection to the license server.

In-Playlist Timed Metadata

HLS now also supports a new metadata format which is called In-playlist timed metadata. It has similar features as ID3 tags but it comparison to ID3 the metadata information is available in the manifest and not packaged in the media stream. This makes the information available when you download the manifest, which means you get the entire information in advance and therefore it can be used for things like navigation control. It also contains bindings for SCTE-35 tags that can now be carried within In-playlist timed metadata.

Conclusion

Apple announced some very nice HLS features at WWDC16 that could have huge impact on the media industry. In summary the most important new features are:

Support for fragmented MP4

• Makes HLS content compatible with DASH and vice versa
• Encode and multiplex your content once and use it with HLS or DASH
• Increase CDN efficiency while reusing the same content for HLS and DASH
• Reduces the storage footprint of your encoded content by half
• Minimal changes in HLS playlists

Support for MPEG Common Encryption (CENC)

• Supports FairPlay streaming with cbcs mode
• Encode and encrypt your content once and use it with Widevine, PlayReady, PrimeTime and FairPlay
• Also increases CDN efficiency for your encrypted content
• This also further reduces the storage footprint of your encrypted content

Offline HLS playback

• Configurable downloading of media renditions
• Can be used for watch later functionally in airplanes, trains and other places where no internet connection is available

Offline FairPlay playback

• Encrypted content can be played back without internet connection
• Assets must be marked in the backend to allow offline playback

In-playlist metadata

• Same feature support as ID3 tags
• Entire information when you download the manifest and therefore it can be used for things like navigation control
• Contains bindings for SCTE-35 tags

We at bitmovin are now working on some demos using HLS with fragmented MP4 with our player and encoding, to show the huge impact to the streaming ecosystem and its potentials, so stay tuned!

More insights about Apple’s streaming protocol in general can be found in one of our recent blog posts and a detailed overview of streaming technologies and covered platforms in our support section.

All the best,
Reinhard & the Bitmovin Team!

www.bitmovin.com