Support for Apple latest adaptive streaming format

As some of you may already know, Apple has announced during WWDC2016 the support for fragmented MP4 files in HLS:

https://developer.apple.com/videos/play/wwdc2016/504/

We have been adding support in GPAC for fmp4 HLS, both at the client side and at MP4Box side. You can translate a fmp4 m3u8 manifest into a DASH manifest:

For DASH main profile (using segment lists):

For DASH live profile (using segment template):

Here is GPAC playing the demo fmp4 stream for Apple (https://developer.apple.com/streaming/examples/advanced-stream-fmp4.html)

GPAC_HLS_FMP4

This is early stage work, feedback is of course welcome !

MPEG-DASH SRD and HEVC tiling for VR videos

GPAC@MMSys2016

We were at MMSys 2016 talking about new nice features in GPAC: support for MPEG-DASH Spatial Relation Description and HEVC motion-constrained tiling!
We had a quick poster presenting our two demos:

Tile-based adaptation using independent video encoded in H264

In this demo, a 4K tears of steel is split in 9 tiles, each of them encoded in various bitrates and resolutions. The resulting composition of tiles is played in GPAC with frame-level accuracy to rebuild the complete video. Of course, all of this is 100% DASH conformant and produced with MP4Box. MP4Box has the ability to add any MPD descriptor at period, adaptation set or representation level. For SRD, we use adaptation set descriptors as specified in MPEG DASH; for example:

MP4Box -dash 1000 [other dash params]  source.mp4:desc_as=<SupplementalProperty schemeIdUri=\"urn:mpeg:dash:srd:2014\" value=\"0,0,1,1,1,2,2\"/>

indicates that source.mp4 is placed at X=0, Y=1 with width 1 and height 1 on a tiling grid of size 2×2. This information needs to be manually specified at command line for independent videos, but  is automatically inserted if the source file contains an HEVC tile track (see second demo).

The full content of the demo can be browsed here, and you can have fun with the HD version or the 4K version of the DASH session.

You can use GPAC player with gui to watch the different tiles quality and stats:

ToS_Tile_Screenshot

HEVC Motion-constrained Tile-based adaptation

In this demo, we use HEVC tiling tools with constrained motion to allow replacing a tile of an HEVC bitstream with a tile from another HEVC bitstream with same configuration but different quality. In this use case the different HEVC bitstream represent the same content in order to perform bitrate adaptation at the tile level. The nice thing about this is that the reconstructed bitstream is HEVC compliant and requires only a single decoder for the playback!

The full content of the demo can be browsed here, and you can have fun with the HD version of the DASH session.

You can use GPAC player with gui to watch the different tiles quality and stats:

gpac_dash_hevc_tile

WARNING: the file format for the HEVC tile track or the MPD format are not standardized yet, there will be some changes in GPAC regarding this demo any time soon. The MP4Box documentation will be updated once the standard is finalized.

MP4Box.js : our FOSDEM 2016 talk

Dear readers,

Like every year the GPAC team went to the FOSDEM conference. FOSDEM is the biggest worldwide event about Open-Source Software (OSS). It occurs every year in January in Brussels, Belgium.

The event contained an Open Media track led by EBU and Kaltura. Kudos to them! For a report of what happened in the Open Media room, please read the excellent review from Nicolas Weil.

This year Cyril Concolato made a presentation of the current leads for MP4Box.js:

 

 

https://fosdem.org/2016/schedule/event/mp4box/attachments/slides/937/export/events/attachments/mp4box/slides/937/MP4Box_js@FOSDEM2016.pdf

 

Links:

All the presentations from the Open Media room.

All the videos from the Open Media room.

 

GPAC selected for the Google Summer Of Code (GSoC) 2016

GPAC GSoC page: https://summerofcode.withgoogle.com/organizations/6481702784335872/

Subjects: http://gpac.io/jobs/google-summer-of-code-ideas/

The timeline to see the subject and apply is quite short (25 March for student application deadline): https://developers.google.com/open-source/gsoc/timeline

Please pass along the message. GSoC is a good opportunity for projects to improve and find contributors!

If you have any question, please contact gsoc@gpac.io.

Check key-frame alignment with MP4Box

Dear adaptive streaming followers,

Many of you reported issues when packaging your encoded content to respectively HLS or MPEG-DASH using GPAC (resp. the MP42TS and MP4Box tools). Most of the time, this is due to misalignment of key-frames across the different encoded qualities. In this article, we are going to show how to check key-frame alignment.

A packager like MP4Box doesn’t re-encode your content

MP4Box does two things for you:

  1. Import: It understands your media to import it to the MP4 container. For example: ‘MP4Box -add video.h264 -add audio.aac av.mp4’
    When MP4Box doesn’t understand the format, you may want to specify manually this step with NHML.
  2. Manipulates: MP4Box manipulates the MP4 container (e.g. edit, fragment, cut, dash, encrypt, etc.). One key feature of the MP4 container is the ability to manipulate content without any knowledge about the content format. Theoretically it means that MP4Box can package some MPEG-DASH content even for a codec it wouldn’t know.

But in any case MP4Box does not re-encode the content. For that, please use an encoder (such as FFmpeg – see references at the end of the article). It is your responsibility, as the content editor, to feed MP4Box with some appropriate content at the encoder level.

If your content is not prepared correctly, MP4Box works on a best-effort basis and may (or may not) do its job. MP4Box may or may not print warnings. But some players (like dash.js for MPEG-DASH) may silently fail with the packaged content.

Command-line to get quick summary of key-frames intervals

You will get the average key-frame interval computed for the track with ID TRACK_ID:

Having different numbers for the average GOP length on different files mean that your GOP size differ and key-frames won’t be aligned across various qualities: you will have to re-encode. However there may be cases where the average GOP length is the same, but slight variations may occur resulting in misalignment when DASHing.

Command-line to get complete key-frames list and indexes to check alignment

You’ll get:

Then do it with another source file, and compare:

Additional resources for encoding properly

Some tutorials are available at:

 

Open positions in our lab

The GPAC team at Telecom ParisTech has two new open positions:

We also have an open PhD position on peer-to-peer streaming in web browsers, description available here.

If you would like to join us in our offices in Paris and participate in our open-source and R&D activities, don’t hesitate to contact us!

 

GPAC: build MP4Box only (all platforms)

Dear readers,

As many of you know, GPAC is a full framework providing authoring tools, packagers, streamers, a player and now some js stuff. This can make GPAC cumbersome to build.

However many of you are mainly interested in MP4Box which is easy to build. Whether you are on Linux, Windows or MacOS, here is the definite guide to build MP4Box easily 🙂

Edit 2015/11/03: added MSVC MP4Box minimal build.

Build MP4Box with Visual Studio

We published a minimal solution to build MP4Box.exe only, with zero dependency. No DLL to distribute.

You need Visual Studio 2010 or more recent (a pop-up will ask you to upgrade your projects, just accept). Open build/msvc10/gpac_mp4box_mini.sln and build (Debug or Release).

That’s all!

Build MP4Box with Make

On your system, you need the following tools installed on your system:

  • git
  • gcc
  • make
  • libpthread-dev (should be available on any development system)

In your favorite terminal, type:

To execute MP4Box, assuming you are still in the same directory (i.e. the root of the GPAC repository), you will find the MP4Box executable at:

To install MP4Box on your system:

Check installation

now when you type

you should see

which is the default install folder for the version we have just built. You can tweak this with the ‘-prefix=’ configure option.

If you see /usr/bin/MP4Box (no “local/” in here), uninstall GPAC from your local packager. On MacOS, go to Finder and unmount the app on the left panel, or uninstall it from your package manager (‘port’ or ‘homebrew’ or ‘fink’).

Update to a newer revision

Of course you want to keep updated with the latest build without having to download the full repository again or re-execute the configuration when not necessary (it may be necessary sometimes, see below “Clean your build” if things go unexpectedly):

That’s all!

Clean your build

If things go wrong and you suspect there is an issue in your configuration, the fastest way to restart a clean build is:

Cross-compilation

Cross-compiling GPAC is quite standard, and requires only a modification at the ‘configure’ step. Use ‘–extra-cflags=’ and ‘–extra-ldflags=’ to add your environmment flags:

For example, to use the ‘x86_64-w64-mingw32’ toolchain to cross-build from Linux to Windows:

You can find more examples in our TravisCI script.

Next Step

We have started a multimedia component-level build system called Zenbuild. Zenbuild builds FFmpeg/libav, VLC or GPAC with most of their features enabled (librtmp, jack, openHEVC, etc.). You can start using Zenbuild, it is fully operational!