SRD

GPAC@MMSys2016

We were at MMSys 2016 talking about new nice features in GPAC: support for MPEG-DASH Spatial Relation Description and HEVC motion-constrained tiling!
We had a quick poster presenting our two demos:

Tile-based adaptation using independent video encoded in H264

In this demo, a 4K tears of steel is split in 9 tiles, each of them encoded in various bitrates and resolutions. The resulting composition of tiles is played in GPAC with frame-level accuracy to rebuild the complete video. Of course, all of this is 100% DASH conformant and produced with MP4Box!

The full content of the demo can be browsed here, and you can have fun with the HD version or the 4K version of the DASH session.

You can use GPAC player with gui to watch the different tiles quality and stats:

ToS_Tile_Screenshot

HEVC Motion-constrained Tile-based adaptation

In this demo, we use HEVC tiling tools with constrained motion to allow replacing a tile of an HEVC bitstream with a tile from another HEVC bitstream with same configuration but different quality. In this use case the different HEVC bitstream represent the same content in order to perform bitrate adaptation at the tile level. The nice thing about this is that the reconstructed bitstream is HEVC compliant and requires only a single decoder for the playback!

The full content of the demo can be browsed here, and you can have fun with the HD version of the DASH session.

You can use GPAC player with gui to watch the different tiles quality and stats:

gpac_dash_hevc_tile

WARNING: the file format for the HEVC tile track or the MPD format are not standardized yet, there will be some changes in GPAC regarding this demo any time soon. The MP4Box documentation will be updated once the standard is finalized.

MP4Box.js : our FOSDEM 2016 talk

Dear readers,

Like every year the GPAC team went to the FOSDEM conference. FOSDEM is the biggest worldwide event about Open-Source Software (OSS). It occurs every year in January in Brussels, Belgium.

The event contained an Open Media track led by EBU and Kaltura. Kudos to them! For a report of what happened in the Open Media room, please read the excellent review from Nicolas Weil.

This year Cyril Concolato made a presentation of the current leads for MP4Box.js:

 

 

https://fosdem.org/2016/schedule/event/mp4box/attachments/slides/937/export/events/attachments/mp4box/slides/937/MP4Box_js@FOSDEM2016.pdf

 

Links:

All the presentations from the Open Media room.

All the videos from the Open Media room.

 

GPAC selected for the Google Summer Of Code (GSoC) 2016

GPAC GSoC page: https://summerofcode.withgoogle.com/organizations/6481702784335872/

Subjects: http://gpac.io/jobs/google-summer-of-code-ideas/

The timeline to see the subject and apply is quite short (25 March for student application deadline): https://developers.google.com/open-source/gsoc/timeline

Please pass along the message. GSoC is a good opportunity for projects to improve and find contributors!

If you have any question, please contact gsoc@gpac.io.

Check key-frame alignment with MP4Box

Dear adaptive streaming followers,

Many of you reported issues when packaging your encoded content to respectively HLS or MPEG-DASH using GPAC (resp. the MP42TS and MP4Box tools). Most of the time, this is due to misalignment of key-frames across the different encoded qualities. In this article, we are going to show how to check key-frame alignment.

A packager like MP4Box doesn’t re-encode your content

MP4Box does two things for you:

  1. Import: It understands your media to import it to the MP4 container. For example: ‘MP4Box -add video.h264 -add audio.aac av.mp4’
    When MP4Box doesn’t understand the format, you may want to specify manually this step with NHML.
  2. Manipulates: MP4Box manipulates the MP4 container (e.g. edit, fragment, cut, dash, encrypt, etc.). One key feature of the MP4 container is the ability to manipulate content without any knowledge about the content format. Theoretically it means that MP4Box can package some MPEG-DASH content even for a codec it wouldn’t know.

But in any case MP4Box does not re-encode the content. For that, please use an encoder (such as FFmpeg – see references at the end of the article). It is your responsibility, as the content editor, to feed MP4Box with some appropriate content at the encoder level.

If your content is not prepared correctly, MP4Box works on a best-effort basis and may (or may not) do its job. MP4Box may or may not print warnings. But some players (like dash.js for MPEG-DASH) may silently fail with the packaged content.

Command-line to get quick summary of key-frames intervals

You will get the average key-frame interval computed for the track with ID TRACK_ID:

Having different numbers for the average GOP length on different files mean that your GOP size differ and key-frames won’t be aligned across various qualities: you will have to re-encode. However there may be cases where the average GOP length is the same, but slight variations may occur resulting in misalignment when DASHing.

Command-line to get complete key-frames list and indexes to check alignment

You’ll get:

Then do it with another source file, and compare:

Additional resources for encoding properly

Some tutorials are available at:

 

Open positions in our lab

The GPAC team at Telecom ParisTech has two new open positions:

We also have an open PhD position on peer-to-peer streaming in web browsers, description available here.

If you would like to join us in our offices in Paris and participate in our open-source and R&D activities, don’t hesitate to contact us!

 

GPAC: build MP4Box only (all platforms)

Dear readers,

As many of you know, GPAC is a full framework providing authoring tools, packagers, streamers, a player and now some js stuff. This can make GPAC cumbersome to build.

However many of you are mainly interested in MP4Box which is easy to build. Whether you are on Linux, Windows or MacOS, here is the definite guide to build MP4Box easily 🙂

Edit 2015/11/03: added MSVC MP4Box minimal build.

Build MP4Box with Visual Studio

We published a minimal solution to build MP4Box.exe only, with zero dependency. No DLL to distribute.

You need Visual Studio 2010 or more recent (a pop-up will ask you to upgrade your projects, just accept). Open build/msvc10/gpac_mp4box_mini.sln and build (Debug or Release).

That’s all!

Build MP4Box with Make

On your system, you need the following tools installed on your system:

  • git
  • gcc
  • make
  • libpthread-dev (should be available on any development system)

In your favorite terminal, type:

To execute MP4Box, assuming you are still in the same directory (i.e. the root of the GPAC repository), you will find the MP4Box executable at:

To install MP4Box on your system:

Check installation

now when you type

you should see

which is the default install folder for the version we have just built. You can tweak this with the ‘-prefix=’ configure option.

If you see /usr/bin/MP4Box (no “local/” in here), uninstall GPAC from your local packager. On MacOS, go to Finder and unmount the app on the left panel, or uninstall it from your package manager (‘port’ or ‘homebrew’ or ‘fink’).

Update to a newer revision

Of course you want to keep updated with the latest build without having to download the full repository again or re-execute the configuration when not necessary (it may be necessary sometimes, see below “Clean your build” if things go unexpectedly):

That’s all!

Clean your build

If things go wrong and you suspect there is an issue in your configuration, the fastest way to restart a clean build is:

Cross-compilation

Cross-compiling GPAC is quite standard, and requires only a modification at the ‘configure’ step. Use ‘–extra-cflags=’ and ‘–extra-ldflags=’ to add your environmment flags:

For example, to use the ‘x86_64-w64-mingw32’ toolchain to cross-build from Linux to Windows:

You can find more examples in our TravisCI script.

Next Step

We have started a multimedia component-level build system called Zenbuild. Zenbuild builds FFmpeg/libav, VLC or GPAC with most of their features enabled (librtmp, jack, openHEVC, etc.). You can start using Zenbuild, it is fully operational!

Using GDB in Visual Studio

Dear followers,

One of the target platform for GPAC is Windows. Developing for Windows in a comfortable manner requires Visual Studio, in particular for debugging. Microsoft recently announced great changes that will make Visual Studio users (including the GPAC developers) happy:

Edit: update from Microsoft on 2015/11/18.

Setup for local debugging

Install software:

  • Install Visual Studio 2015. Install the “Tools for Visual C++ Mobile Development” in a sub-menu (disabled by default).
  • Install gdb/gdbserver. Use MinGW-w64 binaries and put them in your PATH (you can rely on the excellent MSYS2 for a full development environment).
    /!\ Legacy MinGW and Cygwin gdb binaries are known to fail /!\

Build:

  • Use your usual build system and gcc or clang compiler. Don’t forget to enable debugging!

Debugging:

  • In your command-line, launch:
  • In Visual Studio:
    • Open the “command window” (either press CTRL+ALT+A or click: View menu -> Other Windows -> Command Window)
    • Launch:

That’s all! Once Debug.GDBLaunch is attached your program will run. Breakpoints are effective. And you can enjoy the Visual Studio debugging experience:

20150610_msvs_gdb

Setup for remote debugging

This is an extension of the gdbserver we used above.  It allows to debug mobile (android/iOS) or any platform (linux, …) executables from a remote

For specific remote parameters, see:

GPAC scalable and hybrid demo, including French Tennis Open 2015

Dear readers,

For the last few years, the GPAC tools backed demos at the French Tennis Open (Roland Garros) in its demo corner called RG Labs. This year, through the H2B2VS project, we participated in a demonstration showing the use of Hybrid Broadcast Broadband delivery for Ultra HD video content.

The video was encoded using the SHVC format with two layers. The encoder was MPEG reference software. The base HEVC layer, corresponding to HD content, was delivered using MPEG-2 TS, produced by our MP42TS tool. The enhancement layer, enhancing the spatial resolution to UHD, was delivered using MPEG-DASH generated by MP4Box. Both streams were synchronized based on the recent MPEG-2 TS TEMI extensions (inserted by MP42TS in the mux) and played by MP4Client, rendered on this nice curved 4K TV. You can find the press release by France TV here.

RG_TV_small

The GPAC player is very powerful. It features a graphical statistics module able to display and control the player. Look at the screenshot below. It can also be used to choose your HLS or MPEG-DASH quality or limit the download bandwidth to simulate a saturated server environment.

statistics

During the first week of June we were also present at the MediaSync 2015 workshop for talk and demos, as well as invited in the demo session of TVX 2015. We presented some demonstrations developed during the H2B2VS project:

  • HD/UHD hybrid delivery, same demo as the one in RG labs, bug using Tears of Steel (left on the picture).
  • Sign language picture-in-picture service enhancement, with main content on broadcast and sign language video on broadband using MPEG-DASH (right on the picture).
  • Dynamic switch between broadcast and broadband versions of the same content, to allow trick modes in a broadcast (time shifting, rewind, fast forward).

TVX_small

The demonstrations were all using MP42TS for broadcast génération, MP4Box for live or onDemand DASH packaging and MP4Client for the playback. Complete instructions for reproducing the test bed are available here.