While hardware accelerators are necessary to provide reasonable performance for video playback on embedded devices, the drivers that are provided as binary blobs by hardware vendors cause a lot of problems. They are linked to specific versions of the Linux kernel, may contain security and performance issues, and are pretty much impossible to debug by developers trying to build a system based on these drivers.

We built an i.MX6 based embedded system that simultaneously decodes and displays four videos. Our system solely uses open source drivers to control the available hardware accelerators.

The GUI consists of a Qt application based on QML. Using Qt and QML allows us to use OpenGL for compositing the user interface. OpenGL is backed by the open source etnativ GPU driver and the Vivante GPU.

The Qt application receives the video streams from a Gstreamer pipeline (using playbin). The Gstreamer pipeline contains a v4l2 decoder element, which uses the coda v4l2 driver for the CODA 960 video encoder and decoder IP core (VPU in the Freescale/NXP Reference Manual), and a sink element to make the frames available to the Qt application.

The entire pipeline including the Gstreamer to Qt handover uses dma_bufs to avoid copies in software.

This example shows how to use open source drivers to ease the development of video and graphics applications on embedded systems.