Decoding video with a mainline kernel on i.MX6
The i.MX6 processors contain a Video Processing Unit (VPU) that allows video decoding and encoding to be done in hardware.
Thanks to the excellent work from Philipp Zabel at Penguntronix the VPU driver
is supported in the mainline kernel at
The VPU hardware needs a firmware to operate and this firmware is available via NXP BSP.
Firmware are usually installed in the root file system at the
The coda driver in mainline used to be selected as built-in, but this causes problems because the coda driver could be probed prior to the rootfs getting mounted. In this case the VPU firmware is not found and the coda driver does not probe successfully.
In order to fix this problem I changed the coda driver to be built as module in this commit
At the time of this writing the most recent stable kernel is 4.13.5 and the above commit will land in the 4.14 kernel.
Adjusting the CMA size
In order to prevent memory allocation errors when displaying full HD videos at full HD resolutions it is necessary to adjust the contiguous memory allocation size properly.
The default CMA area in mainline is 16MB, which is not enough for the full HD playback case.
In this case passing
cma=128M in the kernel command line is needed.
Patch for adding coda support in Buildroot
Here is a patch I did for adding VPU support for the
Add support for it. Find this patch here.
From a0004f5547d4ae54b5bb5a68dfbcaecce5468d0c Mon Sep 17 00:00:00 2001 From: Fabio Estevam <firstname.lastname@example.org> Date: Mon, 9 Oct 2017 14:45:53 -0300 Subject: [PATCH] configs/imx6-sabresd: Add VPU decoding support Mainline kernel is able to decode video via the coda driver. Signed-off-by: Fabio Estevam <email@example.com> --- board/freescale/imx6-sabresd/linux_qt5.fragment | 1 + configs/imx6-sabresd_qt5_defconfig | 20 ++++++++++++++++++++ 2 files changed, 21 insertions(+) diff --git a/board/freescale/imx6-sabresd/linux_qt5.fragment b/board/freescale/imx6-sabresd/linux_qt5.fragment index ffa5f63..d4f26b1 100644 --- a/board/freescale/imx6-sabresd/linux_qt5.fragment +++ b/board/freescale/imx6-sabresd/linux_qt5.fragment @@ -2,3 +2,4 @@ # ondemand governor with a mainline kernel. # CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y +CONFIG_VIDEO_CODA=m diff --git a/configs/imx6-sabresd_qt5_defconfig b/configs/imx6-sabresd_qt5_defconfig index 832e326..7199cb0 100644 --- a/configs/imx6-sabresd_qt5_defconfig +++ b/configs/imx6-sabresd_qt5_defconfig @@ -75,3 +75,23 @@ BR2_PACKAGE_QT5BASE_GIF=y BR2_PACKAGE_QT5BASE_JPEG=y BR2_PACKAGE_QT5CINEX=y BR2_PACKAGE_QT5CINEX_HD=y + +# NXP firmware package +BR2_PACKAGE_FREESCALE_IMX=y +BR2_PACKAGE_FREESCALE_IMX_PLATFORM_IMX6Q=y +BR2_PACKAGE_FIRMWARE_IMX=y + +# libdrm tests +BR2_PACKAGE_LIBDRM_INSTALL_TESTS=y + +# gstreamer-1.0 +BR2_PACKAGE_GSTREAMER1=y +BR2_PACKAGE_GST1_PLUGINS_GOOD=y +BR2_PACKAGE_GST1_PLUGINS_GOOD_PLUGIN_V4L2=y +BR2_PACKAGE_GST1_PLUGINS_GOOD_PLUGIN_V4L2_PROBE=y +BR2_PACKAGE_GST1_PLUGINS_BAD=y +BR2_PACKAGE_GST1_PLUGINS_BAD_PLUGIN_VIDEOPARSERS=y +BR2_PACKAGE_GST1_PLUGINS_BAD_PLUGIN_KMS=y + +# For automatic firmware loading +BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_MDEV=y -- 2.7.4
This patch basically does the following:
- Select the coda driver as module in the defconfig,
- Install the VPU firmware
- Install some basic Gstreamer plugins
BR2_PACKAGE_LIBDRM_INSTALL_TESTSpackage, so that
modetestapplication can be used.
After applying this patch, generate a Buildroot image for the imx6-sabresd board:
$ make imx6-sabresd_qt5_defconfig $ make
Flash the SD card:
$ sudo dd if=output/images/sdcard.img of=/dev/mmcblk0; sync
Download a video sample file and copy it to the
root directory of the SD
$ wget http://linode.boundarydevices.com/videos/trailer_1080p_h264_mp3.avi $ sudo cp trailer_1080p_h264_mp3.avi /media/<user>/<rootfs-patch>/root/; sync
Insert the SD card in the mx6sabresd and boot it.
Running video decoding with Gstreamer
As the mx6sabresd has an LVDS and HDMI outputs, it is necessary to know what is the connector that corresponds to the HDMI output.
This information can be found by running:
This commands shows lots of information. These are the relevant lines that gives the HDMI connector id number. In our case it shows up as 37:
Connectors: id encoder status name size (mm) modes encoders 37 36 connected HDMI-A-1 510x280 33
Now we are ready to play a video file.
In the mx6sabresd prompt run the following pipeline:
# gst-launch-1.0 filesrc location=/root/trailer_1080p_h264_mp3.avi ! \ avidemux ! h264parse ! v4l2video1dec capture-io-mode=dmabuf ! \ kmssink connector-id=37 name=imx-drm sync=0
Then the video will be displayed in the HDMI output.
The VPU interrupts can be checked to confirm that the VPU is actually being used:
# cat /proc/interrupts | grep vpu 29: 1085 0 0 0 GPC 12 Level 2040000.vpu
Running the Gstreamer pipeline example above with a
& symbol, allows us to
top command to observer the CPU utilization while the video is running:
it only consumes 1% of CPU as all the decoding work is done by the hardware VPU
block. Awesome! :-)
At the time of this writing the kmssink is not able to display a video into a display, whem their resolutions do not match. For example, playing a full HD video on the 1024x768 LVDS display is currently not supported via IPU scaler. It is possible to use the standard scaling plugins from Gstreamer though.