4K HEVC Video Processing with GPU Optimization on Jetson TX1

Transcription

1 4K HEVC Video Processing with GPU Optimization on Jetson TX1 Tobias Kammacher Matthias Frei Hans Gelke Institute of Embedded Systems / High Performance Multimedia Research Group Zurich University of Applied Sciences

2 Goal 1. 4K Video Realtime 2. Capturing 3. Scaling, Mixing 4. Encoding H.265/HEVC 2

3 Requirements Streaming Use Case 2nd Video Source Gbps Mbps Video Input Color Space Conversion Scaling Picture in Picture H.264/H.265 Encoder Audio Audio/Video Mux Encryption Transport Protocol Packer Forward Error Correction Ethernet Output Recorder 3

4 Requirements Hardware Limitations Nvidia Jetson TX1 Development Board Gigabit Ethernet Video Output HDMI x Hz WIFI 802.1ac 2x2 Video Input 4 Images: anandtech.com, wccftech.com

5 Video Input MIPI CSI-2 Video Interface (Camera Serial Interface) 1-4 Lanes Images: wccftech.com, nvidia.com 5

6 Video Input Use Case / Hardware Design Capture HDMI sources 4K p and process them CSI HDMI 4K HDMI 1080p Toshiba TC TC Jetson TX1 Dev Board Process Video Ethernet Stream 4K* requires 8 CSI lanes in our case (hardware limitation) * 2160p30 YUV422 6

7 Video Input Hardware Design Nvidia Jetson TX1 Development Board HDMI2CSI module 7

8 Video Input Capture Driver Development Driver Overview Bridge: tc Host: tegra_vi2 Features Capture 720p60, 1080p60, 2160p30 Capture custom formats Available Open Source: 8

9 Video Rendering Frameworks GStreamer Pipeline-based Multimedia Framework OpenGL Graphics API with GPU Acceleration Plugins for Gstreamer available GStreamer is free software available under the terms of the LGPL license OpenGL and the oval logo are trademarks or registered trademarks of Silicon Graphics, Inc 9

10 Video Processing Example: Scaling, Mixing Gstreamer Pipeline V4L2 Source V4L2 Source Format Convert Scale Mix (PiP) Render HDMI Mixing two sources (4K and 1080p) 10

11 Video Processing Example: Scaling, Mixing Logo 1080p Video 4K Video Images: CC BY-SA Wikimedia 11

12 Video Processing Example: Scaling, Mixing Gstreamer Pipeline V4L2 Source V4L2 Source Format Convert Scale Mix (PiP) Render HDMI Mixing two sources (4K and 1080p) CPU: Using compositor element: 1.2 FPS gst-launch-1.0 v4l2src! 'video/x-raw, format=uyvy, framerate=30/1, width=3840, height=2160'! compositor name=comp sink_0::alpha=1 sink_1::alpha=0.5! xvimagesink sync=false videotestsrc pattern=1! 'video/xraw,format=uyvy, framerate=30/1, width=1000, height=1000'! comp. OpenGL (glvideomixer & glimagesink): 6.8 FPS PiP Pipeline FPS? CPU OpenGL Required Need a solution with better performance => GPU 12

13 GPU Processing GPU Memory Access Methods Unified Virtual Addressing TX1 Zero Copy TX1 Managed Memory TX1 CPU GPU CPU GPU CPU GPU L2 Cache L2 Cache L2 Cache L2 Cache L2 Cache L2 Cache Memory Controller DRAM 4GB Memory Controller DRAM 4GB Memory Controller DRAM 4GB CPU Buffer GPU Buffer Shared Buffer Shared Buffer 13

14 GPU Processing Results Tested use cases PiP Dynamic Logo Fixed Images: CC BY-SA Wikimedia 14

15 GPU Processing PiP Test (GPU Data Transfer and Kernel Execution) Unified Virtual Addressing Step 1: cudamemcpy() to GPU * 12.5 ms Step 2: Execute kernel 9-11 ms Step 3: cudamemcpy() to host * 7.2 ms Step 1: Zero Copy cudamallochost(): Allocate memory on host ** Step 2: Execute kernel ms - Total: 30 ms Total: 25 ms * Upload 4K p, Download 4K ** One time only operation Step 1: Managed Memory cudamallocmanaged(): Allocate shared memory ** Step 2: Execute kernel 9-11 ms Step 3: synchronize with CPU 0.2 ms - Total: 10 ms 15

16 GPU Processing Results PiP pipeline achieves 30 FPS Using managed memory Logo pipeline achieves 30 FPS Using UVA, copy once Additional: Consecutive kernels executed faster PiP Pipeline FPS CPU OpenGL GPU 16

17 Conclusion Hardware Mapping 2nd Video Source Gbps Mbps Video Input Color Space Conversion Scaling Picture in Picture H.264/H.265 Encoder Audio Audio/Video Mux Encryption Transport Protocol Packer Forward Error Correction Ethernet Output GPU Recorder HW Block CPU 17

18 Conclusion Driver for 4K HDMI capturing Video processing on GPU Shared Memory Data transfer mode TX1 for 4K video Capturing Processing Streaming 18

19 Find us on the web: Blog: Github: Hardware Board: 19