Realtime Signal Processing on Embedded GPUs

Transcription

1 Realtime Signal Processing on Embedded s Dr. Matthias Rosenthal Armin Weiss Dr. Amin Mazloumian Institute of Embedded Systems Realtime Platforms Research Group Zurich University of Applied Sciences

2 Motivation In Comparison to FPGA / DSP Solutions: Performance Gain: 100x (e.g. Analog Devices SHARC) Fast Development Time System on Single Chip Cost Effective CPU Nvidia TX Series 2

3 Signal Processing Applications Embedded System mit Video CPU Realtime Linux CUDA OpenCL Video Audio Audio Sensor Data Processed Data 3

4 Challenges Short and Deterministic Latency Video: 16.7 ms / Frame (60 Hz) Audio: 0.33 ms / 32 Samples (@ 96 khz) High Input and Output Data Rate Video: 3.0 Gb/s (1080p@60 24 bit RGB): Audio: 5.5 Gb/s (256x7 Channels 32Bit@96 khz) 4

5 Short and Deterministic Latency Variability in Kernel Launch 99.8% as expected global void identity(float *input, float *output, int numelem) { for (int index = 0; index < numelem; index++) { output[index] = input[index]; } } Outliers numelem = % - 0.2% < 0.1% 5

6 Short and Deterministic Latency Variability in Kernel Launch Latency ~ Buffer Size Outliers 50 ms 6

7 Short and Deterministic Latency Problems How to Improve Deterministic Behavior? à Solution: Persistent CUDA Kernel Eliminate Launch Time 7

8 Short and Deterministic Latency Persistent Kernel CPU Host Application while (running) { if (new_audio_samples() == true) { send_sync_to_(); wait_for sync(); } } global void audiokernel( ) { } // Infinite loop while (true) { } wait_for_cpu_sync(); // Audio channel processing CUDA Kernel wait_for_all_threads_to_finish(); send_sync_to_cpu(); Sync with Memory Accessible by CPU and 8

9 Short and Deterministic Latency CUDA Memory Comparison Zero Copy <-> Managed Zero Copy Managed Memory TX2 TX2 CPU CPU Memory Controller Incl. SMMU DRAM 8GB Buffer Memory Controller Incl. SMMU DRAM 8GB Buffer Not usable for Persistent Kernels 9

10 Short and Deterministic Latency Problems Memory Accessible by CPU and Use Zero Copy Memory for Synchronization What about Parameters? 10

11 Short and Deterministic Latency Infinite Loop: Parameter Communication CPU Application Writes at Arbitrary Time Audio Processing Thread Local Parameter Copy Periodic Update System Memory Parameter Memory (Zero Copy) Slow Access! 11

12 Short and Deterministic Latency Conclusion Use Persistent Kernels Use Memory Accessible by CPU and Use Zero Copy Memory for Synchronization What about Parameters? Speed-up with Local Copy of Parameters How to Make CPUs Deterministic? CPU Core Isolation Custom Linux: Initramfs built with Yocto No Flash Access Anymore During Runtime Minimize Influence from other Applications 12

13 Challenges Short and Deterministic Latency Video: 16.7 ms / Frame (60 Hz) Audio: 0.33 ms / 32 Samples (@ 96 khz) High Input and Output Data Rate Video: 3.0 Gb/s (1080p@60 24 bit RGB): Audio: 5.5 Gb/s (256x7 Channels 32Bit@96 khz) 13

14 High Input / Output Data Rate Separate Buffers TX2 Signal I / O Card PCIe CPU Memory Controller Incl. SMMU DRAM 8GB I/O Buffer memcpy Buffer 14

15 High Input / Output Data Rate memcpy() Measurements memcpy() Time khz for 12h on 3 CPUs (A57) 1.E+09 # Occurrences 1.E+06 1.E % CPU Usage! 1.E Time (µs) TX1 ( Kernel v ) TX2 (Kernel v4.4.15) 15

16 High Input / Output Data Rate Shared Buffer TX2 Signal I / O Card PCIe CPU Memory Controller Incl. SMMU DRAM 8GB I/O Buffer memcpy Buffer 16

17 High Input / Output Data Rate Shared Buffer TX2 Signal I / O Card PCIe CPU Memory Controller Incl. SMMU DRAM 8GB Shared Buffer 17

18 High Input / Output Data Rate Shared Buffer Existing Solutions for Buffer Sharing Direct RDMA (Remote Direct Memory Access) 18

19 High Input / Output Data Rate Direct RDMA Desktop Discrete TX2 Integrated CPUs CPUs Bridge System Memory PCIe Controller Memory Interconnect System Memory PCIe PCIe 3rd Party Device Direct RDMA Memory 3rd Party Device 19

20 High Input / Output Data Rate Shared Buffer Existing Solutions for Buffer Sharing Direct RDMA à Not Available CudaHostRegister() 20

21 High Input / Output Data Rate CudaHostRegister() TX2 Signal I / O Card PCIe CPU Memory Controller Incl. SMMU DRAM 8GB I/O Buffer CudaHostRegister() 21

22 High Input / Output Data Rate Shared Buffer Existing Solutions for Buffer Sharing Direct RDMA à Not Available CudaHostRegister() à Not Implemented on TX2 Video4Linux2 à Userptr Mode 22

23 High Input / Output Data Rate Video4Linux - Userptr TX2 Embedded Camera Video Input CPU Memory Controller Incl. SMMU DRAM 8GB Userptr Mode Mapped Access Buffer 23

24 High Input / Output Data Rate Video4Linux - Userptr TX2 Signal I / O Card PCIe CPU Memory Controller Incl. SMMU DRAM 8GB Userptr Mode Mapped Access Buffer 24

25 High Input / Output Data Rate Shared Buffer Existing Solutions for Buffer Sharing Direct RDMA à Not Available CudaHostRegister() à Not Implemented on TX2 Video4Linux2 à Userptr Mode 25

26 Conclusion Short and Deterministic Latency à Persistent CUDA Kernel High Input / Output Data Rate à Shared Buffer I/O <-> (based on Video4Linux) Feasibility of Low-Latency Signal Processing on Audio Signal Processing on Nvidia TX2 with kHz 26

27 Questions Website: Blog: Github: