FPGAs Customize Your Video Solutions

Transcription

1 FPGAs Customize Your Video Solutions Franco Leung Business Manager, Asia Pacific Programmable Solutions Group (PSG), Intel Corporation Copyright Intel Corporation. All rights reserved. Intel Corporation, 2200 Mission College Blvd., Santa Clara, CA , USA

2 Video Industry Trends 4K Ultra HD HDR 8K? Altera video IP technology is deployed in 6 market segments today! 2

3 Per-market FPGA Video Use Cases

4 Dual LVDS 4D+C minilvds, DDR FPGA Automotive IVI Video Applications CPU Switch + Format LCD Video MUX & Format Conversion (YUV & edp) Atom + Cyclone V synergy A B DES Interleave TCON A B A Interface Bridge & Timing Controller (LVDS & minilvds) Navi Scale + Select Video MUX with Scaling LCD ISP IO HPS CAN B R R MAC Decode CPU LCD Early Video / Off-loaded (AVB, BT656) Gesture Recognition (RGB, CAN) 5

5 Professional-AV Example Applications Arria 10: UHD & beyond Cyclone V: legacy Some Atom and Core synergy opps. Video Conferencing / Unified Communications Display and Projection AV Networking Media Processing 6

6 Professional-AV Example Applications Arria 10: UHD & beyond Cyclone V: legacy Some Atom and Core synergy opps. 7

7 Studio Use Cases Arria 10 today Stratix 10 next Synergy with Xeon for server/cloud compute Acquisition Distribution & Contribution Production Consumption 8

8 Studio Use Cases Arria 10 today Stratix 10 next Synergy with Xeon for server/cloud compute 9

9 Medical Diagnostic Imaging (CT, MRI, PET ) Apply range of FPGA families to meet scalable requirements AV Networking (similar to requirement in Pro-AV) Life Sciences (e.g. Endoscopy) Hardware Acceleration (part video processing) 10

10 Industrial Video Surveillance (including WDR, vision ) Low-end to mid-range FPGA families Single-chip Camera SoC (vision continued...) Atom synergy potential Machine Vision (image processing, frame store ) Internet of Things (many emerging exciting IOT video applications for FPGA) 11

11 Visit Altera Business Specific Landing Pages for More Insights 12

12 Altera Video IP Solutions

13 Alpha Blending Mixer PCIe Switch Design Modularity DDR Altera Video Framework DDR4 PLUG & PLAY SYSTEM Connectivity Support DisplayPort 1.3 / 12G SDI HDMI 2.0a / DVI PCIe, DDR, GbE And much more SDI VGA DP HDMI / DVI SMPTE 2022 Custom CVI Scaler JPEG2K H.264 H.265 TICO De- Interlacer Custom FIR Custom Gamma Corrector CVO DP HDMI / DVI TR-03 Chroma Re-sampler VGA Color plane Sequencer Video over IP Support Ethernet MAC/PHY 1588 timing (incl. SMPTE 2059) SMPTE 2022, TR-03, eavb Other protocols available Video Compression H.265, H.264 JPEG2000 TICO Video Processing Suite 28 Highly Configurable Building Block IP for Fast time to Market 4K ready Differentiate your Product Qsys Productivity Tool Create custom IP blocks 14

14 Alpha Blending Mixer PCIe Switch Design Modularity DDR Altera Video Framework DDR4 PLUG & PLAY SYSTEM Connectivity Support DisplayPort 1.3 / 12G SDI HDMI 2.0a / DVI PCIe, DDR, GbE And much more SDI VGA DP HDMI / DVI SMPTE 2022 Custom CVI Scaler JPEG2K H.264 H.265 TICO De- Interlacer Custom FIR Custom Gamma Corrector CVO DP HDMI / DVI TR-03 Chroma Re-sampler VGA Color plane Sequencer Video over IP Support Ethernet MAC/PHY 1588 timing (incl. SMPTE 2059) SMPTE 2022, TR-03, eavb Other protocols available Video Compression H.265, H.264 JPEG2000 TICO Video Processing Suite 28 Highly Configurable Building Block IP for Fast time to Market 4K ready Differentiate your Product Qsys Productivity Tool Create custom IP blocks 15

15 Connectivity IP Connectivity IP Enabling 4K Video Processing and Beyond Avalon ST Video 1080p60 ~ 3G 1 pixel x clk at MHz 4Kp60 ~ 12G 4 pixels clk at MHz 2 pixels clk at 297 MHz 1 pixel at 594 MHz 8kp60 ~ 48G 4 pixels clk at 594 MHz Scalability 16

16 FPGA Value Proposition vs ASSP: Video Connectivity FPGA Flexible lane number configuration, color depth, data mapping and video resolutions Adaption between video interface standards Proven high speed transceiver technology I/O banks support range of I/O standards Fast adaption of newer features and standards Integration with Altera s VIP suite and other IO protocols Multiple source / sink devices integration in single FPGA with single or dual mode ASSP Dual mode not available High volume, short lifespan consumer devices Fixed link rate, lane count, color depth and video resolution 17

17 Video Processing

18 Video and Image Processing IP Suite Collection of highly configurable MegaCore functions that you can use to facilitate the development of custom video and image processing designs suitable for a wide range of applications. Background Picture Scaling in 480i Picture + to Background 1080p60 19

19 20 Altera Video IP Suite Functions Suite of 27 video processing Megacores Developed by in-house system solution team Highly parameterisable IPs Video resolutions from SD to UHD Optimum resource count (bus width adapted to suite maximum resolution) Performance vs. size trade-off, for example: Bi-linear vs. Polyphase edge sharpened scaling Bob/weave vs. motion adaptive de-interlacing Run-time control (e.g. custom coefficients) Clocked video IPs support any video I/O DVI, DL-DVI, DP etc. Embedded or external sync signals Gen-lock for lowest system latency/no DDR Easy system creation/control Using Qsys system builder tool

20 VIP Suite Hardware Verification Trace monitor Wireshark-like video packet inspection Includes pixel viewer for live subsampled video stream image 21

21 Video IP Suite Landing Page Getting Started Videos New! 3 linked tutorials on VIP suite from Altera training partner ALSE Including building video design (in 21 minutes) New! links to latest reference designs Now available for immediate download form Altera design store 22 1) Basic complexity: using ALSE AVDB board More details at: 2) Medium complexity: using Altera Cyclone V development kit (previous favourite on Cyclone III)

22 DisplayPort and HDMI

23 DisplayPort vs HDMI HDMI Video + Audio on same cable Full HD, 4k60fps and 3D support Popular on consumer devices Offers Media content protection Users pay royalty Controlled by HDMI consortium Display Port Video + Audio on same cable Full HD, 4k60fps and 3D support Appearing newer Laptops/PCs Apple big proponent Offers Media content protection No royalty Controlled by VESA 27

24 DisplayPort Standard Next generation video display interface technology Defined by VESA (Video Electronics Standards Association). NO ROYALTY AC coupling and low EMI 3 m cable lengths (15m for lower resolutions) and latchable DP 1.2 with HDCP 1.3 optional 28

25 Altera Advantage: DP Interop Testing Interop with Unigraf, Dell, Samsung, NVidia, AMD DP Interop with 16 different devices (by Altera) DP1.2 RX Certification passed by 2 customers on AVGX DP1.2 TX Certification approved by VESA on AVGX DP1.1a DP1.2 DP 1.3 Sink Unigraf DPR-100 Unigraf DPA-400 (MST) Dell U2410 Unigraf DPR-120 (MST) Unigraf DPA-400 (MST) Dell U2713H, U2913WM (2K) Dell P2715Q (4K60) Samsung U28D590 (4K60) Quantum Data 980 Philips 288P6 (4K60) Viewsonic VP2780-4K Plugfest early testing Source Unigraf DPT-200 Unigraf DPT-200 Nvidia GTX660 (4K60) Nvidia GTX960 (4K60) Nvidia GTX970 (4K60) AMD HD7750 (4K60) AMD HD7700 (4K60) Quantum Data 980 Plugfest early testing 29

26 Altera Advantage: DP Interop Testing Multiple frame rates, resolutions, and fast switching Resolutions. RBG, Resolutions. RBG, Unigraf YCbCr 4:2:2, 4:4:4, Unigraf source Sink: Unigraf, YCbCr 4:2:2, 4:4:4, source Sink: Unigraf, 4:2:0 capability Samsung, Dell 4:2:0 capability Samsung, Dell 640x480 30,60 30, x768 30,60 30,60 720x480 30,60,75,85 30, x900 30,60 30,60 720x576 30,60 30, x900 30, x600 30,60 30, x ,60 30, x768 30,60 30, x ,60 30, x x ,60 30, x720 30, x ,60 30, x768 30,60 30, x ,60 30, x800 30,60,75, x ,60 30, x960 30,60, x ,60 30, x ,60, x x768 30,60 30, x ,60 30, x ,60 30, x ,60 30,60 30

27 HDMI Standard HDMI is the go-to for most consumer electronic vendors Consumer usage driving other markets New Market 31 HDMI 2.0 also supports Controlled by HDMI consortium Users pay royalty 3D support HDMI 2.0: x3 Lanes at 6Gbps -> 4K60p. RGB and YCbCr modes HDCP 2.2 optional Video + Audio on same cable Scrambling mode Deep Color 76% of Visible Colors!

28 Altera HDMI Solution HDMI 2.0 with 4k60p supported on ArriaV, StratixV and Arria10 2 to 8 audio channels 16 bit deep color support 1, 2 or 4 symbols per clock. Scalable Core across multiple families 32

29 Altera Advantage: HDMI Interop Testing HDMI Interop (by Altera and customers) with 27 different devices Plugtest participation since 2012 Plugtest in Q2 15 with HDMI 2.0 on AV Lead customers passed certification for HDMI 1.4 HDMI 2.0 Sink QD980 (4k60) Dell U2713H (2K) Philips (4K30) Samsung U28D590 (4K30) Samsung UN50JS7000 (4K60) UK Samsung UE40HU6900U (4K60) Panasonic TC-58AX800U (4Kp60) UK Panasonic TX-40CX400B (4K60) Viewsonic VP2780-4K (4K60) Sony XBR-55X900C (4K60) Source QD980 (4k60) Nvidia GTX660 (4K30) Nvidia 970 (4K60) AMD HD7750 (4K30) AMD HD7700 (4K30) 33

30 Altera Advantage: HDMI Interop Testing Multiple frame rates, resolutions, and fast switching Resolutions. RBG, YCbCr QD source 4:2:2, 4:4:4, 4:2:0 capability Sinks: QD, Samsung, Dell Resolutions. RBG, YCbCr 4:2:2, 4:4:4, 4:2:0 QD source capability Sinks: QD, Samsung, Dell 1440x x576 50, 100, x x x x x x288 25, 50, x480 60, 120, x x x720 24, 25, 30, 50, 60, 100, , x x , 25, 30, 50, 60, 100, ,30, x240 30, 60, x540 24, 25, 30, 50, 60, 100, x x , 25, 30, 50, 60 30,60 34

31 Altera Connectivity Landing Pages Per function landing pages: SDI: HERE HDMI: HERE DP: HERE Includes: Design examples: now including +VIP Suite Getting started videos, documentation and tutorials Quality metrics, features and device support And much more 35

32 The next chapter H.265HEVC+Video+Encoder+at+NAB+2016/ html

33 Low Latency H.265 Codec on FPGA

34 43 HD Low Latency H.265 SoC Single chip CODEC no external CPU required for algorithm processing User interface flexibility Video I/O 3G SDI, HDMI, DisplayPort PCI-e Gen 2 or 3 Integrate VIP components Integrate customer specific pre-analysis Host CPU SW Linux target Linux drivers provided Elementary stream to Host over PCI-e User code Audio, AV sync, Transport Stream, scrambling, etc.

35 Low Latency H.265 Configurations Available Today: Ultra low latency (I&P) Encoders: 1080p60, 8bit, p60, 10bit, kp30, 8bit Arria Arria Arria Decoders: 1080p60, 12bit, kp30, 12bit, kp60, 12bit, Arria Arria Arria Encode & Decode single chip 1080p60 8bit, StratixV-D8 44

36 FPGA Decoders

37 Single chip FPGA Arria 10 - Decoders Full roadmap for H.265 decoders HEVC 4Kp60 Jan 2015 HEVC 4:2:0 8 Bit Main Profile p60 + 4Kp15 Mar 2015 HEVC 4:2:0 10 Bit Main Profile p60 + 4Kp15 May 2015 HEVC 4:2:2 10/12/16 Bit Main Profile p60 + 4Kp15 Nov rd party decoders also available (MPEG2, H,264 J2K etc). Contact Altera for further details nforse@altera.com 46

38 Other CODEC standards

39 3 rd Party Video Codec IP: Legacy standards Company (Location) Codecs Altera Family IBEX (Japan) MPEG2 Main/High 1080i Codec, H.264, & AVC-I. Encoders / Decoders Arria, Stratix Adaptive Microware (US) MPEG2, H.264 Codecs Cyclone, Arria, Stratix SoC Technologies (Canada) MPEG2, H.264 Codecs (Low Latency) Cyclone, Arria, Stratix A2E Technologies (US) H.264 Codec Cyclone, Arria SoC Digital Design Corp (US) MPEG2, H.264 Codecs (part of design services) Cyclone, Arria, Stratix JointWave (China) H.264 Baseline, Main, High Codec Cyclone, Arria, Stratix Teklium (US / Japan) H.264 Baseline, Main, High Cyclone, Arria, Stratix Tata Elxsi (India) MPEG2, H.264 Main, High Arria, Stratix CAST (US / EMEA) H.264 IP Codec Cyclone, Arria, Stratix Coreel (India) MPEG2 / H.264 Main/High 422 / 10 bit decoders, AVC-I Encoder Cyclone, Arria, Stratix Omnitek (UK) MPEG2 Decoder Cyclone, Arria, Stratix IntoPIX (Belgium) JPEG2000, TICO Arria, Stratix Korusys (UK) H.265 4k bit decoder & H.264 HD bit decoder. Arria Fraunhofer Institute (Germany) H.264 low latency video encoder Arria Please contact 3 rd parties direct for more information

40 Altera Video System Design Examples

41 HDMI 2.0 HDMI 2.0 UDX10: 4K Video Processing with HDMI 2.0 Provided as solution demonstrator (or IP design example) Video available at Includes key functions to support first 4K/UHD processing requirements Plug and play replacement of HDMI with DP, Ethernet or other interface Video source 4K60 10-bit 4:4:4 UDX pipeline Demonstrating: video scaling, format convert, alpha blending mixing, logo insertion, more UHD Panel 4K60 API SW control (from PC command line) Arria 10 Tablet control 50

42 Cyclone V Video Design Example Download from Altera.com/vip Bitec Quad Video Card Bitec DVI Card DVI Input Demo in a box Composite Input Monitor/Display (DVI TX output) Cyclone V GT Development Kit 51

43 Conclusions

44 Conclusions Video: most ubiquitous technology across Altera market segments Altera video framework: best in class to support your needs today and tomorrow Utilize the full range (Max 10 to Stratix 10) of Altera product families to get the right fit for your application Exciting potential synergy options with Intel processors across market segments join us for the future! 53

45 Thank You Copyright Intel Corporation. All rights reserved. Intel Corporation, 2200 Mission College Blvd., Santa Clara, CA , USA