Working with Live Video and Graphics

Transcription

1 Working with Live Video and Graphics FTF-AUT-F0464 Oliver Tian Auto FAE MAY.2014 TM External Use

2 Agenda Trend of Video and Graphics in Vehicle Roadmap of Cluster Introduction of Rainbow/Vybrid Working scenario Development Ecosystem Conclusion External Use 1

3 Safety for Everyone Cleaner world for Everyone Mobility for Everyone Securely Connected External Use 2

4 Infotainment Today CLOUD Camera Instrument Cluster i.mx, Vybrid, Qorivva Infotainment i.mx, Vybrid Driver and Passenger Mobile Devices Today s infotainment systems are focused on: Connectivity to mobile devices (USB, Bluetooth) Display of analog rear view camera Sharing HMI information with the instrument cluster External Use 3

5 The Connected Vehicle Infotainment + Communication + Security Consumer electronics trends are dictating features in the car Always connected, applications driven, advanced graphics Infotainment systems becoming battleground for Auto differentiation As more connected systems get introduced into the vehicle, the need for security is critical Increasing external communication features (Bluetooth, TPMS, Ethernet, Wi-Fi, etc). Future interface for vehicle-to-vehicle and vehicle-to-infrastructure. TM External Use 4

6 Mobility for Everyone Affordable Solutions for Emerging Markets 100M vehicles annually forecasted before 2020, on top of motorcycle & e-bike growth 80% of quantity growth after 2015 happening in emerging markets Safety and emissions reduction are key for a sustainable development Source: IHS Automotive, February 2014 External Use 5

7 More, More, More for Less, Less, Less More performance, more embedded memory, more safety for less cost, less power and less development effort More Electronic complexity ECUs per car (50+) MCUs per car (100+) In-car Wi-Fi (7.2Mbps and 3.7Bpcs by 2017) isuppli Less Reuse Other markets have less critical applications Some automotive specific challenges External Use 6

8 Today s Car Complex computerized control Millions of lines of code, from multiple vendors Dozens of distinct ECUs, from multiple vendors Shared internal networking (e.g., CAN, FlexRay) Increasing external communications features Telematics, Bluetooth, TPMS, RDS, XM radio, GPS, keyless start/entry, USB ports, Wi-Fi, etc. External Use 7

9 Tomorrow s car -> much more of everything The Infrastructure The Intelligent Transportation System (ITS) V2V/ACAS, V2I, traffic control, autonomous driving, real-time data fusion, pervasive sensing External Use 8

10 Driver Information Systems Roadmap Applications Q3 15 I.MX 8 SOLO 1 st Si High-end Best Graphics, Performance & Integration I.MX53 Jan12 ARM Cortex-Dual A7/M4, VSPA, 2D-ACE, GC355 May12 I.MX 6 DUAL I.MX 6 DUALLITE ARM Cortex-Dual A7/M4, VSPA, 2D-ACE, GC355 I.MX 6 SOLO Q1 14 Q1 15 I.MX 6 SLX I.MX 6 SLX +VPU Proposal Planning Mid-range Optimized Graphics, High Integration Cost Effective Performance MPC5645S (Rainbow) MPC5606S (Spectrum) Jan 13 SVF5/3R (Faraday/Vybrid) ARM Cortex-Dual A5/M4, 2D-ACE, GC355 Color Graphics MACHxx Jun 14 (Halo) ARM Cortex-A5/M4/M0+, 2D- ACE, 2D-GPU, GC355 Pin Compatible QFP/BGA s Q2 15 MACCxx (Corona) ARM Cortex-/M4/M0+, 2D-ACE, 2D-GPU, GC255 Pin Compatible QFP s Execution Production S08 Core S12 Core S12z Core (MagniV) Power Architecture ARM Cortex Security Low-end Lowest System Cost, High Integration S12XHY (Sea Wasp) S12HY (Jellyfish) S08LG32 Aug 12 S12ZVH (Lumen-4WL) Nov 12 S12ZVHY (Lumen-2W) Q1 16 LumenNG512 ARM CortexM0+, 2xCANPhy, MagniV External Use 9 r6 11-Feb-14

11 MPC5645S: Rainbow 2M System Integration VReg Oscillator FMPLLx2 RTC/32Kosc Interrupt Controller PIT WDOG Sys Timer Boot Assist Module (BAM) emios0 16ch Debug JTAG Nexus Class 3+ 2M Flash 64k SRAM Crossbar Slaves PPC TM e200z4d Core (4k I-cache) MMU Communications I/O System emios1 16ch EEE 16ch DMA 3 FlexCAN Crossbar Masters CROSSBAR SWITCH Memory Protection Unit (MPU) 1M Graphics SRAM 4 LINFlex 3 DSPI Z160 2D GFX I/O Bridge 4 I2C VIU RLE Decode SGM DCU Lite Quad SPI2 DCU 20 ch ATD 10bit DDR-2 DRAM Interface TCON RSDS SSD 6 SMD General Characteritics: PPC e200z4d Dual Issue core, 5 stage pipeline, 4k I-Cache 16 entry Memory Management Unit 2M FLASH with ECC 64k SRAM with ECC 16 channel DMA Memory Protection Unit (16 regions) QuadSPI Serial Flash Interface Voltage Regulator with external ballast transistor Real Time Counter + 32kHz crystal oscillator Watchdog, Periodic Interrupt Timer, System Timer 4-16MHz XOSC Frequency Modulated PLL (x2) Nexus 3+ / JTAG Graphics Features: 2D Graphics Accelerator: AMD z160 OpenVG 1M Graphics SRAM Display Control Unit: 4 planes / 16 layers Display Conrol Unit Lite: 2 planes / 4 layers DDR DRAM interface (324BGA only) Video input Unit (VIU) RLE Decoder General Characteristics: Up to 120MHz operation Low power modes -40 to +105C, 3.0V to 5.5V 176LQFP, 208LQFP, 324BGA package options Peripherals and Communications: 6 Stepper Motor Drivers with Stall Detection Sound Generator Module 3xCAN, 3xDSPI, 4xI2C, 4xLIN 32 channel emios (PWM+Timer) 20 channel, 10bit ADC External Use 10

12 Vybrid F Series System NEON optimized CODECS/Libraries Power Management - single 3.3V supply - low voltage reset System Connectivity CAN x2 I2C x4 UART/LIN x6 SPI x3 Media Local Bus 3-wire 10/100 Ethernet External Memory DRAM (16-bit) LPDDR2, DDR3 Dual Quad-SPI Flash (SDR and DDR) NAND/NOR Flash 8/16b HMI Tools Comm Stacks Cortex-A5 Up to 400MHz NEON/FPU 32KB/32KB L1 1.5MB SRAM System and General Purpose DMA HAB 4.1 Security Tamper Detect Watchdog Timer Other Timers (x8) Real Time Clock Pulse Width Modulator GPIO 2x 12-bit SAR ADC with Touchscreen 2x 12-bit DAC With Tone Generation Internal Temperature Monitor Royalty-free RTOS Multimedia Connectivity USB OTG + Phy USB OTG + Phy emmc/sd x2 Audio I/O SAI x4 (i2s x4) ESAI x1 (i2s x6) SP/DIF Receiver/Transmitter Sample Rate Convertor Audio Reciever Multiplexer Display I/O 2D-ACE x2 Animation &Comp Engine OpenVG GPU (GC355) Segment Display Controller (4x40) Camera Input, 18-bit + Composite (4 to 1)) Key Functional Characteristics: Cortex-A5 value processor for best MIPs/mW On-chip SRAM, 2D-ACE, Quad-SPI and RTOS result in low system cost (no DRAM) Flexible memory solution configurable based on application needs (1.5MB SRAM or 1MB SRAM + 512K L2) DDR3 and OpenVG support for performance critical applications Synchronous Audio Interface (SAI) supporting independent I2S, TDM, AC97 and Codec/DSP interfaces Enhanced Serial Audio Interface (ESAI) with I2S and AC97 modes Key Electrical Characteristics: A5 at up to 400MHz, and DDR to +85C (ambient) 3.0V to 3.6V supply (3.3V nominal) Package: 144/176 LQFP; 364MAPBGA Initial Samples: Ready Enablement Production Software including CODECs, Stacks, RTOS UI development Tools for 2D-ACE Radio Reference Design HW and SW External Use 11

13 MACHxx Block Diagram Power Management Single 3.3V supply Low Voltage Detection Low Power Control CAN(FD) x3 UART/LIN x3 SPI x5 MLB 3-wire 10/100 Ethernet + AVB Peripherals Cortex-A5 Up to 320MHz NEON / FPU/ MMU 32K / 32K L1 cache System and General Purpose Memory Protection DMA Security-CSE2 I2C x2 Watchdog Timers Audio / GFx / Video / Display SGM (includes I2S) Digital Video In GC355 OpenVG GPU Autonomous RTC IC/OC Timers / PWM SMD / SSD x6 12-bit SAR ADC 2D-ACE + inline HUD Warping 2D - ACE Open LDI & RSDS Cortex-M4 Up to 160MHz FPU 16K / 16K L1 cache 64 KB TCM I/O Processor Cortex M0+ Up to 80MHz 32K ECC SRAM Internal Memory 4MB ECC Flash 1.3MB Non ECC SRAM 2 x 512KB ECC SRAM External Memory DDR QuadSPI Flash x2 16/32-bit DDR2 16-bit SDR ARM Core Architecture: - Cortex M4 vehicle processor - Cortex A5 application processor - Cortex M0+ I/O processor 4MByte ECC flash 2x 512KB ECC SRAM 1.3MB non-ecc SRAM Supports 2 x WVGA displays: - OpenVG 1.1 GPU - 2 x 2D-ACE display interfaces - DigitalRGB, RSDS, LVDS i/f - Hardware HUD warping engine - Digital camera input Extensive connectivity : - Ethernet AVB, MLB50, CAN-FD I/O Processor (Cortex M0+) - Supports autonomous operation Stepper Motor Drivers - Peripheral control and Low power operation Security (CSE2) - Meets SHE specification - Meets GM s Global B Cybersecurity requirements Functional Safety - Built in support for ASIL-B Software Support - AutoSAR - GC355 - I/OP Stepper motor driver External Use 12 BGA and QFP package options: - 176/208LQFP + 516MAPBGA -40 to +105C T A

14 MPC56xxS vs MACHxx vs Vybrid Feature MPC56xxS MACHxx Vybrid Target Use-Case Spectrum - 1 x WQVGA Rainbow 2 x WQVGA MACCxx 1 x WVGA MACHxx- 2 x WVGA Up to 2 x WVGA Core(s) Spectrum - e200z0h Rainbow - e200z4d Corona - ARM CM4 Halo - ARM CM4/CA5 ARM CM4/CA5 Security Censorship only CSE2 (encrypted protection) CAAM Safety Limited features ASIL-B Limited Features Flash Spectrum-1M, Rainbow-2M Corona-1-2M, Halo-2-4M None supports XIP QuadSPI Flash Graphics SRAM Spectum-160kB, Rainbow-1MB Corona-256kB Halo-1.3MB (+1MB ECC RAM) Up to 1.5MB on-chip SRAM in total (1MB no ECC, 512k ECC) Stepper Motors SMD/Stall Detect Intelligent SMD/SSD None Head-Up Display None Hardware warping engine on Halo GC355 can support warping GPU Spectrum - None Rainbow - Z160-OpenVG Corona/Halo - 2D-GPU Halo - GC355 OpenVG GC355 OpenVG High Speed Serial None MLB50, ENET-AVB, CAN-FD MLB50, ENETx2, USB-HS x2 GRAM Expandability Spectrum None Rainbow SDR/DDR2 Corona SDR Halo (SDR)/DDR2 LPDDR2 / DDR3 Enablement EVB, Basic Compiler tools, Lab Bench Demos EVBs, Complete High-Performance Demos, Optimized Graphics Tools EVBs, Complete High-Performance Demos, Optimized Graphics Tools External Use 13 r0: 23-Sep-13

15 Next Generation Cluster System External Use 14

16 Key Differences Next gen Cluster have internal Flash memory while Vybrid doesn t supports Next gen Cluster DDR2 can be 32-bit wide while Vybrid is 16-bit Vybrid has in addition a L2 cache controller Vybrid ports for internal SRAM are all AXI while in Next gen Cluster some are AXI and others are AHB Vybrid operates the core and some other masters at 400MHz while Next gen Cluster operates at 320MHz (System Frequency is 166MHz and 160MHz respectively) External Use 15

17 About the Masters Masters are the ones that initiate and drive the access to the slaves. A5 and M4 can consume a lot of BW but caches reliefs the system from most of the load. Most of the opcodes require more than 1 cycle to execute and load is reduced based on the type of encoding used. Masters may operate at different frequencies depending whether they are clocked at system frequency or a multiple of it. Latencies and peak BW on each master also depends on the slave being accessed External Use 16

18 r0: 23-Sep-13 2D-ACE: Object Management BW per Layer = Pixel Clock * Bytes of Pixels Maximum 6 Layers blend in a single pixel Higher 0 Priority. Arbitrate / pixel 24bpp Lower 32 External Use 17

19 2D-ACE layers & the pixel-blend stack At each pixel position up to six layers may be blended User can globally configure the DCU to blend with 2, 3, 4, 5 or 6 layers The blend stack determines how each pixel is blended Layers below the lowest priority pixel are not visible The blending settings for the lowest priority pixel are ignored Layers active at pixel position (x,y) Selected pixels in blend stack Blended pixel Pixel ignored External Use 18

20 Vybrid working scenario Single 3.3V Rail DRAM Serial Serial Flash Flash 364BGA MOST INIC Radio Chipset CAN PHY CAN PHY Power Management Unit I 2 S SPI CAN CAN MLB -50 ASRC 16-bit DRAM Interface Dual DDR Quad-SPI Interface Vybrid F ARM Cortex-A5 400MHz NEON 32K/32K Caches OpenVG GPU 1.5MByte SRAM 2D-ACE GPIO RTC Camera Interface with resizing 4-ch ADC I 2 S USB OTG SD/MMC USB I 2 C SD/MMC I 2 C OTG UART Color LCD Up to WVGA 24bpp Rearview Camera input Optional Touch Screen CD drive 2xSD Card or Managed NAND ipod Auth. External Use 19

21 r0: 23-Sep-13 MACCxx/MACHxx working scenario Common Family Platform Premium HUD ENET CAN Common GPU Platform WVGA WVGA /HUD Common ARM Core Architecture: - Cortex A5 application processor for high resolution graphics capability* - Cortex M4 vehicle processor common platform for network communications - Cortex M0+ I/O processor for stepper motor control, peripheral control, and low power performance Scalablbe QFP Pin Compatible Gauges w/ High Res TFT & HUD Advanced Gauges w/ High Res TFTʼs Regular Gauges with TFT Basic SDRAM ENET CAN SDRAM CAN FLASH WVGA FLASH WQVGA LCD WVGA Common GPU Platform: - Common 2D-ACE provides low memory footprint with advanced graphics capability - Less requirement for external memory - Scalable vector graphics & font support - Vivante GC355 & GC255 External Components: - 1x WVGA supported without external RAM - 2x WVGA supported with external RAM - High performance DDR Quad SPI interface** - 16/32 Bit DDR2/SDR** * Product dependent (Halo supports A5) ** Package dependent CAN Gauges with LCD Additional Use-cases possible with different package & memory options External Use 20

22 Digital and Analog Video Video is produced as a series of raster images The Vybrid family includes digital and analog input sources Analog source is a composite video source which is converted to digital format and connected to the digital video source Digital source allows connection of various different video standards In both cases the video data is stored in RAM This allows the image to be displayed by the 2D-ACE is the same way as any other graphic The video images can be automatically scaled to a smaller or larger size BUT the aspect ratio of the video is normally respected External Use 21

23 Video signal flow Analogue In: CVBS The Video Subsystem Analogue Front End (AFE) Video Decoder (VDEC) Digital out: YUV888 Mux VIU Graphics Memory 2D-ACE TFT LCD Digital Path YUV888,RGBnnn, or ITU656 External Use 22

24 A note on aspect ratio In some cases the format of an incoming video will be different to the LCD panel Typical video aspect ratios are 4:3 or 16:9 Some common LCD panels do not conform to this format 480 x 272 (7:4), 800 x 480 (15:9), In cases where the video source does not match the panel aspect ratio it is usual to crop the video or display it as a subset of the panel size External Use 23

25 Synchronizing sources The 2D-ACE refreshes the panel at a rate of around 60 Hz The video input can arrive at one half or less of that Video sources are often interlaced Interlaced video can mean a new frame every 25 Hz By double-buffering the video it is possible to receive a frame while the previous one is being displayed but unless care is taken it is possible to have uneven video playback External Use 24

26 Development Ecosystem CodeWarrior iar GCC PVG illustrator plug-in PVG Converter tool Adobe flash Photoshop External Use 25

27 Conclusion Live Video and Graphics The 2D-ACE can composite multiple sources of data including video and static graphics Allocation and use of memory is an important consideration in system design Animation effects are very easy to implement External Use 26

28 Designing with Freescale Tailored live, hands-on training in a city near you 2014 seminar topics include QorIQ product family update Kinetis K, L, E, V series MCU product training freescale.com/dwf External Use 27

29 Freescale Semiconductor, Inc. External Use