ARM Multimedia IP: working together to drive down system power and bandwidth

Transcription

1 ARM Multimedia IP: working together to drive down system power and bandwidth Speaker: Robert Kong ARM China FAE Author: Sean Ellis ARM Architect 1

2 Agenda System power overview Bandwidth, bandwidth, bandwidth! Bandwidth costs more than just power Reducing bandwidth A new member of the ARM Mali TM family Q&A 2

3 System Power Overview Core power ARM Cortex CPU ARM Mali ARM Mali-V550 VPU ARM Mali-DP550 DPU Peripherals Audio Radio Memory and interconnect Bandwidth, bandwidth, bandwidth Encode Memory CPU Driver, app, data Decode Final 3

4 Trends Core power More complex content, more computationally expensive shaders, etc. Peripherals Screen resolution increasing Faster frame rates Memory and interconnect More pixels, more bandwidth More power Unless we do something 4

5 Bandwidth, Bandwidth, Bandwidth! Energy per DRAM access not coming down as fast as bandwidth is going up See graph Energy pj/b Total power mw Effect of DRAM access patterns R in DRAM not so random any more Burst lengths going up Latency staying flat So, we need to do more with less And make better use of what we get Cliché Work smarter, not harder Energy Bandwidth GB/s LPDDR LPDDR2 LPDDR3 LPDDR4 Memory Technology For illustration purposes. Values for 2 channel LPDDR, averaged from various online sources Power 5

6 Designing for Bandwidth Interconnect CPU Encode Must handle peak bandwidth Must guarantee delivery of important data Must meet latency guarantees Must support coherency Interconnect Decode Cores and Peripherals Operate on standard interconnects Driver, app, data Memory Final Memory and controller Must handle bandwidth Off-chip connections physically limited 6

7 Designing for Bandwidth Lower bandwidth reduces stress on the interconnect Reduces requirements Reduces design complexity Reduces bill of materials cost CPU Interconnect Encode Decode The media system is the main user of high-bandwidth memory access in most systems Driver, app, data Memory Final 7

8 Texture Compression Texture access is a big user of bandwidth Up to 90% of content memory reads in 3D content are texture reads Encode Decode Texture compression is the solution ARM texture compression is best in class CPU See the ASTC demo video from CES 2014 at /01/21/arm-mali-gpu-technology-at-ces--whats-new-in-2014 Memory Driver, app, data Final 8

9 Texture Compression - ASTC Mali s support ARM s Adaptive Scalable Texture Compression (ASTC ) Better reduction in texture size Better quality at the same time ASTC is awesome! Texture compression that is higher quality, lower bitrate and with more control than any existing compression formats? Yes please! - Aras Pranckevičius, Unity 3D Better flexibility More choice of pixel formats More choice of bit rates Allows content developers to choose best tradeoff of size v quality Now an industry standard 9

10 Frame Compression - AFBC Mali processors support ARM Frame Buffer Compression (AFBC) Lossless compression for frame buffers Invisible to user 40% bandwidth reduction in typical content Encode Decode Internal and external use cases frame bandwidth reduction Compressed transfer from video processor (VPU) to Memory CPU Driver, app, data Final 10

11 Frame Compression - AFBC ARM Mali s also support AFBC Useful in reducing output bandwidth Also available for input as a texture Useful for various applications VPU- transfer Compression of sensitive textures Encode Decode Particularly useful in G-buffer rendering Compressed render-to-texture Texture is re-read by the later Memory CPU Driver, app, data Final 11

12 Bandwidth Analysis Simple Content Boundary (Triangles) Vertex Shader Rasterizer Fragment Shader Blender Tile Buffer Tiled triangle Triangle Triangle lists, one per lists, tile one per tile Read Frame Written Top-level dataflow 12

13 Bandwidth Analysis Content with G-Buffers Boundary (Triangles) Vertex Shader Rasterizer Fragment Shader Blender Tile Buffer Tiled triangle Triangle Triangle lists, one per lists, tile one per tile G-buffers (Render G-buffers to (Render texture) to texture) +n Written +m Read Top-level dataflow 13

14 G-Buffer Usage Content A 5 Read 5 Written Phase 3 G5 Phase 1 G1 G3 G2 G4 Phase 2 G6 Phase 4 D2 14

15 G-Buffer Usage Content B 13 Read G1 9 Written G2 Phase 2 G5 Phase 4 G7 Phase 1 G3 G4 G6 Phase 3 Phase 5 D1 D2 15

16 Transaction Elimination List of CRCs calculated per tile for frame N CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC Frame N Cyclic Redundancy Check (CRC) Generate CRC for rendered tile contents Check CRCs against previous frame If they match Pixels must be the same Eliminate memory transaction Don t write out pixels for that tile CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC Compared with CRCs calculated for frame N+1 CRC CRC CRC CRC CRC CRC Frame N+1 CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC CRC Eliminated tiles matching previous frame are highlighted by CRC CRC CRC 16

17 Transaction Elimination Reduces bandwidth for frame buffer From 5% to 80% saving, depending on content Even in dynamic content, e.g. First Person Shooter games Best savings during UI and browsing where long usage times and low power are hugely important Encode Decode CPU Memory Driver, app, data Final 17

18 Smart Composition composition can also benefit from CRCs Changed CRCs indicate changed areas of the inputs to the compositor Encode Decode The ARM Mali-DP550 can use this information to avoid re-composing unchanged areas For highly dynamic content, can instead compose directly to display Memory CPU Driver, app, data Final 18

19 New! The ARM Mali-DP550 DPU processor (DPU) Newest member of the ARM Mali multimedia processor family Just Launched Encode Decode CPU Memory Driver, app, data Final 19

20 Mali-DP550 Overview Energy efficient processing all the way to the glass Low power and low memory bandwidth usage Composition, rotation, scaling, post-processing and display output in a single pass Supports system-wide AFBC and Motion Search Elimination Efficient composition of up to 7 layers Scales to 4K resolution Enable 3 rd party display differentiation Co-processor interface enables 3 rd party / SoC vendor IP in Mali display pipeline Feature rich IP Wide range of pre-processing and post-processing features Wide range of YUV/RGB pixel formats including 10-bit YUV Single or dual display output Compatible with all major display standards. 20

21 A smart choice for composition Mali Browser rendering Input Method rendering Mali-DP500 (SD) Mali-DP500 (DD) Popup Window rendering Status Bar rendering Navigation Bar rendering Flattening Composition and display control Frame Buffers Rotation and display control Secondary Core EXTERNAL DISPLAY Memory bandwidth reduction and lower power consumption when doing composition in the Mali-DP550 DPU Mali Presentation activity rendering Status bar rendering Navigation bar rendering Frame Buffers Scale to 1080p Composition and display control Primary Core LOCAL DISPLAY 21

22 Optimization control enhancements More autonomous Less work for CPU More efficient use of Encode Decode Driver becomes more efficient Lower overhead for CPU Lower memory bandwidth More efficient use of CPU Memory Driver, app, data Final 22

23 Future Directions HEVC video support Lower bandwidth for encoded streams Codec supports 10-bit video AFBC 10-bit Encode Decode Introduce 10-bit support across the board Add a visually-lossless AFBC variant Wider engagement with ecosystem AFBC everywhere CPU Memory Driver, app, data Final 23

24 Future Directions Smart geometry is becoming a bigger proportion of data Tessellation makes it easy to create huge amounts of geometry without much effort Encode Decode Change in approach to geometry Developer education is part of the answer Smarter handling of geometry already in drivers Additional hardware support Memory CPU Driver, app, data Final 24

25 System Power Reduction Overview ARM is continuing to reduce power across the whole of the multimedia system: VPU core power encode bandwidth decode bandwidth core power texture bandwidth frame output G-buffer rendering processing Encode Memory CPU Driver, app, data Decode Final 25

26 Conclusion System power is important, and growing ARM has an architecture-to-transistors approach to power Bandwidth reduction is key ARM Mali s and VPUs, and now DPUs too, are low bandwidth and getting lower 26

27 Thank You The trademarks featured in this presentation are registered and/or unregistered trademarks of ARM Limited (or its subsidiaries) in the EU and/or elsewhere. All rights reserved. Any other marks featured may be trademarks of their respective owners 27