Stream Processing for High-Performance Embedded Systems

Transcription

1 Stream Processing for High-Performance Embedded Systems William J. Dally Computer Systems Laboratory Stanford University HPEC September 25, 2002 Stream Proc: 1 Sept 25, 2002

2 Report Documentation Page Form Approved OMB No Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE 25 SEP REPORT TYPE N/A 3. DATES COVERED - 4. TITLE AND SUBTITLE Stream Processing for High-Performance Embedded Systems 5a. CONTRACT NUMBER 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Stanford University 8. PERFORMING ORGANIZATION REPORT NUMBER 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR S ACRONYM(S) 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release, distribution unlimited 13. SUPPLEMENTARY NOTES Also see ADM001473, The original document contains color images. 14. ABSTRACT 15. SUBJECT TERMS 11. SPONSOR/MONITOR S REPORT NUMBER(S) 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT UU a. REPORT unclassified b. ABSTRACT unclassified c. THIS PAGE unclassified 18. NUMBER OF PAGES 23 19a. NAME OF RESPONSIBLE PERSON Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18

3 Outline Embedded computing demands high arithmetic rates with low power VLSI technology can deliver this capability but microprocessors cannot Stream processors realize the performance/power potential of VLSI while retaining flexibility Stream Proc: 2 Sept 25, 2002

4 Embedded systems demand high arithmetic rates with low power Filter Bank Beam Former Equalizer Equalizer Combine Filter Bank Beam Former Equalizer Equalizer Combine N Antennae NS Subbands SB Beams For N=10, BW=100MHz, S=16, B=4, about 500GOPs B Beams Stream Proc: 3 Sept 25, 2002

5 VLSI provides high arithmetic rates with low power microprocessors do not PowerPC G4 95mm 2 ~1nJ/op 32b adder + RF, 512 x 163 tracks 205µm x 65µm ~ 0.013mm 2 ~5pJ/op Area 7300:1, Energy 200:1, Ops 4:1 Stream Proc: 4 Sept 25, 2002

6 VLSI provides high arithmetic rates with low power microprocessors do not Operation 32b ALU Operation 32b Register Read Read 32b from 8KB RAM Transfer 32b across chip (10mm) Execute a up instruction (SB-1) Transfer 32b off chip (2.5G CML) Transfer 32b off chip (200M HSTL) Energy (0.13um) (0.05um) 5pJ 0.3pJ 10pJ 0.6pJ 50pJ 3pJ 100pJ 17pJ 1.1nJ 130pJ 1.3nJ 400pJ 1.9nJ 1.9nJ 300: 20:1 off-chip to global to local ratio in : 56:1 in 2010 Stream Proc: 5 Sept 25, 2002

7 Why do Special-Purpose Processors Perform Well? Lots (100s) of ALUs Fed by dedicated wires/memories Stream Proc: 6 Sept 25, 2002

8 Care and Feeding of ALUs Instruction Bandwidth IP Instr. Cache Data Bandwidth IR Regs Feeding Structure Dwarfs ALU Stream Proc: 7 Sept 25, 2002

9 Stream Programs Expose Locality and Concurrency Kernels exploit both instruction (ILP) and data (SIMD) level parallelism. Kernels can be partitioned across chips to exploit task parallelism. Image 0 convolve convolve SAD Depth Map Image 1 convolve convolve Streams expose producer-consumer locality. The stream model exploits parallelism without the complexity of traditional parallel programming. Stream Proc: 8 Sept 25, 2002

10 A Bandwidth Hierarchy exploits locality and concurrency SDRAM SDRAM SDRAM SDRAM Stream Register File ALU Cluster ALU Cluster ALU Cluster 2GB/s 32GB/s 544GB/s VLIW clusters with shared control bit floating-point operations per word of memory BW Stream Proc: 9 Sept 25, 2002

11 Producer-Consumer Locality in the Depth Extractor Memory/Global Data SRF/Streams Clusters/Kernels row of pixels previous partial sums new partial sums blurred row previous partial sums new partial sums sharpened row filtered row segment filtered row segment previous partial sums new partial sums depth map row segment Convolution (Gaussian) Convolution (Laplacian) SAD 1 : 23 : 317 Stream Proc: 10 Sept 25, 2002

12 A Bandwidth Hierarchy exploits kernel and producer-consumer locality SDRAM SDRAM SDRAM SDRAM Stream Register File ALU Cluster ALU Cluster ALU Cluster 2GB/s 32GB/s 544GB/s Memory BW Global RF BW Local RF BW Depth Extractor 0.80 GB/s GB/s GB/s MPEG Encoder 0.47 GB/s 2.46 GB/s GB/s Polygon Rendering 0.78 GB/s 4.06 GB/s GB/s QR Decomposition 0.46 GB/s 3.67 GB/s GB/s Stream Proc: 11 Sept 25, 2002

13 Bandwidth Demand of Applications Stream Proc: 12 Sept 25, 2002

14 Local registers increase effective size and bandwidth of SRF ~90% of live variables are captured in local registers Only 10% of live variables need be stored in stream register file Fixed-size SRF is effectively 10x the size of a VRF that must hold all live variables Bandwidth into FPUs is 10x the SRF bandwidth Stream Proc: 13 Sept 25, 2002

15 Cluster Occupancy > 80% Stream Proc: 14 Sept 25, 2002

16 Performance demonstrated on signal and image processing bit applications 19.8 floating-point application bit kernels 20 GOPS floating-point kernel 0 depth mpeg qrd dct convolve fft Stream Proc: 15 Sept 25, 2002

17 Prototype Prototype of Imagine architecture Proof-of-concept 2.56cm 2 die in 0.18um TI process, 21M transistors Collaboration with TI ASIC Runs all benchmarks at 240MHz Dual-Imagine development board Platform for rapid application development Test & debug building blocks of a 64- node system Collaboration with ISI-East Stream Proc: 16 Sept 25, 2002

18 Imagine is programmed in C at two levels Streams: Sequences of records Kernels: Functions that operate on streams Written in KernelC Compiled by kernel scheduler Stream program: Defines streams, control- and and data-flow between kernels Written in StreamC and C++ Compiled by stream compiler Span Convert Rasterize Stream Proc: 17 Sept 25, 2002

19 Simple example StreamC: void main() { Stream<int> a(256); Stream<int> b(256); Stream<int> c(256); Stream<int> d(1024);... example1(a, b, c); example2(c, d);... } KernelC: KERNEL example1( istream<int> a, istream<int> b, ostream<int> c) { loop_stream(a) { int ai, bi, ci; a >> ai; b >> bi; ci = ai * 2 + bi * 3; c << ci; } } Stream Proc: 18 Sept 25, 2002

20 Communication scheduling achieves near optimum kernel performance NSELECT SHIFTA16 NSELECT S HUFFLE S HUFFLE SHUFFLE SHUFFLE SHUFFLE SHUFFLE SHUFFLE SELECT S HUFFLE SHUFFLE SHUFFLE SELECT COMMUCPERM SELECT S HUFFLE SHUFFLE SHUFFLE NSELECT DATA_IN SHUFFLE DATA_IN GEN_CISTATE COND_IN_D DATA_IN SELECT DATA_IN GEN_CCEND NSELECT DATA_OUT SPCREAD_WT SPCWRITE SELECT DATA_IN DATA_OUT COMMUCDATA NSELECT DATA_IN NSELECT SELECT SELECT DATA_OUT DATA_OUT DATA_OUT DATA_OUT DATA_OUT COMMUCPERM COMMUCPERM COMMUCPERM SELECT NSELECT NSELECT CHK_ANY LOOP I A DD S 1 6 I A D DS 1 6 SHI F T A1 6 I A DD S 1 6 SELECT SELECT I M UL RND1 6 S EL EC T I M UL RND1 6 I M UL RND16 I M ULRND1 6 NSEL ECT I M UL RND1 6 I M U L R N D1 6 CO ND_ I N_ D SPCREAD_W T SPCW RI TE CO M M UCDATA CO M M UCP E RM CO M M UCP E RM CO M M UCP E RM SELECT CO M M UCP E RM I M UL RND1 6 I M UL R ND 1 6 I M ULRND1 6 I M UL RND1 6 I M UL RND16 I M ULRND1 6 I M UL RND1 6 I M U L R N D1 6 I M UL RND1 6 I M UL R ND 1 6 I M ULRND1 6 I M UL RND1 6 I M UL RND16 I M ULRND1 6 I M UL RND1 6 I M U L R N D1 6 I M UL RND1 6 I M UL R ND 1 6 I A D DS 1 6 I M ULRND1 6 I M UL RND1 6 NSELECT I ADDS1 6 I A DD S 1 6 I M UL RND16 I M ULRND1 6 S EL EC T I A D DS 1 6 I M UL RND1 6 I M U L R N D1 6 NSEL ECT I A DD S 1 6 I ADDS1 6 I M UL RND1 6 I M UL R ND 1 6 NSELECT SHUFFLE SHUFFLE I A DD S 1 6 I A D DS 1 6 I A DD S 1 6 I ADDS1 6 I A D DS 1 6 I M ULRND1 6 I M UL RND1 6 I M UL RND16 I M ULRND1 6 I M UL RND1 6 I M U L R N D1 6 I A D DS 1 6 I A DD S 1 6 I M UL RND1 6 I M UL R ND 1 6 I A D DS 1 6 I A D DS 1 6 I M ULRND1 6 I M UL RND1 6 I A DD S 1 6 SHUFFLE I A DD S 1 6 I M UL RND16 I M ULRND1 6 I A D DS 1 6 SHUFFLE I A DD S 1 6 SHUFFLE SHUF F LE I A D DS 1 6 I A DD S 1 6 I A D DS 1 6 I A D DS 1 6 I A DD S 1 6 I ADDS1 6 I A D DS 1 6 I A D DS 1 6 I A DD S 1 6 I M UL RND1 6 I M U L R N D1 6 I M UL RND1 6 I M UL R ND 1 6 I M ULRND1 6 I M UL RND1 6 I M UL RND16 I M ULRND1 6 I M UL RND1 6 I M U L R N D1 6 I A DD S 1 6 I ADDS1 6 I M ULRND1 6 I M UL R ND 1 6 SHUFFLE SHUFFLE SHUF F LE SHUF F LE I A DD S 1 6 I A D DS 1 6 I A DD S 1 6 I A D DS 1 6 I ADDS1 6 I A D DS 1 6 I A DD S 1 6 I A D DS 1 6 I A DD S 1 6 I A D DS 1 6 I ADDS1 6 I A D DS 1 6 SHUFFLE SHUFFLE SHUF F LE DA T A_ I N I A DD S 1 6 SHUFFLE I A DD S 1 6 DA T A_ I N I A D DS 1 6 I A D DS 1 6 I A D DS 1 6 DA T A_ I N I A DD S 1 6 I A DD S 1 6 SELECT DA T A I N I A D DS 1 6 I A D DS 1 6 I A D DS 1 6 N S ELECT DATA O UT S ELECT I A DD S 1 6 I ADDS1 6 I A DD S 1 6 SELECT DA T A_ I N DATA_O UT I A D DS 1 6 NSEL ECT DA T A_ I N DATA_O UT I ADDS1 6 NSEL ECT DATA_O UT NSELECT I A D DS 1 6 I A D DS 1 6 DATA_O UT DA T A_ O UT DATA_O UT CHK_ ANY LO O P G EN C I S TATE G E N _ C C E N D ADD0 ADD1 ADD2 MUL0 MUL1 DIV0 INP0 INP1 INP2 IN P3 OUT0 OUT1 SP_0 SP_0 COM0 MC_0 JUK0 VAL0 ADD0 ADD1 ADD2 MUL0 MUL1 DIV0 INP0 INP1 INP2 INP3 OUT0 OUT1 SP_0 SP_0 COM0 MC_0 JUK0 VAL0 7x7 convolution kernel from depth extraction application (Above) Single iteration schedule (Right) Software pipelining shown Stream Proc: 19 Sept 25, 2002

21 Stream scheduling reduces bandwidth demand by up to 12:1 compared to caching 4 Stream program 4 SRF allocation 30 srf Hash Matrix Coordinate Transform Span Prep Sort Viewport Project Span Convert Merge Lights Shader Rasterize Compact Z Compare Depth buffer Image buffer Open GL graphics pipeline Current DSP programmers attempt to stage data in this manner by hand Stream Proc: 20 Sept 25, 2002

22 We have developed A stream architecture that exploits locality and concurrency Keeps 99% of the data accesses on chip Aligned accesses to SRF Enables efficient use of large numbers (100s) of ALUs Imagine: a prototype stream processor that demonstrates the efficiency of stream architecture Working in the lab at 240MHz 9.6GFLOPS, 19.2GOPS, 6W Programmed in C Sustains ~5GOPS/W at 1.2V (200pJ/OP) and demonstrated image-processing, signal processing, and graphics applications on the Imagine stream processor Stream Proc: 21 Sept 25, 2002

23 Stream processing can be applied to scientific computing Extensions to architecture 64b floating point 100GFLOPS/chip Support 2-D, 3-D, and irregular data structures Stream cache Indexable SRF Estimates suggest we can achieve <$20/GFLOPS <$10/M-GUPS Stream Proc: 22 Sept 25, 2002

24 Conclusion Streams expose locality and concurrency Concurrency across stream elements Producer/consumer locality Enables compiler optimization at a larger scale than scalar processing A stream architecture exploits this to achieve high arithmetic intensity (arithmetic rate/bw) Keeps most (>90%) of data operations local (544GB/s, 10pJ) with low overhead Keeps almost all (>99%) of data operations on chip (32GB/s, 100pJ) The Imagine processor demonstrates the advantages of streaming for image and signal processing 9.6GFLOPs, 19.2GOPs, 6W - measured Stream processing is applicable to a wide range of applications Scientific computing Packet processing Stream Proc: 23 Sept 25, 2002