Many-core back to the future. Matt Horsnell ARM Research and Development

Transcription

1 Many-core back to the future Matt Horsnell ARM Research and Development 1

2 Outline Introduction How we got to multi-core Focus on multi-core Evolution to many-core? Predicting the future microprocessor? Why I believe it s a great time to be a computer architect Wrap-up 2

3 About me MEng Computer Science, 2003 Final year project: multi-core processors (David May) PhD Computer Science, A Chip Multi-Cluster architecture with locality aware taskdistribution. Research Associate, Object-based Hardware Transaction Memory. Software Engineer, 2010 Clock-tree synthesis and clock-concurrent optimization. Research and Development Architecture and micro-architecture. 3

4 ARM The leading silicon IP company Leading 32 bit RISC architecture Leading Physical IP ~8Bn ARM chips shipped in 2011 Design and license CPUs, GPUs and associated IP > 920 CPU licences > 275 licensees ~1500 designs/year use ARM physical IP 2400 people world-wide >30 locations LSE (FTSE100) and NASDAQ, > 10Bn *Image from FastCompany s 50 most innovative companies 2011 (#12) 4

5 ARM Business model Technology energy-efficient chips 2-3 years to design a processor 2-3 designs introduced per year Range of design points Range of end-markets Partners select a design License fee to access design 3-4 years integrating into SoC Royalty fee per chip 20+ years reuse 5

6 002: Ubiquitous Environments ARM - 1mm 3 to 1km 3 *NSF and University of Wisconsin-Madison *University of Michigan * Samsung Galaxy S3 * Google Nexus 10 1mm 3 1km 3 10c $1000 A B C 6 Cortex-M0; 65 University of Michigan 8.75mm 3 system 2 x solar cell 0.18µm CM Cortex -M3 near-thresh 12µAh Li-ion battery *NXP Cortex-M0 * Fujitsu Calypso Cortex-R4 * Samsung Exynos Cortex-A15 * ARM Cortex-A57

7 ARM - Partnership Momentum ARM Architecture is the number one by volume ARM is growing into new markets and product categories 920 Licenses, 300 Partners, ~50% Shipping Billion Cumulative Billion Cumulative Billion Cumulative 7

8 ARM Research and Development Focus on technologies impacting +10 years from now product pipeline is typically 7 years Always interesting and challenging work research ideas become products can make an influence on the whole industry 8

9 The state of micro-architecture, or how we got to MULTI-CORE 9

10 Feature Size Moore s Law 1.5 um 1.0 um 0.68 um 0.50 um 0.35 um 0.25 um 0.18 um 0.13 um 90 nm 65 nm 45 nm 32 nm Intel transistors 108 KHz Intel Pentium 3.1M transistors 66 MHz Intel Corei7 1.4B transistors 3GHz doubling the number of transistors, economically placed on a chip, every 2 years doubling performance every 2 years *data sourced from 10

11 Dennard Scaling 1974 Robert Dennard at IBM MOSFETs continue to function as voltage-controlled switches while all key figures of merit such as layout density, operating speed, and energy efficiency improve provided geometric dimensions, voltages, and doping concentrations are consistently scaled to maintain the same electric field. 30% dimension shrink 50% area shrink 40% performance increase maintain a 30% supply voltage drop ~50% power reduction 40% faster, 2x transistors, constant power 11

12 Frequency (MHz) Frequency x *data sourced from 12

13 Performance vs. Dennard <2 orders orders Feature Size (um) *data sourced from 13

14 Increase (X) Micro-architecture gains On-die cache, pipelining Super-scalar OOO-Speculative Deep pipeline, Replay, Trace-cache Back to non-deep pipeline * Borkar et. al, The future of the microprocessor, ACM Comms

15 Normalized Performance Pollack s rule Normalized Core Area *data sourced from 15

16 End of Dennard Scaling Classic scaling ended at 130nm Parameter (scale factor = a) Classic Scaling Dimensions 1/a 1/a Voltage 1/a ~1 Current 1/a 1/a Current Scaling Capacitance (A/t) 1/a >1/a Power/Circuit (V.I) 1/a 2 1/a Power Density (VI/A) 1 Delay Circuit 1/a ~1 a Innovation needed to keep driving Moore s Law Materials and Lithography 16

17 Power 2000 Nuclear reactor Hot Plate mw/mm infeasible to economically dissipate heat beyond 800mW/mm 2 *data sourced from 17

18 CPU Clocks/DRAM Latency Memory DRAM density increases with Moore s Law Speed increases far slower Typically 100s of cycles for a memory access to DRAM Hidden to date by caches and bandwidth * Borkar et. al, The future of the microprocessor, ACM Comms

19 On-die cache % of total die area On-die cache (KB) Cache um 0.5um 0.25um 0.13um 65nm energy concerns and inefficient μ-arch led to more cache for efficiency cache sizes increased slowly decreasing die area given to $ most transistors core μ-arch um 0.5um 0.25um 0.13um 65nm * Borkar et. al, The future of the microprocessor, ACM Comms

20 Trend Summary Process scaling continues to follow Moore s Law ITRS suggests 7nm in 2024, Intel 5nm ~2020 Power budget remains constant Frequency frequency increases stopped in years of increasing beyond Dennard scaling increased power Micro-architectural techniques ILP speculation increases power practical pipelining limits <FO4 delays increases power Design complexity Performance Memory wall remains so larger caches 20

21 A focus on MULTI-CORE 21

22 Multi-core Performance increases drive the microprocessor industry performance enables new applications performance enables new markets performance enables new form factors Power fundamentally prevents more frequency scaling performance must come from parallelism ILP already exploited in single core microprocessors Must exploit thread-level parallelism* Multiple cores on the same die gave the industry a new road map Moore s Law now applied to doubling cores every 2 years 22

23 MPCore Power Single CPU unused processors turned off 260MHz, consumes ~160mW Dual-CPU (same MHz, same Vt) Same workload single-threaded, concurrency from OS only could lower MHz (V 2 power saving) Lower power in dual-cpu at same MHz Reduced context switching Increase in cache effectiveness With threaded code, MP offers more performance at lower MHz 23

24 IO coherent devices ARM MPCORE ARM roadmap introduced ARM11 MPCore 2003, shipped ARM-Cortex A-class MPCore roadmap : A8, A9, A5, A15 Quad Cortex-A15 MPCore Quad Cortex-A15 MPCore A15 A15 A15 A15 A15 A15 A15 A15 Processor Coherency (SCU) Up to 4MB L2 cache Processor Coherency (SCU) Up to 4MB L2 cache 128-bit AMBA bit AMBA 4 CoreLink CCI-400 Cache Coherent Interconnect MMU-400 System MMU 24

25 Cortex-A15 MPCore Cortex-A15 uses the similar MP model to Cortex-A9/A5 Data side cache coherence maintained by Snoop Control Unit (SCU) Two ACE master ports and optional ACP slave interface supporting coherent data transfers to non-cached external devices. Integrated GIC Interrupt controller and timer-watchdog units Tightly integrated L2 cache APB APB APB APB ETM v7 Debug ETM v7 Debug ETM v7 Debug ETM v7 Debug STB DPU PFU DPU PFU DPU PFU DPU PFU DuTLB IuTLB DuTLB IuTLB DuTLB IuTLB DuTLB IuTLB A15 Cortex-A5 A15 Core Cortex-A5 A15 Core Cortex-A5 DCU Main TLB ICU STB DCU Main TLB ICU STB DCU Main TLB ICU STB DCUA15 Main TLB Core ICU CP15 CP15 CP15 CP15 Cortex-A5 Core BIU BIU BIU BIU IRQ[n] GIC Timer Duplicate Tags Timer Timer Timer Snoop Controller Unit DDI ACP L2 Cache ACE AXI 25

26 Snoop Control Unit Self-sizing intelligent block to join multiple CPU together Manages the impact and control over support of the coherence protocol Support cache-2-cache transference, monitors for migratory lines Arbitrates multiple CPU s across 1 or 2 load balanced system AXI bus Manages adaptive power down and interfaces with system power controller Decodes and manages private peripheral access Runs at CPU frequency 26

27 New Capabilities in the Cortex-A15 Full compatibility with the Cortex-A9/A5/A8 Supporting the ARMv7 Architecture Addition of Virtualization Extension (VE) Run multiple OS binary instances simultaneously Isolates multiple work environments and data Supporting Large Physical Addressing Extensions (LPAE) Ability to use up to 1TB of physical memory With AMBA 4 System Coherency (AMBA-ACE) Other cached devices can be coherent with processor Many-core multiprocessor scalability Basis of concurrent big.little Processing 27

28 Evolution to MANY-CORE? 28

29 Evolution to Many-Core Base theorem Simpler and smaller processor designs require far less energy to accomplish same amount of compute as a more complex and larger processor design. Approximate rule of thumb held within ARM To increase performance 50% you double the power and area cost of the processor design Quickly reaches point of diminishing returns 29

30 Power Micro-architecture trade-offs High switching power at high performance points big core High-leakage power at low performance points ideal high dynamic range core small core Performance 30

31 Strategy Focus: The Thermal Wall SOC sustained power is limited in mobile devices by thermals; 1.5W to 2W with low-cost POP and stacked memories Power 3W without stacked memories Burst for responsiveness (e.g. Browsing) T >= Tjmax, Tskin Responsiveness is a must Complex active management is needed Opportunistic Residency Managed Sustained Power Tj >= Tmax Tj < Tmax Un-managed Max Power (@Tjmax) Sustained performance (e.g. HD Video Record, Gaming) Power Optimised Low End (e.g. , Voice, MP3) Time 31

32 Use Case Typical day for heavy smartphone user 90 mins voice calls 60 mins 30 mins reading web 30 mins watching HW-accelerated video 50 mins playing Angry Birds or similar 90 mins jogging, listening to MP3s and logging GPS coordinates 10 mins video recording/photo capture 7 hrs sleep with music alarm clock OS typically executing ~28 active processes background synchronizing 32

33 Use Case Measurements 33

34 Use Case Conclusion 34

35 Multiprocessing Capable Many-core Benefits 35

36 ARM s big processor Cortex-A15 Processor announced September Core MP configurable Dual-cores shipped Oct 12 Advanced Capabilities Full ARMv7A architecture Thumb -2, Trustzone, VFP, Neon Virtualization, LPAE AMBA 4 ACE Coherency Shipped Oct 12 Samsung Exynos GHz Products Nexus10 Samsung Chromebook High Performance Up to 1.5GHz for mobile on 28nm 36

37 ARM LITTLE Processor Cortex-A7 Processor announced October Core MP configurable Same Advanced Capabilities Full ARMv7A architecture Thumb -2, Trustzone, VFP, Neon Virtualization, LPAE AMBA 4 ACE Coherency ISA identical to Cortex-A15 Cortex A-7 & Cortex-A15 testchip taped out in Q4 11 Performance efficiency exceeded expectations High Performance Up to 1.2GHz in mobile 37

38 Comparison of big.little Pipelines Cortex-A7 Pipeline Focused on energy efficiency 8-11 stages, in-order, limited dual-issue Cortex-A15 Pipeline Focused on efficient peak performance 15+ stages, out-of-order, multi-issue 38

39 Size Matters Large silicon area costs: Less die per wafer Higher yield impact from silicon imperfections Higher leakage power whenever power applied Typically contains more transistor raising dynamic switching power Not necessarily providing increased performance if gates are required to support architectural complexity rather than instruction execution ARM s LITTLE processor Single Core Cortex-A7 (incl. NEON, FPU, 32kB L1) Device with comparable performance 0.45 mm 2 in 28nm 39

40 Performance Comparison 40

41 Power Efficiency Comparison 41

42 Extending DVFS DVFS sweep over entire operational voltage range of ARM s first big.little processor pair 42

43 Software Use Models big.little switching one CPU active switch between A15 and A7 depending on performance requirements big.little MP both CPUs can be active allocate threads that need high-performance to A15 allocate threads that don t need high-performance, but benefit from best energy efficiency to A7 AMBA 4 hardware coherency between A15 and A7 43

44 Predicting the future MICROPROCESSOR 44

45 A word on prediction 1989 Microprocessors circa 2000 IEEE Spectrum, Gelsinger, P., Intel M transistors, 250MHz no. of transistors 20% over, frequency 2x under, performance 4-8x under 1996 The future of micro-processors IEEE Micro, Yu, A., Intel M transistors, 4GHz no. of transistors 2x over, frequency 5% over predicted the power wall without breakthrough voltage scaling understandably missed the trend to mobile computing 2005 The future of micro-processors ACM Queue, Olukotun et. al, Stanford. no predictions just a discussion of CMPs 2011 The future of micro-processors Comms. ACM, Borkar et. al, Intel. Moore s law continues μ-arch goes beyond homogeneous parallelism, exploit heterogeneity, exploit custom logic software must be able to take advantage of it Various by we ll have 100, 1000, 10,000, 100,000 cores 45

46 Evolution of Mobile Performance Cloud ARM + MP + MP + MP ARM MHz & uarch ARM + MP ARM ARM MHz & uarch ARM + MP GPU ARM ARM HW MHz Architecture U-Architecture MHz Architecture U-Architecture MP MHz Architecture U-Architecture MP Multi-Performance GPGPU MHz Architecture U-Architecture MP Multi-Performance GPGPU Heterogeneity* Domain Specific Off-Load Future? 46

47 Wireless Baseband Packet Processing Subsystem H.265 Video Dec & Enc Image Proc/ recognition Display Near Future Smartphone Peak: 2.5Gb/s Avg: 400Mb/s Peak: 10Gb/s 5 G wireless Graphics & Composition Subsystem Ext Display 4K 240fps Native Display 1080p 120fps Peak: 1Gb/s Power Manager Applications Subsystem Memory Interface 22MP 4K 13Wh Battery 16GB 100GB/s 512GB 47

48 System scaling Today Near Future Increase Notes Cellular 20Mbps 400Mbps 20X Wifi 300Mbps 10Gbps 30X Display 720P 4K 17X Video 720P H.264 4K H X-102X Battery 5.7W 13W 2.2X Assumes constant complexity - 2-4X more H.256 complexity Input Compute Output 20-30x??X 17X 68x How does compute scale in a power and thermal constrained environment? 48

49 Dark Silicon Node 45nm 22nm 11nm Year Area Peak freq Lack 1 of power scaling 1.6 severely limits 2.4 Source: ITRS 2008 Power 1 the complexity 1 of systems! 0.6 (4 x 1) -1 = 25% (16 x 0.6) -1 = 10% Exploitable Si (in 45nm power budget) 10% 25% Source: ITRS

50 The Many-core wall Figure 1: Overview of the models and the methodology * Eseilzadeh et el, ISCA 11 50

51 The Many-core wall? assumes conservative scaling assumes ITRS scaling * Eseilzadeh et el, ISCA 11 Eseilzadeh et el predict the end of multi-core scaling at the 16nm node as early as

52 Increasing Heterogeneity big.little How far can the specialization of micro-architecture improve energy efficiency within a common instruction set architecture? GP/GPU Exposing the compute capability of GPU through a general purpose language OpenCL available on mobile parts (Samsung Chromebook, Nexus 10) HSA foundation My biggest question... How can the benefits of homogeneity in a programming environment be maintained with this increasing heterogeneity? 52

53 ASIC vs. General Purpose ASIC efficiency far greater than [Hameed et al, ISCA 10] x more energy efficient 50x more performance Difficult to identify targets in the general case beyond the obvious candidates (audio, video, crypto, packet) what granularity to target? QScores target multiple general purpose computations [Venkatesh et. al, MICRO 11] Integration how to offload, how to handle contention static compile time target, or dynamic raises the software bar even higher 53

54 a word on software Finding thread level parallelism is hard. * Blake et. al, ISCA 10 Performance portable ensuring optimal performance becomes exponentially harder does this finally mandate runtimes and virtual machines? 54

55 and there s more Fault tolerance how to design for and overcome hard or soft faults Leakage avoidance system designed for power off software designed to enable more power off Bandwidth feeding more and more cores requires huge off-chip bandwidth power and latency concerns 55

56 Why I believe its an interesting time to be a COMPUTER ARCHITECT 56

57 Energy proportional computing Reality is a finite (fixed) energy budget need to re-evaluate architecture and implementation 90/10 rule becomes 10x10 special purpose function acceleration Energy proportional computing becomes the goal in the near term multi-core will likely become many-core many-core will certainly be heterogeneous identifying accelerators becomes necessary Software agnostic performance portable, interfaces, parallel code An new era of rapid dynamics within computer architecture solutions will change at a much quicker tempo 57

58 Something cool transaction elimination Bandwidth to memory = Power Loosely speaking power budget for GPU is ~1W 150pJ to read/write a byte from memory 2x32 LPDDR2 peaks at 4-8GB/s 150pJ * 8GB = 1.2 W Mali-T604 transaction elimination compute a checksum/hash for each completed tile write out tile only if checksum changes trades-off extra compute for reduction of bandwidth 58

59 Memory transaction elimination * 59

60 Conclusion Power constrained micro-architecture is challenging long held design principles Although processes will continue to scale may not be economically sound to increase transistors on a chip unless they can be put to good work The multi-core era is already heterogeneous more heterogeneity not just in the compute over-provisioning of transistors makes accelerators likely Innovation to reduce power at all levels in the micro-architecture compute, interconnect, software, silicon 60

61 Fin Questions? Always looking for good candidates ARM University program Cortex-A programming guide* 61

62 62 BACK-UP

63 FO4/cycle FO4 delay