On Power and Multi-Processors

Transcription

1 Brehob -- Portions Brooks, Dutta, Mudge & Wenisch On Power and Multi-Processors Finishing up power issues and how those issues have led us to multi-core processors. Introduce multi-processor systems.

2 Brehob -- Portions Brooks, Dutta, Mudge & Wenisch Stuff upcoming soon 3/20: HW4 due (date later than scheduled) 3/22: MS2 3/23: MS2 meetings Quiz pushed back from 3/22. Date announced next week.

3 Brehob -- Portions Brooks, Dutta, Mudge & Wenisch Power from last time Power is perhaps the performance limiter Can t remove enough heat to keep performance increasing Even for things with plugs, energy is an issue $$$$ for energy, $$$$ to remove heat. For things without plugs, energy is huge Cost of batteries Time between charging

4 Brehob -- Portions Brooks, Dutta, Mudge & Wenisch What we did last time (1/4) Power is important It has become a (perhaps the) limiting factor in performance

5 Brehob -- Portions Brooks, Dutta, Mudge & Wenisch When last we met (2/4) Power is important to computer architects. It limits performance. With cooling techniques we can increase our power budget (and thus our performance). But these techniques get very expensive very very quickly. Both to build and operate active cooling devices. Costs power (and thus $) to cool Active cooling system costs $ to build

6 Brehob -- Portions Brooks, Dutta, Mudge & Wenisch When we last met (3/4) Energy is important to computer architects Energy is what we really pay for (10 per kwh) Energy is what limits batteries (usually listed as mah) AA batteries tend to have mah or (assuming 1.5V) Wh* ipad battery is rated at about 25Wh. Some devices limited by energy they can scavenge. * Watch those units. Assuming it takes 5Wh of energy to charge a 3Wh battery, how much does it cost for the energy to charge that battery? What about an ipad?

7 What uses power in a chip? Power vs. Energy (again ) Brehob -- Portions Brooks, Dutta, Mudge & Wenisch

8 Brehob -- Portions Brooks, Dutta, Mudge & Wenisch Energy For things without plugs, energy is huge Cost of batteries Time between charging (somewhat extreme) Example: ipad has ~42Wh (11,560mAh) which is huge Can Still run down a battery in a couple of hours Costs $99 for a new one (2 years of use or so)

9 Brehob -- Portions Brooks, Dutta, Mudge & Wenisch When last we met (4/4) How do performance and power relate?* Power approximately proportional to V 2 f. Performance is approximately proportional to f The required voltage is approximately proportional to the desired frequency.** If you accept all of these assumptions, we get that an X increase in performance would require an X 3 increase in power. This is a really important fact *These are all pretty rough rules of thumb. Consider the second one and discuss its shortcomings. **This one in particular tends to hold only over fairly small (10-20%?) changes in V.

10 What uses power in a chip? Dynamic (Capacitive) Power Dissipation I V IN V OUT C L Data dependent a function of switching activity 10

11 What uses power in a chip? Capacitive Power dissipation Capacitance: Function of wire length, transistor size Power ~ ½ CV 2 Af Activity factor: How often, on average, do wires switch? Supply Voltage: Has been dropping with successive fab generations Clock frequency: Increasing 11

12 What uses power in a chip? Static Power: Leakage Currents q V T V IN V OUT I DSub k e a k a T Igate I Sub C L Subthreshold currents grow exponentially with increases in temperature, decreases in threshold voltage But threshold voltage scaling is key to circuit performance! Gate leakage primarily dependent on gate oxide thickness, biases Both type of leakage heavily dependent on stacking and input pattern 12

13 Performance & Power So? Brehob -- Portions Brooks, Wenisch What we ve concluded is that if we want to increase performance by a factor of X, we might be looking at a factor of X 3 power! But if you are paying attention, that s just for voltage scaling! What about other techniques? 13

14 Performance & Power Other techniques? Brehob -- Portions Brooks, Wenisch Well, we could try to improve the amount of ILP we take advantage of. That probably involves making a wider processor (more superscalar) What are the costs associated with doing that? How much bigger do things get? What do we expect the performance gains to be? How about circuit techniques? Historically the threshold voltage has dropped as circuits get smaller. So power drops. This has (mostly) stopped being true. And it s actually what got us in trouble to begin with! 14

15 Performance & Power So we are hosed Brehob -- Portions Brooks, Wenisch I mean if voltage scaling doesn t work, circuit shrinking doesn t help (much), and ILP techniques don t clearly work What s left? How about we drop performance to 80% of what it was and have 2 processors? How much power does that take? How much performance could we get? Pros/Cons? What if I wanted 8 processors? How much performance drop needed per processor? 15

16 Multiprocessors Keeping it all working together

17 Why multi-processors? Why multi-processors? Multi-processors have been around for a long time. Originally used to get best performance for certain highly-parallel tasks. But as noted, we now use them to get solid performance per unit of energy. So that s it? Not so much. We need to make it possible/reasonable/easy to use. Nothing comes for free. If we take a task and break it up so it runs on a number of processors, there is going to be a price.

18 Thread-Level Parallelism struct acct_t { int bal; }; shared struct acct_t accts[max_acct]; int id,amt; if (accts[id].bal >= amt) { accts[id].bal -= amt; spew_cash(); } Thread-level parallelism (TLP) Collection of asynchronous tasks: not started and stopped together Data shared loosely, dynamically Example: database/web server (each query is a thread) 0: addi r1,accts,r3 1: ld 0(r3),r4 2: blt r4,r2,6 3: sub r4,r2,r4 4: st r4,0(r3) 5: call spew_cash 6: accts is shared, can t register allocate even if it were scalar id and amt are private variables, register allocated to r1, r2

19 Shared-Memory Multiprocessors Shared memory Multiple execution contexts sharing a single address space Multiple programs (MIMD) Or more frequently: multiple copies of one program (SPMD) Implicit (automatic) communication via loads and stores P 1 P 2 P 3 P 4 Memory System

20 What s the other option? Basically the only other option is message passing We communicate via explicit messages. So instead of just changing a variable, we d need to call a function to pass a specific message. Message passing systems are easy to build and pretty efficient. But harder to code. Shared memory programming is basically the same as multithreaded programming on one processors And (many) programmers already know how to do that.

21 So Why Shared Memory? Pluses For applications looks like multitasking uniprocessor For OS only evolutionary extensions required Easy to do communication without OS being involved Software can worry about correctness first then performance Minuses Proper synchronization is complex Communication is implicit so harder to optimize Hardware designers must implement Result Traditionally bus-based Symmetric Multiprocessors (SMPs), and now the CMPs are the most success parallel machines ever And the first with multi-billion-dollar markets

22 Shared-Memory Multiprocessors There are lots of ways to connect processors together P 1 P 2 P 3 P 4 Cache M 1 Cache M 2 Cache M 3 Cache M 4 Interface Interface Interface Interface Interconnection Network

23 Paired vs. Separate Processor/Memory? Separate processor/memory (left) Uniform memory access (UMA): equal latency to all memory + Simple software, doesn t matter where you put data Lower peak performance Bus-based UMAs common: symmetric multi-processors (SMP) Paired processor/memory (right) Non-uniform memory access (NUMA): faster to local memory More complex software: where you put data matters + Higher peak performance: assuming proper data placement Mem R Mem R Mem R Mem R Mem Mem Mem Mem

24 Shared vs. Point-to-Point Networks Shared network: e.g., bus (left) + Low latency Low bandwidth: doesn t scale beyond ~16 processors + Shared property simplifies cache coherence protocols (later) Point-to-point network: e.g., mesh or ring (right) Longer latency: may need multiple hops to communicate + Higher bandwidth: scales to 1000s of processors Cache coherence protocols are complex Mem R Mem R Mem R Mem R Mem R R Mem Mem R R Mem

25 Organizing Point-To-Point Networks Network topology: organization of network Tradeoff performance (connectivity, latency, bandwidth) cost Router chips Networks that require separate router chips are indirect Networks that use processor/memory/router packages are direct + Fewer components, Glueless MP Point-to-point network examples Indirect tree (left) Direct mesh or ring (right) R R R Mem R R Mem Mem R Mem R Mem R Mem R Mem R R Mem

26 Implementation #1: Snooping Bus MP Two basic implementations Bus-based systems Mem Mem Typically small: 2 8 (maybe 16) processors Typically processors split from memories (UMA) Sometimes multiple processors on single chip (CMP) Symmetric multiprocessors (SMPs) Common, I use one everyday

27 Implementation #2: Scalable MP Mem R R Mem Mem R R Mem General point-to-point network-based systems Typically processor/memory/router blocks (NUMA) Glueless MP: no need for additional glue chips Can be arbitrarily large: 1000 s of processors Massively parallel processors (MPPs) In reality only government (DoD) has MPPs Companies have much smaller systems: processors Scalable multi-processors

28 Issues for Shared Memory Systems Two in particular Cache coherence Memory consistency model Closely related to each other

29 An Example Execution Processor 0 0: addi r1,accts,r3 1: ld 0(r3),r4 2: blt r4,r2,6 3: sub r4,r2,r4 4: st r4,0(r3) 5: call spew_cash Processor 1 0: addi r1,accts,r3 1: ld 0(r3),r4 2: blt r4,r2,6 3: sub r4,r2,r4 4: st r4,0(r3) 5: call spew_cash CPU0 CPU1 Mem Two $100 withdrawals from account #241 at two ATMs Each transaction maps to thread on different processor Track accts[241].bal (address is in r3)

30 No-Cache, No-Problem Processor 0 0: addi r1,accts,r3 1: ld 0(r3),r4 2: blt r4,r2,6 3: sub r4,r2,r4 4: st r4,0(r3) 5: call spew_cash Processor 1 0: addi r1,accts,r3 1: ld 0(r3),r4 2: blt r4,r2,6 3: sub r4,r2,r4 4: st r4,0(r3) 5: call spew_cash Scenario I: processors have no caches No problem

31 Cache Incoherence Processor 0 0: addi r1,accts,r3 1: ld 0(r3),r4 2: blt r4,r2,6 3: sub r4,r2,r4 4: st r4,0(r3) 5: call spew_cash Processor 1 0: addi r1,accts,r3 1: ld 0(r3),r4 2: blt r4,r2,6 3: sub r4,r2,r4 4: st r4,0(r3) 5: call spew_cash 500 V: D: D:400 V: D:400 D: Scenario II: processors have write-back caches Potentially 3 copies of accts[241].bal: memory, p0$, p1$ Can get incoherent (inconsistent)

32 D$ tags Brehob -- Portions Falsafi, Hill, Hoe, Lipasti, Martin, Hardware Cache Coherence D$ data CPU CC Coherence controller: Examines bus traffic (addresses and data) Executes coherence protocol What to do with local copy when you see different things happening on bus bus

33 Snooping Cache-Coherence Protocols Bus provides serialization point Each cache controller snoops all bus transactions take action to ensure coherence invalidate update supply value depends on state of the block and the protocol

34 Snooping Design Choices Controller updates state of blocks in response to processor and snoop events and generates bus xactions Often have duplicate cache tags Snoopy protocol set of states state-transition diagram actions Processor ld/st Cache State Tag Data... Snoop (observed bus transaction) Basic Choices write-through vs. write-back invalidate vs. update

35 The Simple Invalidate Snooping Protocol PrRd / -- Valid PrWr / BusWr Write-through, nowrite-allocate cache Actions: PrRd, PrWr, BusRd, BusWr PrRd / BusRd BusWr Invalid PrWr / BusWr

36 Example time Brehob -- Portions Falsafi, Hill, Hoe, Lipasti, Martin, Processor 1 Processor 2 Bus Processor Transaction Read A Read A Write A Write A Cache State Processor Transaction Read A Read A Write A Actions: PrRd, PrWr, BusRd, BusWr Cache State

37 More Generally: MOESI [Sweazey & Smith ISCA86] M - Modified (dirty) O - Owned (dirty but shared) WHY? E - Exclusive (clean unshared) only copy, not dirty S - Shared I - Invalid Variants MSI MESI MOSI MOESI S O M E I ownership validity exclusiveness

38 Actions: PrRd, PrWr, BRL Bus Read Line (BusRd) BWL Bus Write Line (BusWr) BRIL Bus Read and Invalidate BIL Bus Invalidate Line MESI example Brehob -- Portions Falsafi, Hill, Hoe, Lipasti, Martin, Processor 1 Processor 2 Bus Processor Transaction Read A Read A Write A Write A Cache State Processor Transaction Read A Read A Write A Cache State M - Modified (dirty) E - Exclusive (clean unshared) only copy, not dirty S - Shared I - Invalid