Engineering Parallel Software with

Transcription

1 Engineering Parallel Software with Our Pattern Language Profeor Kurt Keutzer and Tim Matton and (Jike Chong), Ekaterina Gonina, Bor-Yiing Su and Michael Anderon, Bryan Catanzaro, Chao-Yue Lai, Mark Murphy, David Sheffield, Naryanan Sundaram, 1/77 Main Point of Previou Lecture Many approache to parallelizing oftware are not working Profile and improve Swap in a new parallel programming language Rely on a uper parallelizing compiler My own experience ha hown that a ound oftware architecture i the greatet ingle indicator of a oftware project ucce. Software mut be architected to achieve productivity, efficiency, and correctne SW architecture >> programming environment >> programming language >> compiler and debugger (>>hardware architecture) If we had undertood how to architect equential oftware, then parallelizing oftware would not have been uch a challenge Key to architecture (oftware or otherwie) i deign pattern and a pattern language At the highet level our pattern language ha: Eight tructural pattern Thirteen computational pattern Ye, we really believe arbitrarily complex parallel oftware can built jut from thee! 2/77 Kurt Keutzer 1

2 Outline Our Pattern Language for parallel programming Detailed example uing Our Pattern Language 3/77 Alexander Pattern Language Chritopher Alexander approach to (civil) architecture: "Each pattern decribe a problem which occur over and over again in our environment, and then decribe the core of the olution to that problem, in uch a way that you can ue thi olution a million time over, without ever doing it the ame way twice. Page x, A Pattern Language, Chritopher Alexander Alexander 253 (civil) architectural pattern range from the creation of citie (2. ditribution of town) to particular building problem (232. roof cap) A pattern language i an organized way of tackling an architectural problem uing pattern Main limitation: It about civil not oftware architecture!!! 4 4/77 Kurt Keutzer 2

3 A Pattern Language Pattern embody generalizable olution to recurrent problem Collection of individual pattern (e.g. Deign Pattern, Gamma, Helm, Johnon, Vliide) are great, but we need more help in the oftware development enterprie We would like a comprehenive pattern language which cover the entire proce of parallel oftware development and implementation Keutzer, Matton, the PALLAS group, and other have developed jut uch a pattern language Pattern language i overviewed in: Today we don t have time to go through all the pattern, but we will briefly decribe the tructure of OPL and how how it can be ued 5/77 Our Pattern Language: At a High Level Application Structural Pattern Computational Pattern Implementation Pattern Execution Pattern 6/77 Kurt Keutzer 3

4 Architecting Parallel Software Application Identify the Software Structure Pipe-and-Filter Agent-and-Repoitory Event-baed Bulk Synchronou MapReduce Layered Sytem Model-view controller Arbitrary Tak Graph Puppeteer Model-view-controller Identify the Key Computation Graph Algorithm Dynamic programming Dene/Spare Linear Algebra Un/Structured Grid Graphical Model Finite State Machine Backtrack Branch-and-Bound N-Body Method Circuit Spectral Method Monte-Carlo 7/77 Our Pattern Language 2010: Detail Structural Pattern Pipe-and-Filter Agent-and-Repoitory Proce-Control Event-Baed/Implicit- Invocation Puppeteer Model-View-Controller Iterative-Refinement Map-Reduce Layered-Sytem Arbitrary-Static-Tak-Graph Computational Pattern Graph-Algorithm Dynamic-Programming Dene-Linear-Algebra Spare-Linear-Algebra Untructured-Grid Structured-Grid Graphical-Model Finite-State-Machine Backtrack-Branch-and- Bound N-Body-Method Circuit Spectral-Method Monte-Carlo Tak-Parallelim Divide and Conquer Data-Parallelim Dicrete-Event Geometric-Decompoition Implementation Strategy Pattern Shared-Queue SPMD Fork/Join Loop-Par. Ditributed-Array Shared-map Data-Par/index-pace Actor Tak-Queue Shared-Data Partitioned Graph Program tructure Data tructure Parallel Execution Pattern MIMD SIMD Thread-Pool Tak-Graph Tranaction 8/77 Kurt Keutzer 4

5 Our Pattern Language: At a High Level Application Structural Pattern Computational Pattern Implementation Pattern Execution Pattern 9/77 The Algorithm Strategy Deign Space Start Organize By Flow of Data Organize By Tak Organize By Data Regular? Irregular? Linear? Recurive? Linear? Recurive? Dicrete Event Tak Parallelim Divide and Conquer Geometric Decompoition??? Tak-Parallelim Data-Parallelim Divide and Conquer Dicrete-Event Geometric-Decompoition 10/77 Kurt Keutzer 5

6 Our Pattern Language: At a High Level Application Structural Pattern Computational Pattern Implementation Pattern Execution Pattern Implementation Strategy Pattern SPMD Fork/Join Data-Par/index-pace Actor Program tructure Loop-Par. Tak-Queue Shared-Queue Shared-map Partitioned Graph Ditributed-Array Shared-Data Data tructure 11/77 Our Pattern Language: At a High Level Application Structural Pattern Computational Pattern Implementation Pattern Execution Pattern Parallel Execution Pattern MIMD SIMD Thread-Pool Tak-Graph Tranaction 12/77 Kurt Keutzer 6

7 Outline Our Pattern Language for parallel programming Detailed example uing Our Pattern Language 13/77 Speech Recognition Application Software Architecture uing Pattern Identify Structural Pattern Identify Computational Pattern Parallelization: (for each module) Algorithm trategy pattern Implementation trategy pattern Execution pattern Concluion Outline 14/77 Kurt Keutzer 7

8 Automatic Speech Recognition Key technology for enabling rich human-computer interaction Increaingly important for intelligent device without keyboard Interaction require low latency repone Only one of many component in exciting new real-time application Main contribution: A detailed analyi of the concurrency in large vocabulary continuou peech recognition (LVCSR) ummarized in the pattern language An implementation on GPU with 11x peedup over optimized C++ verion Achieving 3x better than real time performance with 50,000 word vocabulary 15/77 Continuou Speech Recognition Challenge: Recognizing word from a large vocabulary arranged in exponentially many poible permutation Inferring word boundarie from the context of neighboring word Hidden Markov Model (HMM) i the mot ucceful approach 16/77 Kurt Keutzer 8

9 CSR Sytem Structure Inference engine baed ytem Ued in Sphinx (CMU, USA), HTK (Cambridge, UK), and Juliu (CSRC, Japan) Modular and flexible etup Shown to be effective for Arabic, Englih, Japanee, and Mandarin 17/77 Recognition Proce Conider a equence of feature one at a time, for each feature: Start with mot-likely-equence ending with previou feature Correponding to a et of active tate Conider the likelihood of current feature in it context Compute mot-likely-equence ending with current feature Correponding to the next et of active tate Recognition i a proce of graph traveral Recognition Network 18/77 Kurt Keutzer 9

10 Recognition Network Feature from one frame Gauian Mixture Model for One Phone State Mixture Component Computing ditance to each mixture component Computing weighted um of all component HMM Acoutic Phone Model aa hh n Pronunciation Model... HOP hh aa p... ON aa n... POP p aa p... Bigram Language Model T P CAT HAT HO IN ON PO P THE Compiled HMM Recognition Network... CAT... HAT HOP IN... ON POP... THE... 19/77 Graph Traveral Characteritic Compiled HMM Recognition Network Linear Lexical Network WFST Recognition Network Operation on a graph i driven by the graph tructure Irregular graph tructure challenging to tatically load balancing Irregular tate acce pattern challenging to dynamically load balancing Multiple tate may hare the ame next tate write contention in arc evaluation Traveral produce memory accee that pan the entire graph poor patial locality Inner loop have high data acce to computation ratio limited by memory bandwidth 20/77 Kurt Keutzer 10

11 Parallel Platform Characteritic Multicore/manycore deign philoophy Multicore: Devote ignificant tranitor reource to ingle thread performance Core Manycore: Maximizing computation throughput at the expene of ingle thread performance Architecture Trend: Increaing vector unit width Increaing number of core per die Core Core Cach Cach e e Cac ch Cach e e Core Core Application Implication: Mut increae data acce regularity Mut optimize ynchronization cot Core Cach e Cach e Core 21/77 Outline Speech Recognition Application Software Architecture uing Pattern Identify Structural Pattern Identify Computational Pattern Parallelization: (for each module) Algorithm trategy pattern Implementation trategy pattern Execution pattern Concluion 22/77 Kurt Keutzer 11

12 OPL 2010 Application Structural Pattern Pipe-and-Filter Agent-and-Repoitory Proce-Control Event-Baed/Implicit- Invocation Puppeteer Model-View-Controller Iterative-Refinement Map-Reduce Layered-Sytem Arbitrary-Static-Tak-Graph Computational Pattern Graph-Algorithm Dynamic-Programming Dene-Linear-Algebra Spare-Linear-Algebra Untructured-Grid Structured-Grid Graphical-Model Finite-State-Machine Backtrack-Branch-and- Bound N-Body-Method Circuit Spectral-Method Monte-Carlo Tak-Parallelim Divide and Conquer Data-Parallelim Dicrete-Event Geometric-Decompoition Implementation Strategy Pattern Shared-Queue SPMD Fork/Join Loop-Par. Ditributed-Array Shared-map Data-Par/index-pace Actor Tak-Queue Shared-Data Partitioned Graph Program tructure Data tructure Parallel Execution Pattern MIMD SIMD Thread-Pool Tak-Graph Tranaction Concurrency Foundation contruct (not expreed a pattern) Thread creation/detruction Proce creation/detruction Meage-Paing Collective-Comm. Tranactional memory Point-To-Point-Sync. (mutual excluion) collective ync. (barrier) Memory ync/fence 23 23/77 Architecting Parallel Software Decompoe Tak Group tak Order Tak Decompoe Data Data haring Data acce Identify the Software Identify the Key Structure Computation 24/77 Kurt Keutzer 12

13 Inference Engine Architecture Intro Recognition i a proce of graph traveral Each time-tep we need to identify the likely tate in the recognition network given the obervation acoutic ignal From a the of active tate we want to compute the next et of active tate uing probabilitie of acoutic ymbol and tate tranition What Structural pattern i thi? 25/77 Key computation: HMM Inference Algorithm An intance of: Graphical Model Implemented with: Dynamic Programming Find the mot-likely equence of tate that produced the obervation Viterbi Algorithm t State 1 State 2 State 3 State 4 Ob 1 Ob 2 Ob 3 Ob 4 x x x x Legend: A State x An Obervation P( x t t ) m [t-1][ t-1 ] P( t t-1 ) m [t][ t ] Markov Condition: J. Chong, Y. Yi, A. Faria, N.R. Satih and K. Keutzer, Data-Parallel Large Vocabulary Continuou Speech Recognition on Graphic Proceor, Emerging Application and Manycore Arch. 2008, pp , June /77 Kurt Keutzer 13

14 Iterative Refinement Structural Pattern One iteration per time tep Identify the et of probable tate in the network given acoutic ignal given current active tate et Prune unlikely tate Repeat 27/77 Digging Deeper Active Set Computation Architecture In each iteration we need to: Compute obervation probabilitie of tranition from current tate Travere the likely non-epilon arc to reach the et of next active tate t Travere the likely epilon arc to reach the et of next active tate What Structural pattern i thi? 28/77 Kurt Keutzer 14

15 Digging Deeper Active Set Computation Architecture Phae 1 Obervation probability computation Graphtraveral Phae 2 In each iteration we need to: Compute obervation probabilitie of tranition from current tate Travere the likely non-epilon arc to reach the et of next active tate Travere the likely epilon arc to reach the et of next active tate What Structural pattern i thi? 29/77 Phae 1: Obervation Probability computation Architecture Obervation probabilitie are computed from Gauian Mixture Model Each Gauian probability in each mixture i independent Probability for one phone tate i the um of all Gauian time the mixture probability for that tate What Structural pattern i thi? Dan Klein CS288, Lecture 9 30/77 Kurt Keutzer 15

16 Map-Reduce Structural Pattern Map each mixture probability computation Reduce the reult accumulate the total probability for that tate Gauian Mixture Model for One Phone State Mixture Component 31/77 Phae 2: Graph Traveral Now that we know the tranition probabilitie from current et of tate, we need to compute the next et of active tate (follow the likely tranition) Each tranition i independent Multiple tranition might end in the ame tate The end reult need to be a et of mot probable tate from all tranition What Structural pattern i thi? 32/77 Kurt Keutzer 16

17 Map-Reduce Structural Pattern- again Map each mixture probability computation Reduce the reult accumulate the total probability for that tate 33/77 High Level Structure of Engine Inference Engine Active Set Computation Inference Engine Active Set Computation Phae 1: Obervation Probability Computation Phae 2: Graph Traveral Phae 1 Phae 2 34/77 Kurt Keutzer 17

18 Outline Speech Recognition Application Software Architecture uing Pattern Identify Structural Pattern Identify Computational Pattern Parallelization: (for each module) Algorithm trategy pattern Implementation trategy pattern Execution pattern Concluion 35/77 What about Computation? Active Set Computation: Phae 1: Compute obervation probability of tranition given current et of tate Phae 2: Travere arc to determine next et of mot likely active tate What Computational Pattern are thee? Computational Pattern Graph-Algorithm Dynamic-Programming Dene-Linear-Algebra Spare-Linear-Algebra Untructured-Grid Structured-Grid Graphical-Model Finite-State-Machine Backtrack-Branch-and- Bound N-Body-Method Circuit Spectral-Method Monte-Carlo 36/77 Kurt Keutzer 18

19 Outline Speech Recognition Application Software Architecture uing Pattern Identify Structural Pattern Identify Computational Pattern Parallelization: (for each module) Algorithm trategy pattern Implementation trategy pattern Execution pattern Concluion 37/77 Now to Parallelim Inference Engine Structural Pattern: Iterative Refinement Computational Pattern: Dynamic Programming Inference engine and Active Set computation i equential Let look at Phae 1 and Phae 2 What Parallel Algorithm Strategy can we ue? Tak-Parallelim Data-Parallelim Divide and Conquer Dicrete-Event Geometric-Decompoition 38/77 Kurt Keutzer 19

20 Now to Parallelim Inference Engine Phae 1: Obervation Probability Computation Structural: MapReduce Computational: Graphical Model Compute cluter Gauian Mixture Probabilitie for each tranition label Hint: Look at data dependencie (or lack there-of) Tak-Parallelim Divide and Conquer Data-Parallelim Dicrete-Event Geometric-Decompoition 39/77 Now to Parallelim Inference Engine Phae 1: Obervation Probability Computation Structural: MapReduce Computational: Graphical Model Compute cluter Gauian Mixture Probabilitie for each tranition label Map reduce necearily implie dataparallelim Tak-Parallelim Data-Parallelim Divide and Conquer Dicrete-Event Geometric-Decompoition 40/77 Kurt Keutzer 20

21 Obervation Probability Pattern Computation Structural Pattern Pipe-and-Filter Agent-and-Repoitory Proce-Control Event-Baed/Implicit- Invocation Puppeteer Model-View-Controller Iterative-Refinement Map-Reduce Layered-Sytem Arbitrary-Static-Tak-Graph Computational Pattern Graph-Algorithm Dynamic-Programming Dene-Linear-Algebra Spare-Linear-Algebra Untructured-Grid Structured-Grid Graphical-Model Finite-State-Machine Backtrack-Branch-and- Bound N-Body-Method Circuit Spectral-Method Monte-Carlo Tak-Parallelim Divide and Conquer Data-Parallelim Dicrete-Event Geometric-Decompoition Implementation Strategy Pattern Shared-Queue SPMD Fork/Join Loop-Par. Ditributed-Array Shared-map Data-Par/index-pace Actor Tak-Queue Shared-Data Partitioned Graph Program tructure Data tructure Parallel Execution Pattern MIMD SIMD Thread-Pool Tak-Graph Tranaction Implementation trategy: Data tructure? 41/77 Structural Pattern Pipe-and-Filter Agent-and-Repoitory Proce-Control Event-Baed/Implicit- Invocation Puppeteer Obervation Probability Pattern Decompoition Model-View-Controller Iterative-Refinement Map-Reduce Layered-Sytem Arbitrary-Static-Tak-Graph Computational Pattern Graph-Algorithm Dynamic-Programming Dene-Linear-Algebra Spare-Linear-Algebra Untructured-Grid Structured-Grid Graphical-Model Finite-State-Machine Backtrack-Branch-and- Bound N-Body-Method Circuit Spectral-Method Monte-Carlo Tak-Parallelim Divide and Conquer Data-Parallelim Dicrete-Event Geometric-Decompoition Implementation Strategy Pattern Shared-Queue SPMD Fork/Join Loop-Par. Ditributed-Array Shared-map Data-Par/index-pace Actor Tak-Queue Shared-Data Partitioned Graph Program tructure Data tructure Parallel Execution Pattern MIMD SIMD Thread-Pool Tak-Graph Tranaction Gauian Mixture Model i hared among all computation read only 42/77 Kurt Keutzer 21

22 Structural Pattern Pipe-and-Filter Agent-and-Repoitory Proce-Control Event-Baed/Implicit- Invocation Puppeteer Obervation Probability Pattern Decompoition Model-View-Controller Iterative-Refinement Map-Reduce Layered-Sytem Arbitrary-Static-Tak-Graph Computational Pattern Graph-Algorithm Dynamic-Programming Dene-Linear-Algebra Spare-Linear-Algebra Untructured-Grid Structured-Grid Graphical-Model Finite-State-Machine Backtrack-Branch-and- Bound N-Body-Method Circuit Spectral-Method Monte-Carlo Tak-Parallelim Divide and Conquer Data-Parallelim Dicrete-Event Geometric-Decompoition Implementation Strategy Pattern Shared-Queue SPMD Fork/Join Loop-Par. Ditributed-Array Shared-map Data-Par/index-pace Actor Tak-Queue Shared-Data Partitioned Graph Program tructure Data tructure Parallel Execution Pattern MIMD SIMD Thread-Pool Tak-Graph Tranaction Program tructure? 43/77 Structural Pattern Pipe-and-Filter Agent-and-Repoitory Proce-Control Event-Baed/Implicit- Invocation Puppeteer Obervation Probability Pattern Decompoition Model-View-Controller Iterative-Refinement Map-Reduce Layered-Sytem Arbitrary-Static-Tak-Graph Computational Pattern Graph-Algorithm Dynamic-Programming Dene-Linear-Algebra Spare-Linear-Algebra Untructured-Grid Structured-Grid Graphical-Model Finite-State-Machine Backtrack-Branch-and- Bound N-Body-Method Circuit Spectral-Method Monte-Carlo Tak-Parallelim Divide and Conquer Data-Parallelim Dicrete-Event Geometric-Decompoition Implementation Strategy Pattern Shared-Queue SPMD Fork/Join Loop-Par. Ditributed-Array Shared-map Data-Par/index-pace Actor Tak-Queue Shared-Data Partitioned Graph Program tructure Data tructure Parallel Execution Pattern MIMD SIMD Thread-Pool Tak-Graph Tranaction Data parallel a ingle intruction tream i applied to multiple data element 44/77 Kurt Keutzer 22

23 Obervation Probability Pattern Decompoition Structural Pattern Pipe-and-Filter Agent-and-Repoitory Proce-Control Event-Baed/Implicit- Invocation Puppeteer Model-View-Controller Iterative-Refinement Map-Reduce Layered-Sytem Arbitrary-Static-Tak-Graph Computational Pattern Graph-Algorithm Dynamic-Programming Dene-Linear-Algebra Spare-Linear-Algebra Untructured-Grid Structured-Grid Graphical-Model Finite-State-Machine Backtrack-Branch-and- Bound N-Body-Method Circuit Spectral-Method Monte-Carlo Tak-Parallelim Divide and Conquer Data-Parallelim Dicrete-Event Geometric-Decompoition Implementation Strategy Pattern Shared-Queue SPMD Fork/Join Loop-Par. Ditributed-Array Shared-map Data-Par/index-pace Actor Tak-Queue Shared-Data Partitioned Graph Program tructure Data tructure Parallel Execution Pattern MIMD SIMD Thread-Pool Tak-Graph Tranaction Execution? a ingle intruction tream i applied to multiple data element 45/77 Obervation Probability Pattern Decompoition Structural Pattern Pipe-and-Filter Agent-and-Repoitory Proce-Control Event-Baed/Implicit- Invocation Puppeteer Model-View-Controller Iterative-Refinement Map-Reduce Layered-Sytem Arbitrary-Static-Tak-Graph Computational Pattern Graph-Algorithm Dynamic-Programming Dene-Linear-Algebra Spare-Linear-Algebra Untructured-Grid Structured-Grid Graphical-Model Finite-State-Machine Backtrack-Branch-and- Bound N-Body-Method Circuit Spectral-Method Monte-Carlo Tak-Parallelim Divide and Conquer Data-Parallelim Dicrete-Event Geometric-Decompoition Implementation Strategy Pattern Shared-Queue SPMD Fork/Join Loop-Par. Ditributed-Array Shared-map Data-Par/index-pace Actor Tak-Queue Shared-Data Partitioned Graph Program tructure Data tructure Parallel Execution Pattern MIMD SIMD Thread-Pool Tak-Graph Tranaction SIMD 46/77 Kurt Keutzer 23

24 Now to Parallelim Pt2 Inference Engine Phae 2: Graph Traveral Structural: MapReduce Computational: Graph algorithm/graph traveral The recognition network i a finite tate tranducer, repreented a a weighted and labeled graph Decoding on thi graph i Breadth- Firt Traveral What Parallel Algorithmic Strategy can we ue? Hint: What are the operand? What are the dependence? Tak-Parallelim Divide and Conquer Data-Parallelim Dicrete-Event Geometric-Decompoition 47/77 Now to Parallelim Inference Engine Phae 2: Graph Traveral Structural: MapReduce Computational: Graph Traveral The recognition network i a finite tate tranducer, repreented a a weighted and labeled graph Decoding on thi graph i Breadth- Firt Traveral Data Parallelim! Each next tate computation can be computed independently. Tak-Parallelim Divide and Conquer Data-Parallelim Dicrete-Event Geometric-Decompoition 48/77 Kurt Keutzer 24

25 Active Set Computation Structural Pattern Pipe-and-Filter Agent-and-Repoitory Proce-Control Event-Baed/Implicit- Invocation Puppeteer Model-View-Controller Iterative-Refinement Map-Reduce Layered-Sytem Arbitrary-Static-Tak-Graph Computational Pattern Graph-Algorithm Dynamic-Programming Dene-Linear-Algebra Spare-Linear-Algebra Untructured-Grid Structured-Grid Graphical-Model Finite-State-Machine Backtrack-Branch-and- Bound N-Body-Method Circuit Spectral-Method Monte-Carlo Tak-Parallelim Divide and Conquer Data-Parallelim Dicrete-Event Geometric-Decompoition Implementation Strategy Pattern Shared-Queue SPMD Fork/Join Loop-Par. Ditributed-Array Shared-map Data-Par/index-pace Actor Tak-Queue Shared-Data Partitioned Graph Program tructure Data tructure Parallel Execution Pattern MIMD SIMD Thread-Pool Tak-Graph Tranaction SIMD 49/77 Shared Queue Implementation In each iteration: Arc tranition from the current et of tate are travered to contruct a et of next active tate Each thread enqueue one or more next active tate to a global queue of next active tate Thi approach caue ignificant contention a the queue head pointer In our application - thouand of thread are acceing one memory location Caue erialization of thread memory accee Limit calability 50/77 Kurt Keutzer 25

26 Shared Queue Implementation Solution to the queue head pointer contention -> ditributed queue 1. Each thread in a thread block write to a local queue of next active tate 2. Local queue are merged into a global queue Contention on global queue pointer i reduced from #thread to #block Significantly improve calability 51/77 Speech Reference Implementation Jike Chong, Ekaterina Gonina, Youngmin Yi, Kurt Keutzer, A Fully Data Parallel WFST-baed Large Vocabulary Continuou Speech Recognition on a Graphic Proceing Unit, Proceeding of the 10th Annual Conference of the International Speech Communication Aociation (InterSpeech), page , September, W R W R W R Manycore GPU Data W R R W R Control Phae 0 Iteration Control Prepare ActiveSet Phae 1 Compute Obervation Probability Phae 2 For each active arc: Compute arc tranition probability Copy reult back to CPU Control Read File Initialize data tructure Collect Backtrack Info Backtrack Output Reult CPU Data W R W R Kiun You, Jike Chong, Youngmin Yi, Ekaterina Gonina, Chritopher Hughe, Yen-Kuang Chen, Wonyong Sung, Kurt Keutzer, Parallel Scalability in Speech Recognition: Inference engine in large vocabulary continuou peech recognition, IEEE Signal Proceing Magazine, vol. 26, no. 6, pp , November Jike Chong, Ekaterina Gonina, Kiun You, Kurt Keutzer, Scalable Parallelization of Automatic Speech Recognition, Invited book chapter in Scaling Up Machine Learning, an upcoming 2010 Cambridge Univerity Pre book. 52/77 Kurt Keutzer 26

27 Summary A good architect need to undertand: Structural pattern Computational pattern Refinement through Our Pattern Language Graph algorithm and graphical model are critical to many application Graph algorithm are epecially difficult to parallelize and library upport i inadequate There will be at leat a decade of hand-crafted olution We achieved good reult on parallelizing large-vocabulary automatic peech recognition 53/77 Extra 54/77 Kurt Keutzer 27

28 Example: Breadth Firt Search Breadth firt earch Ditributed Graph Ditributed Queue Ditributed Viitor Ditributed Property Map Algorithm Viitor 55/77 Example: Ditributed BFS Ditributed Graph (a) Ditributed Graph (b) Ditributed adjacency lit repreentation 56/77 Kurt Keutzer 28

29 Example: Ditributed BFS Ditributed Queue pop() from local queue puh() end meage to the vertex owner empty() exhaut local queue and ynchronize with other proceor to determine termination condition Meage are only received after all proceor have completed operation at one level (b) Ditributed adjacency lit repreentation 57 57/77 Example: Ditributed BFS Ditributed Viitor Owner-compute cheme, o ditributed verion i the ame a equential viitor Ditributed Property Map Local Property Map Store local propertie for local vertice and edge Ghot Cell Store propertie for ghot cell Proce Group Communication medium Data Race Reolver Decide among variou put() meage ent to the Ditributed Property Map (a) Ditributed Graph (b) Ditributed adjacency lit repreentation 58 58/77 Kurt Keutzer 29

30 Performance achieved 128 node, 2GHz AMD Opteron, 4GB RAM each, connected over Infiniband, one core active per node BFS: 100k vertice and ~15M edge (C) Crauer et al (E) Eager heuritic BFS: 1M vertice and ~15M edge Dijktra Algorithm: 100k vertice and ~15M edge 59 59/77 Dicuion Graph traveral algorithm characteritic: 1. Input data-driven computation 2. Untructured problem 3. Poor data locality 4. High data acce to computation ratio High demand for low memory latency and poor data locality make it challenging for PE without fine-grain multiproceing Pointer chaing mainly involve integer operation Niagara type platform eem uitable for thi application domain What about GPGPU? What how well would our cheduling algorithm map to Parallel BGL? 60 60/77 Kurt Keutzer 30