Accelerating Pattern Searches with Hardware. Kevin Angstadt CS 6354: Graduate Architecture 7. April 2016

Size: px

Start display at page:

Download "Accelerating Pattern Searches with Hardware. Kevin Angstadt CS 6354: Graduate Architecture 7. April 2016"

Silas Gallagher
6 years ago
Views:

1 Accelerating Pattern Searches with Hardware Kevin Angstadt CS 6354: raduate Architecture 7. April 2016

2 Who am I? Work with Wes Weimer and Kevin Skadron PL + Architecture Programming models for non-traditional computation Software resiliency in autonomous vehicles Interdisciplinary research is fun! 2

3 We re producing lots of data! 3

together detect possible outside Identify consumer attacks

4 What is the common theme? Locate the most Pattern Search Problems probable location for a DNA fragment Find in products the that are human genome most commonly Sniff CP/IP packets to purchased together detect possible outside Identify consumer attacks sentiment based off of Search for Higgs events social media posts based off on paths of subatomic particles 4

5 Parallel searches Key = Active Searches arget Pattern ACA C A C C A A C Incoming Data CCA 5

6 How do we define these patterns? 6

7 What is a Finite Automaton? A finite automaton consists of An input alphabet, A set of states, Q A starting state, q 0 A set of accepting states, F Q A transition function, transition( input, q i ) = q j 7

8 Finite Automaton Example Start 1 0 Check: 1110 Check:

9 Optimization: Non-determinism Can be in multiple states at one time Can start in more than one state ransition function returns a set of states (ε-transitions: we ll ignore these, but you ll see them in literature) Why? More compact design! Exponentially more compact than a (D)FA 9

10 What does this find? Start Start Dd o n u t u g h 10

11 Let s process a DFA/NFA Why is a CPU bad at doing this? Storage Complexity and per Symbol Processing Complexity of an n-state NFA and Equivalent DFA Processing Bandwidth NFA Processing Complexity O(n 2 ) Storage Cost O(n) Memory Bandwidth DFA O(1) O( n ) 11

12 Architectural Underpinnings 12

13 What we need Compact design of a NFA Ability to update all transitions in a single time step We can achieve this by throwing away the conventional von Neumann architecture 13

14 Micron s Automata Processor 14 Figure courtesy of Micron

15 Exploiting DRAM Row Address (Memory Location) Row Access results in one word being retrieved from memory 15

16 Using Bit-Parallelism with NFAs Bit-parallel closure: given a set of states, what is the set of reachable states? Symbol closure: given an input symbol, what states accept this symbol? ransition function: intersection (bitwise AND) of these two closures 16

17 Bit-Parallel Execution Input Stream: 1011 Start Start Start q q 1 q q 2 q 4 Bit-parallel closure({start}): {q 0, q 1, q 2 } Symbol closure(1): {q 0, q 2, q 4 } Bit & Symbol: {q 0, q 2 } 17

18 Bit-Parallel Execution Input Stream: 1011 Start Start Start q q 1 q q 2 q 4 Bit-parallel closure({q 0, q 2 }): {q 0, q 1, q 2, q 4 } Symbol closure(0): {q 0, q 1, q 3 } Bit & Symbol: {q 0, q 1 } 18

19 Bit-Parallel Execution Input Stream: 1011 Start Start Start q q 1 q q 2 q 4 Bit-parallel closure({q 0, q 2 }): {q 0, q 1, q 2, q 3 } Symbol closure(1): {q 0, q 2, q 4 } Bit & Symbol: {q 0, q 2, q 4 } 19

20 Bit-Parallel Execution Input Stream: 1011 Start Start Start q q 1 q q 2 q 4 Bit-parallel closure({q 0, q 2 }): {q 0, q 1, q 2, q 4 } Symbol closure(1): {q 0, q 2, q 4 } Bit & Symbol: {q 0, q 4 } REPOR! 20

21 Architectural Design Memory Array (Symbol Closure) 256 Rows (i.e.256 Input Characters) 49,152 Columns (i.e.49,152 States) Routing Matrix (Bit-Parallel Closure) SaturatingCounters Boolean ates 21

22 he AP at a High Level Row Address (Input Symbol) Automata Processor Routing Matrix Row Access results in 49,152 match & route operations 22

23 Executing NFA in DRAM Columns in DRAM store SE labels (Each SE is a single column) Reconfigurable routing matrix connects the SEs Input: Drivesa Row Columns with 1 : SEs thataccept input symbol & Active = States Active States for Next Clock Cycle 23

24 Why Hierarchical Routing? Half-Core Half-Core Column Column Column Column Block Block Block Block Block Block Block Block Row Row Row Row Row Row Row Row Row Row Row Row Row Row Row Row o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o 24

25 roup of wo Contain two State Machine Elements (SME) Hardware embodiment of SE Memory Cells Detection Cell Common output Loopback and Chaining 25

26 3-to-1 Mux Detection Cell Configurable Switching Elements From Input Decoder 26

27 Row Contain 8 os 1 Special Element (Boolean or Counter) Match Element (Connected to two os) Row Routing Lines SMEs exhibit parity (can only connect to half of input routing lines) 27

28 SPECIAL/OUPU EXERNAL/INPU INERNAL/OUPU 28

29 29

30 RAPID Programming of Pattern-Recognition Processors Kevin Angstadt Westley Weimer Kevin Skadron Department of Computer Science University of Virginia {angstadt, weimer, 7. April 2016

31 Programming Workflow Front End Language Synthesis, Placement, and Routing Compiled Binary Source: 31

language bindings for generation Support for a list of regular

32 Current Programming Models Automata Network Markup Language Directly specify homogeneous NFA design High-level programming language bindings for generation Support for a list of regular expressions Support for PCRE modifiers Compiled directly to binary 32

33 Programming Challenges ANML development akin to assembly programming Requires knowledge of automata theory and hardware properties edious and error-prone development process Regular expressions challenging to implement Often exhaustive enumerations Similarly error-prone 33

34 Programming Challenges Implement single instance of a problem Each instance of a problem requires a brand new design Need for meta-programs to generate final design Current programming models place unnecessary burden on developer 34

35 A researcher should spend his or her time designing an algorithm to find the important data, not building a machine that will obey said algorithm. 35

36 Parallel Searches: oals Fast processing Specialized Hardware Concise, maintainable representation Efficient compilation High throughput Low compilation time RAPID Programming Language 36

37 RAPID Programming Pattern-Based Data Analysis Automata Processor Current Programming Models RAPID Language Overview AP Code eneration and Optimizations Evaluation 37

38 RAPID at a lance Provides concise, maintainable, and efficient representations for pattern-identification algorithms Conventional, C-style language with domain-specific parallel control structures Excels in applications where patterns are best represented as a combination of text and computation Compilation strategy balances synthesis time with device utilization 38

39 Program Structure Macro Basic unit of computation Sequential control flow Boolean expressions as statements for terminating threads of computation Network High-level pattern matching Parallel control flow Parameters to set run-time values macro foo ( ) { } macro bar ( ) { } macro baz ( ) { } macro qux ( ) { } network ( ) { } 39

40 Program Structure Network Macros 40

41 Program Structure macro foo ( ) { } Macro Macro Macro macro bar ( ) { } macro baz ( ) { } M M M Macro M M M Macro M M M Macro macro qux ( ) { } Macro Macro Macro Network network ( ) { } 41

42 Data in RAPID Input data stream as special function Stream of characters input() Calls to input() are synchronized across all active macros All active macros receive the same input character 42

43 Counting and Reporting Counter: Abstract representation of saturating up counters Count and Reset operations Can compare against threshold RAPID programs can report riggers creation of report event Captures offset of input stream and current macro 43

Parallel Control Structures Concise specification of multiple, simultaneous comparisons against a single data stream Support MISD computational model Static and dynamic thread

44 Parallel Control Structures Concise specification of multiple, simultaneous comparisons against a single data stream Support MISD computational model Static and dynamic thread spawning for massive parallelism support Explicit support for sliding window Pattern 44

45 Parallel Control Structures Sequential Structure if else foreach while Parallel Structure either orelse some whenever 45

46 Either/Orelse Statements either { hamming_distance(s,d); //hamming distance y == input(); //next input is y report; //report candidate }orelse{ while( y!= input()); //consume until y } Perform parallel exploration of input data Static number of parallel operations 46

47 Some Statements } network (String[] comparisons) { some(string s : comparisons) hamming_distance(s,5); } Parallel exploration may depend on candidate patterns Iterates over items, dynamically spawn computation 47

48 Whenever Statements whenever( ALL_INPU == input() ) { foreach(char c : "rapid") c==input(); report; } Body triggered whenever guard becomes true ALL_INPU: any symbol in the input stream 48

49 Example RAPID Program Association Rule Mining Identify items from a database that frequently occur together 49

$count(); } network (String[] set) { some(string s : set) { Counter cnt; whenever(sar_of_inpu == input())$

50 Example RAPID Program If all symbols in item set match, increment counter Spawn parallel computation for each item set Sliding window search calls frequent on every input rigger report if threshold reached macro frequent (String set, Counter cnt) { foreach(char c : set) { while(input()!= c); } cnt.count(); } network (String[] set) { some(string s : set) { Counter cnt; whenever(sar_of_inpu == input()) frequent(s,cnt); if (cnt > 128) report; } } 50

51 RAPID Programming Pattern-Based Data Analysis Automata Processor Current Programming Models RAPID Language Overview AP Code eneration and Optimizations Evaluation 51

52 System Overview RAPID Program Annotations Input RAPID Compiler ANML apcompile Driver Code Output AP Binary 52

transformation of RAPID program Input Stream àses Counters à 1

53 Code eneration RAPID Program macro foo ( ) { } macro bar ( ) { } macro baz ( ) { } macro qux ( ) { } network ( ) { } Recursive transformation of RAPID program Input Stream àses Counters à 1 or more physical counter(s) Similar to RegEx ànfa transformation 53

54 Parallel searches Key = Active Searches arget Pattern ACA C A Maximize number of parallel active searches Incoming Data CCA 54

55 Optimizing Compilation RAPID programs are often repetitive Extract repeated design, and compile once Load dynamically at runtime and set exact values (tessellation) Block 55

56 Auto-uning Optimization Block Block Block 56

57 RAPID Programming Pattern-Based Data Analysis Automata Processor Current Programming Models RAPID Language Overview AP Code eneration and Optimizations Evaluation 57

58 Reminder: oals Fast processing Concise, maintainable representation Efficient compilation High throughput Low compilation time 58

59 Description of Benchmarks Benchmark Description eneration Method ARM AssociationRule Mining Meta Program Brill Brill Part of Speech agging Meta Program Exact Exact DNA Alignment ANML appy DNA Alignmentwith aps ANML MOOMAA Planted Motif Search ANML 59

60 RAPID Lines of Code ARM Brill Exact appy MOOMAA Lines of Code Handcrafted RAPID 60

61 RAPID is Maintainable Percent of Code Changed 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% ask: Convert Hamming distance comparison of length 5 to length 12 0% Handcrafted RAPID 61

62 Parallel searches Key = Active Searches C A Incoming Data arget Pattern Maximize number of parallel active searches ACA Reduce SE usage! CCA 62

63 enerated SEs ARM Brill Exact appy MOOMAA Number of SEs Handcrafted RAPID 63

64 Compilation ime ARM* Brill Exact* appy* MOOMAA* Handcrafted RAPID ime (seconds) * RAPID essellation 64

65 Conclusions RAPID is a high-level language for patternsearch algorithms hree domain-specific parallel control structures, and suitable data representations Accelerate using the Automata Processor RAPID programs are concise, maintainable, and efficient 65

RAPID Programming of Pattern-Recognition Processors

RAPID Programming of Pattern-Recognition Processors Kevin Angstadt Westley Weimer Kevin Skadron Department of Computer Science University of Virginia {angstadt, weimer, skadron}@cs.virginia.edu 6. April