Hardware-Sensitive Database Operations

Transcription

1 D B Hardware-Sensitive Database Operations Advanced Topics in Bala Gurumurthy Otto-von-Guericke University Magdeburg Summer 2018

2 Credits Parts of this lecture are based on content by Jens Teubner from TU Dortmund David Broneske. Bala Gurumurthy Hardware-Sensitive Database Operations 0

3 Agenda Motivation Pipelining in CPUs Hazards Loop Unrolling Branch-Free Code Recap Vectorization SIMD: Vectorized Execution SIMD for Database Tasks Bala Gurumurthy Hardware-Sensitive Database Operations 1

4 Motivation Why do we need to care about hardware in a DBMS? Bala Gurumurthy Hardware-Sensitive Database Operations 2

5 Single-thread Performance Throughput Performance Modern Application Performance D B A NEW ERA OF PROCESSOR PERFORMANCE Single-Core Era Multi-Core Era Heterogeneous Systems Era Enabled by: Moore s Law Voltage Scaling Constrained by: Power Complexity Enabled by: Moore s Law SMP architecture Constrained by: Power Parallel SW Scalability Enabled by: Abundant data parallelism Power efficient GPUs Temporarily Constrained by: Programming models Comm.overhead Assembly C/C++ Java pthreads OpenMP / TBB Shader CUDA OpenCL!!!? we are here we are here we are here Time Time (# of processors) Time (Data-parallel exploitation) 9 The Programmer s Guide to the APU Galaxy June 2011 Bala Gurumurthy Hardware-Sensitive Database Operations 3

6 Database Server of the Future CPU APU disk I/O controller memory bus main memory PCIe bus GPU FPGA MIC adapted from [Saecker Markl, 2013]. Bala Gurumurthy Hardware-Sensitive Database Operations 4

7 Database Server of the Future CPU disk CPU I/O controller PCIe bus memory bus APU main memory GPU Central Processing Unit Directly connected to main memory Theoretical bwidth of GB/s Cache hierarchy (up to 3 levels) FPGA MIC adapted from [Saecker Markl, 2013]. Bala Gurumurthy Hardware-Sensitive Database Operations 5

8 Database Server of the Future CPU Capabilities Several physical cores (order of 10) Instruction pipelining Split instruction into subtasks Process subtasks in parallel Branch prediction Guess wether an if-statement is evaluated to true or false. Out-of-order execution If one instruction has to wait for resources, process another instruction meanwhile. Vector processing units (SIMD) If one instruction has to be executed on several subsequent data items, use a SIMD register to do the execution in parallel. Bala Gurumurthy Hardware-Sensitive Database Operations 6

9 Database Server of the Future GPU disk FPGA CPU I/O controller PCIe bus memory bus APU main memory GPU Graphics Processing Unit Connected via PCIe Bus with 4-32 GB/s bwidth Data has to be in GPU-RAM to be processed! Theoretical GPU-RAM bwidth in the order of 100 GB/s MIC adapted from [Saecker Markl, 2013]. Bala Gurumurthy Hardware-Sensitive Database Operations 7

10 Database Server of the Future GPU Capabilities Many cores (in the order of ) Several execution units (ALUs) per core (ca ) working in SIMD-fashion Making 100s to 1000s of stream processors Extremely high parallelism No branch prediction One instruction executed per core Instruction serialization on diverging branches Mismatch of PCIe bwidth GPU memory bwidth Prefer compute-bound problems Bala Gurumurthy Hardware-Sensitive Database Operations 8

11 Database Server of the Future MIC disk FPGA CPU I/O controller PCIe bus memory bus APU main memory GPU MIC adapted from [Saecker Markl, 2013]. Many Integrated Core Architecture Inspired by GPUs many Intel cores packed on one accelerator card Connected via PCIe Bus Data has to be in device RAM to be processed! Theoretical memory bwidth of GB/s Cache hierarchy per core (3 levels) Bala Gurumurthy Hardware-Sensitive Database Operations 9

12 Database Server of the Future MIC Capabilities Intel Xeon cores Each of them has the same capabilities as a normal CPU! High parallelism Wide vector processing units (SIMD) (512 bit) Highest data parallelism capabilities Mismatch of PCIe bwidth Xeon Phi memory bwidth Prefer compute-bound problems Bala Gurumurthy Hardware-Sensitive Database Operations 10

13 Database Server of the Future APU disk CPU I/O controller PCIe bus memory bus APU main memory GPU Accelerated Processing Unit Integration of GPU in CPU Fast access to RAM (same as CPU) CPU cache hierarchy FPGA MIC adapted from [Saecker Markl, 2013]. Bala Gurumurthy Hardware-Sensitive Database Operations 11

14 Database Server of the Future APU Capabilities Normal CPU with integrated GPU Execute control-flow intensive tasks on CPU-part Execute compute-intensive tasks on GPU-part Limited die space limited amount of cores Each part is behind its dedicated counterpart Bala Gurumurthy Hardware-Sensitive Database Operations 12

15 Database Server of the Future FPGA disk CPU I/O controller PCIe bus memory bus APU main memory GPU Field-Programmable Gate Array Idea of reprogrammable hardware Stream processor Little device memory Connected via PCIe Bus FPGA MIC adapted from [Saecker Markl, 2013]. Bala Gurumurthy Hardware-Sensitive Database Operations 13

16 Database Server of the Future FPGA Capabilities Programmable look-up tables (LUTs) interconnects Any logic function programmable in hardware Functional units are freely combinable Efficient instruction pipelining Lower clock frequency than CPUs But can compete with them! Bala Gurumurthy Hardware-Sensitive Database Operations 14

17 Hardware Capabilities Processing Device CPU APU MIC GPU FPGA Processing devices their processing capabilities. Parallelization Properties Pipeline Data Parallelism Parallelism Memory Scaling Memory Capacity Memory Bwidth Bala Gurumurthy Hardware-Sensitive Database Operations 15

18 Hardware Capabilities Processing devices their processing capabilities. Parallelization Properties Memory Scaling Processing Device Pipeline Parallelism Data Parallelism Memory Capacity Memory Bwidth CPU + / APU + / MIC + / / GPU + / FPGA + + / + / Legend: + + = excellent, + = good, = poor [Broneske et al., 2014] Bala Gurumurthy Hardware-Sensitive Database Operations 15

19 Programming for Different Processing Devices Hardware-Oblivious Hardware-Sensitive main code base main code base hardware-oblivious operators parallel programming library device-specific operators device-specific operators compiler binary compiler compiler driver driver driver binary binary CPU GPU1 GPU2 CPU GPU adapted from [Heimel et al., 2013] Bala Gurumurthy Hardware-Sensitive Database Operations 16

20 Programming for Different Processing Devices Hardware-Oblivious Hardware-Oblivious Hardware-Sensitive Programming main code base hardware-oblivious operators parallel programming library Define eachmain operator code basein abstract language (e.g., OpenCL) device-specific operators device-specific operators Parallel programming library One driver per device driver compiler binary driver driver CPU GPU1 GPU2 E.g., Ocelot [Heimel et al., 2013] Properties compiler binary CPU performance compiler binary + Separation of concerns Optimized, but not best GPU adapted from [Heimel et al., 2013] Bala Gurumurthy Hardware-Sensitive Database Operations 16

21 adapted from [Heimel et al., 2013] 1 Bala Gurumurthy Hardware-Sensitive Database Operations 16 D B Programming for Different Processing Devices Hardware-Sensitive Hardware-Oblivious main code base Each operator optimized for a given hardware-oblivious device operators Noparallel abstraction programming layer library between operator binary E.g., CoGaDB 1 compiler Hardware-Sensitive device-specific operators compiler main code base device-specific operators compiler Properties + Best performance driver driver binary driver High implementation effort CPU GPU1 GPU2 binary CPU binary GPU

22 are about: Efficiency Efficiency Efficiency Bala Gurumurthy Hardware-Sensitive Database Operations 17

23 are about: Efficiency Efficiency Efficiency What are specific optimizations for CPUs? Bala Gurumurthy Hardware-Sensitive Database Operations 17

24 are about: Efficiency Efficiency Efficiency What are specific optimizations for CPUs? What can we do about the code? Bala Gurumurthy Hardware-Sensitive Database Operations 17

25 Pipelining in CPUs Increasing Instructions per Cycle Bala Gurumurthy Hardware-Sensitive Database Operations 18

26 Pipelining in CPUs Pipelining is a CPU implementation technique whereby multiple instructions are overlapped in execution. Break CPU instructions into smaller units pipeline. E.g., classical five-stage pipeline for RISC: clock instr. i IF ID EX MEM WB instr. i+1 IF ID EX MEM WB instr. i+2 IF ID EX MEM WB parallel execution Bala Gurumurthy Hardware-Sensitive Database Operations 19

27 Pipelining in CPUs Pipelining is a CPU implementation technique whereby multiple instructions are overlapped in execution. Break CPU instructions into smaller units pipeline. E.g., classical five-stage pipeline for RISC: clock instr. i IF ID EX MEM WB instr. i+1 instr. i+2 Instruction Fetch Cycle IF ID EX MEM WB Fetch OP code at program counter (PC) Increase PC IF ID EX MEM WB parallel execution Bala Gurumurthy Hardware-Sensitive Database Operations 19

28 Pipelining in CPUs Pipelining is a CPU implementation technique whereby multiple instructions are overlapped in execution. Break CPU instructions into smaller units pipeline. E.g., classical five-stage pipeline for RISC: clock instr. i IF ID EX MEM WB instr. i+1 instr. i+2 Instruction Decode / Register Fetch Cycle IF ID EX MEM WB Decode instruction / read registers Branch detection IF ID EX MEM WB parallel execution Bala Gurumurthy Hardware-Sensitive Database Operations 19

29 Pipelining in CPUs Pipelining is a CPU implementation technique whereby multiple instructions are overlapped in execution. Break CPU instructions into smaller units pipeline. E.g., classical five-stage pipeline for RISC: clock instr. i IF ID EX MEM WB instr. i+1 instr. i+2 Execution / Effective Address Cycle Execute operation IF ID EX MEM WB IF ID EX MEM WB Memory reference Register-register ALU instruction parallel Register-immediate ALU execution instruction Bala Gurumurthy Hardware-Sensitive Database Operations 19

30 Pipelining in CPUs Pipelining is a CPU implementation technique whereby multiple instructions are overlapped in execution. Break CPU instructions into smaller units pipeline. E.g., classical five-stage pipeline for RISC: clock instr. i IF ID EX MEM WB instr. i+1 instr. i+2 Memory Access IF ID EX MEM WB Load instruction read data into register IF ID EX MEM WB Store instruction write data into memory parallel execution Bala Gurumurthy Hardware-Sensitive Database Operations 19

31 Pipelining in CPUs Pipelining is a CPU implementation technique whereby multiple instructions are overlapped in execution. Break CPU instructions into smaller units pipeline. E.g., classical five-stage pipeline for RISC: clock instr. i IF ID EX MEM WB instr. i+1 instr. i+2 Write-Back Cycle IF ID EX MEM WB Register-Register ALU instruction or Load instruction: Write result in register IF ID EX MEM WB parallel execution Bala Gurumurthy Hardware-Sensitive Database Operations 19

32 Pipelining in CPUs Pipelining is a CPU implementation technique whereby multiple instructions are overlapped in execution. Break CPU instructions into smaller units pipeline. E.g., classical five-stage pipeline for RISC: clock instr. i IF ID EX MEM WB instr. i+1 IF ID EX MEM WB instr. i+2 IF ID EX MEM WB parallel execution Bala Gurumurthy Hardware-Sensitive Database Operations 19

33 Pipelining in CPUs Pipelining is a CPU implementation technique whereby multiple instructions are overlapped in execution. Break CPU instructions into smaller units pipeline. E.g., classical five-stage pipeline for RISC: clock instr. i IF ID EX MEM WB instr. i+1 IF ID EX MEM WB instr. i+2 IF ID EX MEM WB parallel execution Bala Gurumurthy Hardware-Sensitive Database Operations 19

34 Pipelining in CPUs Pipelining is a CPU implementation technique whereby multiple instructions are overlapped in execution. Break CPU instructions into smaller units pipeline. E.g., classical five-stage pipeline for RISC: clock instr. i IF ID EX MEM WB instr. i+1 IF ID EX MEM WB instr. i+2 IF ID EX MEM WB parallel execution Bala Gurumurthy Hardware-Sensitive Database Operations 19

35 Pipelining in CPUs Pipelining is a CPU implementation technique whereby multiple instructions are overlapped in execution. Break CPU instructions into smaller units pipeline. E.g., classical five-stage pipeline for RISC: clock instr. i IF ID EX MEM WB instr. i+1 IF ID EX MEM WB instr. i+2 IF ID EX MEM WB parallel execution Bala Gurumurthy Hardware-Sensitive Database Operations 19

36 Pipelining in CPUs Ideally, a k-stage pipeline improves performance by a factor of k. Slowest (sub-)instruction determines clock frequency. Ideally, break instructions into k equi-length parts. Issue one instruction per clock cycle (IPC = 1). Example: Intel Pentium 4: 31+ pipeline stages Intel Core i7 (Nehalem): 14 pipeline stages Bala Gurumurthy Hardware-Sensitive Database Operations 20

37 Pipelining in CPUs Hazards Making Simple Things Complicated Bala Gurumurthy Hardware-Sensitive Database Operations 21

38 Hazards The effectiveness of pipelining is hindered by hazards. Structural Hazard Different pipeline stages need same functional unit (resource conflict; e.g., memory access instruction fetch) Data Hazard Result of one instruction not ready before access by later instruction. Control Hazard Arises from branches or other instructions that modify PC ( data hazard on PC register ). Hazards lead to pipeline stalls that decrease IPC. Bala Gurumurthy Hardware-Sensitive Database Operations 22

39 Structural Hazards A structural hazard will occur if a CPU has only one memory access unit instruction fetch memory access are scheduled in the same cycle clock instr. i instr. i+1 instr. i+2 instr. i+3 IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB IF ID IF EX ID MEM EX MEM WB WB Bala Gurumurthy Hardware-Sensitive Database Operations 23

40 Structural Hazards A structural hazard will occur if a CPU has only one memory access unit instruction fetch memory access are scheduled in the same cycle clock instr. i instr. i+1 instr. i+2 instr. i+3 IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB stall IF ID IF EX ID MEM EX MEM WB WB Resolution: Provision hardware accordingly (e.g., separate fetch units) Schedule instructions (at compile- or runtime) Bala Gurumurthy Hardware-Sensitive Database Operations 23

41 Data Hazards LD DSUB AND OR XOR R1, 0(R2) R4, R1, R5 R6, R1, R7 R8, R1, R9 R10, R1, R11 Instructions read R1 before it was written by DADD (stage WB writes register results). Would cause incorrect execution result. LD DSUB AND OR XOR R1, 0(R2) R4, R1, R5 R6, R1, R7 R8, R1, R9 R10, R1, R clock IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB IF ID IF EX ID MEM EX MEM WB WB IF ID EX MEM WB Bala Gurumurthy Hardware-Sensitive Database Operations 24

42 Control Hazards Control hazards are often more severe than are data hazards. Most simple implementation: flush pipeline, redo instr. fetch clock branch instr. i IF ID EX MEM WB instr. i+1 IF idle idle idle idle target instr. IF ID EX MEM WB target instr. i+1 IF ID EX MEM WB With increasing pipeline depths, the penalty gets worse. Bala Gurumurthy Hardware-Sensitive Database Operations 25

43 Control Hazards A simple optimization is to only flush if the branch was taken. Penalty only occurs for taken branches. If the two outcomes have different (known) likeliness: Generate code such that a non-taken branch is more likely. Aborting a running instruction is harder when the branch outcome is known late. Should not change exception behavior. This scheme is called predicted-untaken. Likewise: predicted-taken (but often less effective) Bala Gurumurthy Hardware-Sensitive Database Operations 26

44 Branch Prediction Modern CPUs try to predict the target of a branch execute the target code speculatively. Prediction must happen early (ID stage too late). Thus: Branch Target Buffers (BTBs) Lookup Table: PC predicted target, taken?. Lookup PC Predicted PC Taken?... Consult Branch Target Buffer parallel to instruction fetch. If entry for current PC can be found: follow prediction. If not, create entry after branching. Inner workings of modern branch predictors are highly involved ( typically kept secret). Bala Gurumurthy Hardware-Sensitive Database Operations 27

45 How to overcome hazards? Reduce data hazards in tight loops: Loop unrolling Reduce control hazards: Branch-free Code Bala Gurumurthy Hardware-Sensitive Database Operations 28

46 Pipelining in CPUs Loop Unrolling Optimizing Tight Loops Bala Gurumurthy Hardware-Sensitive Database Operations 29

47 Loops are like eating ice cream on a warm summer s day: Bala Gurumurthy Hardware-Sensitive Database Operations 30

48 Loops are like eating ice cream on a warm summer s day: If your spoon is too small Bala Gurumurthy Hardware-Sensitive Database Operations 30

49 Loops are like eating ice cream on a warm summer s day: If your spoon is too small Your ice cream will melt to fast Bala Gurumurthy Hardware-Sensitive Database Operations 30

50 Loops are like eating ice cream on a warm summer s day: If your spoon is too big Bala Gurumurthy Hardware-Sensitive Database Operations 30

51 Loops are like eating ice cream on a warm summer s day: If your spoon is too big You will get brain freeze Bala Gurumurthy Hardware-Sensitive Database Operations 30

52 Loops are like eating ice cream on a warm summer s day: Only the right size of the spoon will be optimal: Bala Gurumurthy Hardware-Sensitive Database Operations 30

53 Loops are like eating ice cream on a warm summer s day: Only the right size of the spoon will be optimal: Bala Gurumurthy Hardware-Sensitive Database Operations 30

54 Tight loops are a good cidate to improve instruction scheduling. for (i = 1000; i > 0; i = i - 1) x[i] = x[i] + s; l: L.D F0, 0(R1) ; F0=array element ADD.D F4, F0, F2 ; add scalar in F2 S.D F4, 0(R1) ; store result DADDUI R1, R1, #-8 ; decrement pointer BNE R1, R2, l ; branch R1!=R2 naïve code source:[hennessy Patterson, 2012] Bala Gurumurthy Hardware-Sensitive Database Operations 31

55 Tight loops are a good cidate to improve instruction scheduling. for (i = 1000; i > 0; i = i - 1) x[i] = x[i] + s; l: L.D F0, 0(R1) ; F0=array element stall ADD.D F4, F0, F2 ; add scalar in F2 stall stall S.D F4, 0(R1) ; store result DADDUI R1, R1, #-8 ; decrement pointer stall BNE R1, R2, l ; branch R1!=R2 naïve code source:[hennessy Patterson, 2012] Bala Gurumurthy Hardware-Sensitive Database Operations 31

56 Tight loops are a good cidate to improve instruction scheduling. for (i = 1000; i > 0; i = i - 1) x[i] = x[i] + s; l: L.D F0, 0(R1) ; F0=array element stall ADD.D F4, F0, F2 ; add scalar in F2 stall stall S.D F4, 0(R1) ; store result DADDUI R1, R1, #-8 ; decrement pointer stall BNE R1, R2, l ; branch R1!=R2 naïve code l: L.D F0, 0(R1) DADDUI R1, R1, #-8 ADD.D F4, F0, F2 stall stall S.D F4, 0(R1) BNE R1, R2, l re-schedule source:[hennessy Patterson, 2012] Bala Gurumurthy Hardware-Sensitive Database Operations 31

57 Compiler Loop Unrolling l: L.D F0, 0(R1) L.D F6, -8(R1) L.D F10, -16(R1) L.D F14, -24(R1) ADD.D F4, F0, F2 ADD.D F8, F6, F2 ADD.D F12, F10, F2 ADD.D F16, F14, F2 S.D F4, 0(R1) S.D F8, -8(R1) DADDUI R1, R1, #-32 S.D F12, 16(R1) S.D F16, 8(R1) BNE R1, R2, l for (i = 1000; i > 0; i = i - 1) x[i] = x[i] + s; Compiler flags: -funroll-loops for unrolling loops with known number of iterations -funroll-all-loops for unrolling every loop Individual unrolling of arbitrary loops impossible with compiler flags loop unrolling source:[hennessy Patterson, 2012] Bala Gurumurthy Hardware-Sensitive Database Operations 32

58 H Loop Unrolling What to do for loops with undefined number of iterations? for (i = 0; i < array size; ++ i) x[i] = x[i] + s; Bala Gurumurthy Hardware-Sensitive Database Operations 33

59 H Loop Unrolling What to do for loops with undefined number of iterations? for (i = 0; i < array size; ++ i) x[i] = x[i] + s; Unroll loops by h: for (i = 0; i + 3 < array size; i += 4){ x[i] = x[i] + s; x[i+1] = x[i+1] + s; x[i+2] = x[i+2] + s; x[i+3] = x[i+3] + s; } for (; i < array size; ++ i ) x[i] = x[i] + s; Bala Gurumurthy Hardware-Sensitive Database Operations 33

60 Loop Unrolling for Tight loops can be found in selections: SELECT * FROM lineitem WHERE quantity < n Or, extract position list (written in C): for (unsigned int i = 0; i < num tuples; ++i) if (lineitem[i].quantity < n) poslist[pos++]=i; Bala Gurumurthy Hardware-Sensitive Database Operations 34

61 Loop Unrolling for However, best unrolling-depth depends on used CPU: response time in ms Intel Core 2 Quad Q9400 Intel Core 2 Quad Q9550 Intel Core i Simple Scan LU2-Scan LU3-Scan LU4-Scan LU5-Scan LU6-Scan LU7-Scan LU8-Scan Intel Xeon E v2 Bala Gurumurthy Hardware-Sensitive Database Operations 35

62 Pipelining in CPUs Branch-Free Code Bala Gurumurthy Hardware-Sensitive Database Operations 36

63 Selection Conditions Selection queries are sensitive to branch prediction: SELECT * FROM lineitem WHERE quantity < n Or, extract position list (written in C): for (unsigned int i = 0; i < num tuples; ++i) if (lineitem[i].quantity < n) poslist[pos++]=i; Bala Gurumurthy Hardware-Sensitive Database Operations 37

64 Selection Conditions (Intel Xeon E5-2609) response time in ms selectivity factor in % Serial Selection Bala Gurumurthy Hardware-Sensitive Database Operations 38

65 Predication Predication: Turn control flow into data flow. for (unsigned int i = 0; i < num tuples; ++i) poslist[pos] = i; pos += (lineitem[i].quantity < n); This code does not use a branch any more. 1 The price we pay is a + operation for every iteration. Execution cost should now be independent of predicate selectivity. 1 except to implement the loop Bala Gurumurthy Hardware-Sensitive Database Operations 39

66 Predication(Intel Xeon E5-2609) response time in ms selectivity factor in % Serial Selection Serial Branch-Free Selection Bala Gurumurthy Hardware-Sensitive Database Operations 40

67 Predication This was an example of software predication. Some CPUs also support hardware predication. E.g., Intel Itanium2: Execute both branches of an if-then-else discard one result. Bala Gurumurthy Hardware-Sensitive Database Operations 41

68 [Boncz et al., 2005]. Bala Figure Gurumurthy 2: ItaniumHardware-Sensitive DatabasePredication Operations Eliminates42 D B Experiments (AMD AthlonMP / Intel Itanium2) int sel_lt_int_col_int_val(int n, int* res, int* in, int V) { for(int i=0,j=0; i<n; i++){ Itanium2 branch 100 /* branch version */ Itanium2 predicated AthlonMP branch if (src[i] < V) AthlonMP predicated 80 out[j++] = i; /* predicated version */ bool b = (src[i] < V); out[j] = i; j += b; } return j; } msec query selectivity

69 Conjunctive Predicates In general, we have to hle multiple predicates: SELECT A 1,..., A n FROM R WHERE p 1 AND p 2 AND... AND p k The stard C implementation uses && for the conjunction: for (unsigned int i = 0; i < num tuples; ++i) if (p 1 && p 2 &&... && p k )...; Bala Gurumurthy Hardware-Sensitive Database Operations 43

70 Conjunctive Predicates The && introduce even more branches. The use of && is equivalent to: for (unsigned int i = 0; i < num tuples; i++) if (p 1 ) if (p 2 ). if (p k )...; Bala Gurumurthy Hardware-Sensitive Database Operations 44

71 Conjunctive Predicates The && introduce even more branches. The use of && is equivalent to: for (unsigned int i = 0; i < num tuples; i++) if (p 1 ) if (p 2 ). if (p k )...; An alternative is the use of the logical &: for (unsigned int i = 0; i < num tuples; i++) if (p 1 & p 2 &... & p k )...; Bala Gurumurthy Hardware-Sensitive Database Operations 44

72 Conjunctive Predicates This allows us to express queries with conjunctive predicates without branches. for (unsigned int i = 0; i < num tuples; i++) { answer[j] = i; j += (p 1 & p 2 &... & p k ); } Bala Gurumurthy Hardware-Sensitive Database Operations 45

73 Selection Conditions in Main Memory Experiments (Intel Pentium III) Fig. 1. Three implementations: Pentium. [Ross, 2004]. Bala Gurumurthy Hardware-Sensitive Database Operations 46

74 Cost Model A query compiler could use a cost model to select between variants. p && q When p is highly selective, this might amortize the double branch misprediction risk. p & q Number of branches halved, but q is evaluated regardless of p s outcome. j +=... Performs memory write in each iteration. Notes: Sometimes, && is necessary to prevent null pointer dereferences: if (p && p->foo == 42). Exact behavior is hardware-specific. Bala Gurumurthy Hardware-Sensitive Database Operations 47

75 Kenneth A. Ross Experiments (Sun UltraSparc III) D B Fig. 2. Three implementations: Sun. [Ross, 2004]. Bala Gurumurthy Hardware-Sensitive Database Operations 48

76 Conclusion First Part Increasing heterogeneity of hardware Hardware-oblivious vs. hardware-sensitive programming Pipelining to increase IPC Hazards Structural hazards Data hazards Control hazards Loop-Unrolling Predication Bala Gurumurthy Hardware-Sensitive Database Operations 49

77 Recap Loop Unrolling And Branch Predication Bala Gurumurthy Hardware-Sensitive Database Operations 50

78 Loop Unrolling What is the problem? - Tight loops are inefficient How? - They lead to stalls in processing pipeline for (i = 1000; i > 0; i = i - 1) x[i] = x[i] + s; Bala Gurumurthy Hardware-Sensitive Database Operations 51

79 Loop Unrolling What is the problem? - Tight loops are inefficient How? - They lead to stalls in processing pipeline l: L.D F0, 0(R1) ; F0=array element ADD.D F4, F0, F2 ; add scalar in F2 S.D F4, 0(R1) ; store result DADDUI R1, R1, #-8 ; decrement pointer BNE R1, R2, l ; branch R1!=R2 Bala Gurumurthy Hardware-Sensitive Database Operations 51

80 Loop Unrolling What is the problem? - Tight loops are inefficient How? - They lead to stalls in processing pipeline Time LD F0,0(R1) ADD F4, F0,F2 SD F4,0(R1) DADDUI R1,R1,#-8 BNE R1,R2,l Bala Gurumurthy Hardware-Sensitive Database Operations 52

81 Loop Unrolling What is the problem? - Tight loops are inefficient How? - They lead to stalls in processing pipeline Time LD F0,0(R1) F0 LOAD ADD F4, F0,F2 F0 + F2 F4 STORE SD F4,0(R1) F4 STORE DADDUI R1,R1,#-8 DADDUI R1 STORE BNE R1,R2,l BNE Bala Gurumurthy Hardware-Sensitive Database Operations 52

82 Loop Unrolling What is the problem? - Tight loops are inefficient How? - They lead to stalls in processing pipeline How to reduce these stalls? Re-arrange the code Time LD F0,0(R1) F0 LOAD The order has to be preserved ADD F4, F0,F2 F0 + F2 F4 STORE SD F4,0(R1) F4 STORE DADDUI R1,R1,#-8 DADDUI R1 STORE BNE R1,R2,l BNE Bala Gurumurthy Hardware-Sensitive Database Operations 53

83 Loop Unrolling What is the problem? - Tight loops are inefficient How? - They lead to stalls in processing pipeline How to reduce these stalls? Re-arrange the code Time LD F0,0(R1) F0 LOAD DADDUI R1,R1,#-8 ADD F4, F0,F2 F4 STORE SD F4,0(R1) BNE R1,R2,l BNE Bala Gurumurthy Hardware-Sensitive Database Operations 53

84 Loop Unrolling What is the problem? - Tight loops are inefficient How? - They lead to stalls in processing pipeline How to reduce these stalls? Re-arrange the code loop unrolling l: L.D F0, 0(R1) ; F0=array element ADD.D F4, F0, F2 ; add scalar in F2 S.D F4, 0(R1) ; store result DADDUI R1, R1, #-8 ; decrement pointer BNE R1, R2, l ; branch R1!=R2 Bala Gurumurthy Hardware-Sensitive Database Operations 54

85 Loop Unrolling What is the problem? - Tight loops are inefficient How? - They lead to stalls in processing pipeline How to reduce these stalls? Re-arrange the code loop unrolling l: L.D F0, 0(R1) L.D F6, -8(R1) L.D F10, -16(R1) L.D F14, -24(R1) ADD.D F4, F0, F2 ADD.D F8, F6, F2 ADD.D F12, F10, F2 ADD.D F16, F14, F2 S.D F4, 0(R1) S.D F8, -8(R1) DADDUI R1, R1, #-32 S.D F12, 16(R1) S.D F16, 8(R1) BNE R1, R2, l Bala Gurumurthy Hardware-Sensitive Database Operations 54

86 Loop Unrolling What is the problem? - Tight loops are inefficient How? - They lead to stalls in processing pipeline How to reduce these stalls? Re-arrange the code loop unrolling Time Instr: compute branch result Instr+1: Use branch result Bala Gurumurthy Hardware-Sensitive Database Operations 54

87 Loop Unrolling What is the problem? - Tight loops are inefficient How? - They lead to stalls in processing pipeline How to reduce these stalls? Re-arrange the code loop unrolling Time Instr i - V1 Instr i - V2 Instr i+1 - V1 Instr i+1 - V2 Bala Gurumurthy Hardware-Sensitive Database Operations 54

88 Loop Unrolling What is the problem? - Tight loops are inefficient How? - They lead to stalls in processing pipeline How to reduce these stalls? Re-arrange the code loop unrolling Time Instr 1 - v1 Instr 1 - v2 Instr 2 - v1 Instr 2 - v2 Bala Gurumurthy Hardware-Sensitive Database Operations 54

89 Predication What is the problem? - branching leads to pipeline flushing how? - next instruction is not known until branch result is evaluated SELECT * FROM lineitem WHERE quantity < n Bala Gurumurthy Hardware-Sensitive Database Operations 55

90 Predication What is the problem? - branching leads to pipeline flushing how? - next instruction is not known until branch result is evaluated for (unsigned int i = 0; i < num tuples; ++i) if (lineitem[i].quantity < n) poslist[pos++]=i; Bala Gurumurthy Hardware-Sensitive Database Operations 55

91 Predication What is the problem? - branching leads to pipeline flushing how? - next instruction is not known until branch result is evaluated true lineitem[i] < n false poslist[pos++] = i end Bala Gurumurthy Hardware-Sensitive Database Operations 55

92 Predication What is the problem? - branching leads to pipeline flushing how? - next instruction is not known until branch result is evaluated Time Instr i IF ID Instr i+1 IF Bala Gurumurthy Hardware-Sensitive Database Operations 55

93 Predication What is the problem? - branching leads to pipeline flushing how? - next instruction is not known until branch result is evaluated Time Instr i IF ID Instr i+1 IF ID Bala Gurumurthy Hardware-Sensitive Database Operations 55

94 Predication What is the problem? - branching leads to pipeline flushing how? - next instruction is not known until branch result is evaluated Time Instr i IF ID Instr i+1 IF Target Instr IF ID Bala Gurumurthy Hardware-Sensitive Database Operations 55

95 Predication What is the problem? - branching leads to pipeline flushing how? - next instruction is not known until branch result is evaluated How to reduce? predication - convert control to data dependent statement Predication: use the result of the condition statement as input Time Instr: compute branch result Instr+1: Use branch result Bala Gurumurthy Hardware-Sensitive Database Operations 56

96 Predication What is the problem? - branching leads to pipeline flushing how? - next instruction is not known until branch result is evaluated How to reduce? predication - convert control to data dependent statement Predication: use the result of the condition statement as input Time Instr i - V1 Instr i - V2 Instr i+1 - V1 Instr i+1 - V2 Bala Gurumurthy Hardware-Sensitive Database Operations 56

97 Vectorization Leveraging Modern Processing Capabilities Bala Gurumurthy Hardware-Sensitive Database Operations 57

98 Hardware Parallelism Pipelining is one technique to leverage available hardware parallelism. Chip die Task 1 Task 2 Task 3 Separate chip regions for individual tasks execute independently. Advantage: Use parallelism, but maintain sequential execution semantics at front-end (here: assembly instruction stream). We discussed problems around hazards before VLSI technology limits the degree up to which pipelining is feasible [Kaeslin, 2008] Bala Gurumurthy Hardware-Sensitive Database Operations 58

99 Hardware Parallelism Chip area can as well be used for other types of parallelism: in 1 in 2 in 3 Task 1 Task 2 Task 3 out 1 out 2 out 3 Bala Gurumurthy Hardware-Sensitive Database Operations 59

100 Hardware Parallelism Chip area can as well be used for other types of parallelism: in 1 in 2 in 3 Task 1 Task 2 Task 3 out 1 out 2 out 3 Computer systems typically use identical hardware circuits, but their function may be controlled by different instruction streams s i : s 1 s 2 s 3 in 1 in 2 in 3 PU PU PU out 1 out 2 out 3 Bala Gurumurthy Hardware-Sensitive Database Operations 59

101 Special Instances Do you know an example of this architecture? s 1 s 2 s 3 in 1 in 2 in 3 PU PU PU out 1 out 2 out 3 Bala Gurumurthy Hardware-Sensitive Database Operations 60

102 Special Instances Do you know an example of this architecture? s 1 s 2 s 3 in 1 in 2 in 3 PU PU PU out 1 out 2 out 3 This is your multi-core CPU! Also called MIMD: Multiple Instructions, Multiple Data (Single-core is SISD: Single Instruction, Single Data.) Bala Gurumurthy Hardware-Sensitive Database Operations 60

103 Vectorization SIMD: Single Instruction, Multiple Data Vectorized Execution Bala Gurumurthy Hardware-Sensitive Database Operations 61

104 Special Instances (SIMD) Most modern processors also include a SIMD unit: s 1 in 1 in 2 in 3 PU PU PU out 1 out 2 out 3 Execute same assembly instruction on a set of values. Also called vector unit; vector processors are entire systems built on that idea. Bala Gurumurthy Hardware-Sensitive Database Operations 62

105 SIMD Programming Model The processing model is typically based on SIMD registers or vectors: a 1 a 2... a n b 1 b 2... b n a 1 + b 1 a 2 + b 2... a n + b n Typical values (e.g., x86-64): 128 bit-wide registers (xmm0 through xmm15). Usable as 16 8 bit, 8 16 bit, 4 32 bit, or 2 64 bit. Bala Gurumurthy Hardware-Sensitive Database Operations 63

106 SIMD Programming Model Much of a processor s control logic depends on the number of in-flight instructions /or the number of registers, but not on the size of registers. scheduling, register renaming, dependency tracking,... SIMD instructions make independence explicit. No data hazards within a vector instruction. Check for data hazards only between vectors. data parallelism Parallel execution promises n-fold performance advantage. (Not quite achievable in practice, however.) Bala Gurumurthy Hardware-Sensitive Database Operations 64

107 SIMD Example Addition of two integer arrays (4 byte = 32 bit values) A B bits Result bits Have to perform addition 4 times Bala Gurumurthy Hardware-Sensitive Database Operations 65

108 SIMD Example Addition of two integer arrays (4 byte = 32 bit values) A B bits Result bits Have to perform addition once! Bala Gurumurthy Hardware-Sensitive Database Operations 65

109 Coding for SIMD How can I make use of SIMD instructions as a programmer? 1. Auto-Vectorization Some compiler automatically detect opportunities to use SIMD. Approach rather limited; don t rely on it. Advantage: platform independent Bala Gurumurthy Hardware-Sensitive Database Operations 66

110 Coding for SIMD How can I make use of SIMD instructions as a programmer? 1. Auto-Vectorization Some compiler automatically detect opportunities to use SIMD. Approach rather limited; don t rely on it. Advantage: platform independent 2. Compiler Attributes Use attribute ((vector size (size in bytes))) annotations to state your intentions. Advantage: platform independent (Compiler will generate non-simd code if the platform does not support it.) Bala Gurumurthy Hardware-Sensitive Database Operations 66

111 /* * Auto vectorization example (tried with gcc 4.3.4) */ #include <stdlib.h> #include <stdio.h> int main (int argc, char **argv){ } int a[256], b[256], c[256]; for (unsigned int i = 0; i < 256; i++) { a[i] = i + 1; b[i] = 100 * (i + 1); } for (unsigned int i = 0; i < 256; i++) c[i] = a[i] + b[i]; printf ("c = [ %i, %i, %i, %i ]\n", c[0], c[1], c[2], c[3]); return EXIT_SUCCESS; Bala Gurumurthy Hardware-Sensitive Database Operations 67

112 Resulting assembly code (gcc 4.3.4, x86-64): loop: movdqu (%r8,%rcx), %xmm0 ; load a b addl $1, %esi movdqu (%r9,%rcx), %xmm1 ; into SIMD registers paddd %xmm1, %xmm0 ; parallel add movdqa %xmm0, (%rax,%rcx) ; write result to memory addq $16, %rcx ; loop (increment by cmpl %r11d, %esi ; SIMD length of 16 bytes) jb loop Bala Gurumurthy Hardware-Sensitive Database Operations 68

113 Resulting assembly code (gcc 4.3.4, x86-64): loop: movdqu (%r8,%rcx), %xmm0 ; load a b addl $1, %esi movdqu (%r9,%rcx), %xmm1 ; into SIMD registers paddd %xmm1, %xmm0 ; parallel add movdqa %xmm0, (%rax,%rcx) ; write result to memory addq $16, %rcx ; loop (increment by cmpl %r11d, %esi ; SIMD length of 16 bytes) jb loop xmm0, xmm1 are SIMD registers (128 bits wide) Are there ymm, zmm registers? Bala Gurumurthy Hardware-Sensitive Database Operations 68

114 /* Use attributes to trigger vectorization */ #include <stdlib.h> #include <stdio.h> typedef int v4si attribute ((vector_size (16))); union int_vec { int val[4]; v4si vec; }; typedef union int_vec int_vec; int main (int argc, char **argv) { int_vec a, b, c; } a.val[0] = 1; a.val[1] = 2; a.val[2] = 3; a.val[3] = 4; b.val[0] = 100; b.val[1] = 200; b.val[2] = 300; b.val[3] = 400; c.vec = a.vec + b.vec; printf ("c = [ %i, %i, %i, %i ]\n", c.val[0], c.val[1], c.val[2], c.val[3]); return EXIT_SUCCESS; Bala Gurumurthy Hardware-Sensitive Database Operations 69

115 Resulting assembly code (gcc, x86-64): movl $1, -16(%rbp) ; assign constants movl $2, -12(%rbp) ; write them movl $3, -8(%rbp) ; to memory movl $4, -4(%rbp) movl $100, -32(%rbp) movl $200, -28(%rbp) movl $300, -24(%rbp) movl $400, -20(%rbp) movdqa -32(%rbp), %xmm0 ; load b into SIMD register xmm0 paddd -16(%rbp), %xmm0 ; SIMD xmm0 = xmm0 + a movdqa %xmm0, -48(%rbp) ; write SIMD xmm0 back to memory movl -40(%rbp), %ecx ; load c into scalar movl -44(%rbp), %edx ; registers (from memory) movl -48(%rbp), %esi movl -36(%rbp), %r8d Bala Gurumurthy Hardware-Sensitive Database Operations 70

116 Coding for SIMD 3. Use C Compiler Intrinsics Invoke SIMD instructions directly via compiler macros. Programmer has good control over instructions generated. Code no longer portable to different architecture. Benefit (over h-written assembly): compiler manages register allocation. Risk: If not done carefully, automatic glue code (casts, etc.) may make code inefficient. Bala Gurumurthy Hardware-Sensitive Database Operations 71

117 /* * Invoke SIMD instructions explicitly via intrinsics. */ #include <stdlib.h> #include <stdio.h> #include <xmmintrin.h> int main (int argc, char **argv) { int a[4], b[4], c[4]; m128i x, y; } a[0] = 1; a[1] = 2; a[2] = 3; a[3] = 4; b[0] = 100; b[1] = 200; b[2] = 300; b[3] = 400; x = _mm_loadu_si128 (( m128i *) a); y = _mm_loadu_si128 (( m128i *) b); x = _mm_add_epi32 (x, y); _mm_storeu_si128 (( m128i *) c, x); printf ("c = [ %i, %i, %i, %i ]\n", c[0], c[1], c[2], c[3]); return EXIT_SUCCESS; Bala Gurumurthy Hardware-Sensitive Database Operations 72

118 Resulting assembly code (gcc, x86-64): movdqu -16(%rbp), %xmm1 ; _mm_loadu_si128() movdqu -32(%rbp), %xmm0 ; _mm_loadu_si128() paddd %xmm0, %xmm1 ; _mm_add_epi32() movdqu %xmm1, -48(%rbp) ; _mm_storeu_si128() Bala Gurumurthy Hardware-Sensitive Database Operations 73

119 SIMD Instruction Sets History Started for desktop PCs with Intel s MMX (1996) 8 registers (MM0 - MM7) Each 64-bit wide 3DNow! by AMD (1998) AltiVec instruction set (between ) By Apple, IBM, Motorola 32 registers Each 128-bit wide Bala Gurumurthy Hardware-Sensitive Database Operations 74

120 SIMD Instruction Sets History Intel s answer: SSE instruction set (1999) Streaming SIMD Extensions 8-16 registers (XMM0-XMM15) Each 128-bit wide SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2 Increasing SIMD widths: AVX (2008) Advanced Vector Extensions By Intel AMD 8-16 registers (XMM0,YMM0 XMM15,YMM15) Each 256-bit wide Extension of SSE instructions to operate on 256-bit AVX, AVX2, AVX-512 (2013) Bala Gurumurthy Hardware-Sensitive Database Operations 75

121 SSE Instruction Set 2 Contents Data types m128d for double-precision floating point m128 for single-precision floating point m128i non floating-point data Arithmetical operations mm add, mm mul, mm div, mm sub Horizontal operations for SSE3 higher Compare operations mm cmplt, mm cmpgt, mm cmpge, mm cmple, mm cmpeq Create bit mask Logical operations mm, mm or, mm not, mm xor 2 Bala Gurumurthy Hardware-Sensitive Database Operations 76

122 SSE Instruction Set 2 Contents II Move/Blend operations Move parts of a float value Blending: only selected values are copied Shifting in zeros/ones Shuffle/permute data in registers Load/Store operations Loading/storing different data types Often special h or l operations for float Often also unaligned access 2 Bala Gurumurthy Hardware-Sensitive Database Operations 77

123 SIMD Instructions Limitations There are no branching primitives for SIMD registers. What would their semantics be anyhow? Some SIMD instructions require hard-coded parameters. Thus: Exp code explicitly for all possible values of n. Ex: Fits with operator specialization in column-oriented DBMSs Bala Gurumurthy Hardware-Sensitive Database Operations 78

124 SIMD Instructions Limitations 2 Data alignment Alignment Hazard Operates best on 16-byte (128-bit) aligned data Unaligned access much slower 4 byte int* vec cache line (64 byte) Bala Gurumurthy Hardware-Sensitive Database Operations 79

125 SIMD Instructions Limitations 2 Data alignment Alignment Hazard Operates best on 16-byte (128-bit) aligned data Unaligned access much slower 4 byte int* vec 1 cache line (64 byte) Bala Gurumurthy Hardware-Sensitive Database Operations 79

126 SIMD Instructions Limitations 2 Data alignment Alignment Hazard Operates best on 16-byte (128-bit) aligned data Unaligned access much slower 4 byte int* vec 1 2 cache line (64 byte) Bala Gurumurthy Hardware-Sensitive Database Operations 79

127 SIMD Instructions Alignment Hazard How to avoid alignment hazards? Bala Gurumurthy Hardware-Sensitive Database Operations 80

128 SIMD Instructions Alignment Hazard How to avoid alignment hazards? Process unaligned data beforeh int alignment_offset = ((intptr_t)sse_array)%sizeof( m128i); for(unsigned int i=0;i<alignment_offset/sizeof(int);i++){ //Process unaligned data } // Process aligned data using SIMD Bala Gurumurthy Hardware-Sensitive Database Operations 80

129 SIMD Instructions Alignment Hazard How to avoid alignment hazards? Process unaligned data beforeh int alignment_offset = ((intptr_t)sse_array)%sizeof( m128i); for(unsigned int i=0;i<alignment_offset/sizeof(int);i++){ //Process unaligned data } // Process aligned data using SIMD Align pointer of allocated memory to aligned address: /* Make newp a pointer to a 64-bit aligned array of NUM_ELEMENTS 64-bit elements. */ double *p, *newp; p = (double*)malloc (sizeof(double)*(num_elements+1)); newp = (p+7) & (~0x7); Bala Gurumurthy Hardware-Sensitive Database Operations 80

130 Vectorization SIMD for Database Tasks Bala Gurumurthy Hardware-Sensitive Database Operations 81

131 SIMD : Scan-Based Tasks SIMD functionality naturally fits a number of scan-based database tasks: arithmetics SELECT price + tax AS net price FROM orders This is what the code examples on the previous slides did. aggregation SELECT COUNT(*) FROM lineitem WHERE price > 42 How can this be done efficiently? Similar: SUM( ), MAX( ), MIN( ),... Bala Gurumurthy Hardware-Sensitive Database Operations 82

132 SIMD : Scan-Based Tasks Selection queries are a slightly more tricky: Missing branching primitives for SIMD registers. for (unsigned int i = 0; i < num tuples; ++i) if (lineitem[i].quantity < n) poslist[pos++]=i; Moving data between SIMD scalar registers is quite expensive. Either move one data item at a time, or extract sign mask from SIMD registers. Thus: Use SIMD to generate bit vector; interpret it in scalar mode. Bala Gurumurthy Hardware-Sensitive Database Operations 83

133 SIMD : Scan-Based Tasks Selection queries are a slightly more tricky: Missing branching primitives for SIMD registers. for (unsigned int i = 0; i < num tuples; ++i) if (lineitem[i].quantity < n) poslist[pos++]=i; Moving data between SIMD scalar registers is quite expensive. Either move one data item at a time, or extract sign mask from SIMD registers. Thus: Use SIMD to generate bit vector; interpret it in scalar mode. If we can count with SIMD, why can t we play the pos ++ trick? for (unsigned int i = 0; i < num tuples; ++i) poslist[pos] = i; pos += (lineitem[i].quantity < n); Bala Gurumurthy Hardware-Sensitive Database Operations 83

134 SIMD Scan with Bit Mask Evaluation for(unsigned int i=0;i<sse_array_length;i++){ read_value=_mm_load_si128(&sse_array[i]); m128 comp_result = ( m128) mm_cmplt_epi32(read_value,comp_val); int mask= _mm_movemask_ps(comp_result); if(mask){ for(unsigned j=0;j<sizeof( m128i)/sizeof(int);++j){ if((mask >> j) & 1) result_array[pos++]=base_tid+j; } } } Bala Gurumurthy Hardware-Sensitive Database Operations 84

135 SIMD Scan with Bit Mask Evaluation SIMD 1,2,3,4-1,-1,0,0 Bitmap <=2 Scalar interpret 1 2 Bala Gurumurthy Hardware-Sensitive Database Operations 84

136 SIMD : Sorting Sorting is a compute intensive task Often involves control flow: Quick sort Insertion sort Radix sort Has multiple uncertainties: leading to more control flow. What are they? Bala Gurumurthy Hardware-Sensitive Database Operations 85

137 SIMD : Sorting Sorting is a compute intensive task Often involves control flow: Quick sort Insertion sort Radix sort Has multiple uncertainties: leading to more control flow. What are they? Location of placing a value Selection of rom values in some techniques (Quick sort: pivot) Bala Gurumurthy Hardware-Sensitive Database Operations 85

138 SIMD : Sorting Sorting is a compute intensive task Often involves control flow: Quick sort Insertion sort Radix sort Has multiple uncertainties: leading to more control flow. What are they? Location of placing a value Selection of rom values in some techniques (Quick sort: pivot) Is there a sorting strategy involving less control flow more arithmetical operations? Bala Gurumurthy Hardware-Sensitive Database Operations 85

139 SIMD : Sorting Sorting is a compute intensive task Often involves control flow: Quick sort Insertion sort Radix sort Has multiple uncertainties: leading to more control flow. What are they? Location of placing a value Selection of rom values in some techniques (Quick sort: pivot) Is there a sorting strategy involving less control flow more arithmetical operations? Merge sort using sorting/merging networks Bala Gurumurthy Hardware-Sensitive Database Operations 85

140 SIMD Accelerated Merge Sort [Balkesen et al., 2013] Merge sort uses 3 phases: In-register sorting Sorting networks In-cache sorting Merging networks Out-of-cache sorting Multi-way merging Bala Gurumurthy Hardware-Sensitive Database Operations 86

141 Sorting Network A sorting network maps to a sequence of min/max operations with input a, b, c, d output w, x, y, z Data passes several comparators Comparator emits: Smaller value at top Bigger value at bottom adapted from [Balkesen et al., 2013] e = min(a, b) f = max(a, b) g = min(c, d) h = max(c, d) i = max(e, g) j = min(f, h) w = min(e, g) x = min(i, j) y = max(i, j) z = max(f, h) Bala Gurumurthy Hardware-Sensitive Database Operations 87

142 SIMD-Accelerated Sorting Network SIMD min/max input registers sorted between registers SIMD shuffles sorted in each register adapted from [Chhugani et al., 2008] Bala Gurumurthy Hardware-Sensitive Database Operations 88

143 SIMD-Accelerated Sorting Network SIMD min/max input registers sorted between registers SIMD shuffles sorted in each register adapted from [Chhugani et al., 2008] Bala Gurumurthy Hardware-Sensitive Database Operations 88

144 SIMD-Accelerated Sorting Network SIMD min/max input registers sorted between registers SIMD shuffles sorted in each register adapted from [Chhugani et al., 2008] Bala Gurumurthy Hardware-Sensitive Database Operations 88

145 SIMD-Accelerated Sorting Network SIMD min/max input registers sorted between registers SIMD shuffles sorted in each register adapted from [Chhugani et al., 2008] Bala Gurumurthy Hardware-Sensitive Database Operations 88

146 SIMD-Accelerated Sorting Network SIMD min/max input registers sorted between registers SIMD shuffles sorted in each register adapted from [Chhugani et al., 2008] Bala Gurumurthy Hardware-Sensitive Database Operations 88

147 SIMD-Accelerated Sorting Network SIMD min/max input registers sorted between registers SIMD shuffles Each SIMD lane is sorted Sorted lists have to be merged sorted in each register adapted from [Chhugani et al., 2008] Bala Gurumurthy Hardware-Sensitive Database Operations 88

148 SIMD-Accelerated Merge Network Odd-Even Merge Network a 1 a 2 a 3 a 4 b 1 b 2 b 3 b 4 out 1 out 2 out 3 out 4 out 5 out 6 out 7 out 8 Inputs sorted in same order 6 min/max operations Masking/blending needed Bala Gurumurthy Hardware-Sensitive Database Operations 89

149 SIMD-Accelerated Merge Network Bitonic Merge Network a 1 a 2 a 3 a 4 b 4 b 3 b 2 out 1 out 2 out 3 out 4 out 5 out 6 out 7 Second input in reverse order 1 shift needed 6 min/max operations No masking/blending needed each register is rewritten b 1 out 8 Bala Gurumurthy Hardware-Sensitive Database Operations 90

150 SIMD-Accelerated Merge Network Bitonic Merge Network a 1 a 2 a 3 a 4 b 4 b 3 b 2 out 1 out 2 out 3 out 4 out 5 out 6 out 7 Second input in reverse order 1 shift needed 6 min/max operations No masking/blending needed each register is rewritten b 1 out 8 Better suited for SIMD! Bala Gurumurthy Hardware-Sensitive Database Operations 90