Summarizer: Trading Communication with Computing Near Storage

Transcription

1 Summarizer: Trading Communication with Computing Near Storage Gunjae Koo*, Kiran Kumar Matam*, Te I, H.V. Krishina Giri Nara*, Jing Li, Hung-Wei Tseng, Steven Swanson, Murali Annavaram* *University of Southern California North Carolina State University University of California, San Diego

2 Motivation High Data Movement Cost Host Limited data bandwidth High access latency CPU Storage interface External External (host -- storage) (host -- storage) Internal Internal Data host Data transfer from storage 2

3 Near Data Processing (NDP) Host CPU Storage interface Storage Processor (SP) External (host storage) Internal Data host Data transfer from storage 3

4 Near Data Processing (NDP) Host CPU Storage interface Storage Processor (SP) W/O NDP With NDP Data host External (host storage) Data transfer from storage Data storage Internal 4

5 Near Data Processing (NDP) on SSDs Host CPU Storage interface SP Garbage collection Wearleveling W/O NDP With NDP Data host External (host storage) Data transfer from storage Data computation Data storage Internal 5

6 Near Data Processing (NDP) on SSDs Host Obstacles to in-ssd processing CPU Storage interface SP Less powerful embedded processor Garbage collection Dynamic computation resource availability Wearleveling W/O NDP With NDP External (host storage) Manual workload partitioning is difficult Summarizer: Dynamic NDP framework for SSD Data host Data transfer from storage Data storage Internal 6

7 Summarizer Basic Concept Host Monitoring resources CPU Storage interface AP 7

8 Summarizer Basic Concept Host Monitoring resources CPU Storage interface AP 8

9 Summarizer Detailed Firmware Architecture Storage Interface (PCIe / NVMe) Request queue Response queue Host CPU User Applications / Operating Systems SSD Firmware I/O Controller (NVMe command decoder) Translation Layer (FTL) Summarizer Task Controller SSD Embedded Processors NVMe Host Driver TQ Host Memory SSD SoC Interconnection User Functions SQ CQ Controller DRAM Controller NAND SSD DRAM 9

10 Normal Page Read Request Storage Interface (PCIe / NVMe) Request queue Response queue Host CPU User Applications / Operating Systems SSD Firmware I/O Controller (NVMe command decoder) Translation Layer (FTL) (RD) PPA Summarizer Task Controller NVMe Host Driver RD ( LBA) TQ Host Memory SSD SoC Interconnection User Functions SQ CQ Controller DRAM Controller NAND SSD DRAM 10

11 Normal Page Read Request Storage Interface (PCIe / NVMe) Request queue Response queue Host CPU SSD Firmware Summarizer User Applications / Operating Systems I/O Controller (NVMe command decoder) Translation Layer (FTL) Task Controller NVMe Host Driver RD(PPA 2) RD(PPA 1) TQ Host Memory SSD SoC Interconnection User Functions SQ CQ Controller DRAM Controller NAND Page data SSD DRAM 11

12 Normal Page Read Request Storage Interface (PCIe / NVMe) Request queue Response queue Host CPU User Applications / Operating Systems SSD Firmware I/O Controller (NVMe command decoder) Translation Layer (FTL) Summarizer Task Controller NVMe Host Driver TQ Host Memory SSD SoC Interconnection User Functions SQ CQ Controller DRAM Controller NAND SSD DRAM Page data 12

13 Summarizer Initialization (Function Offloading) Storage Interface (PCIe / NVMe) Request queue Response queue Host CPU User Applications / Operating Systems SSD Firmware I/O Controller (NVMe command decoder) Translation Layer (FTL) Summarizer Task Controller New NVMe command NVMe Host Driver INIT ( foo) TQ Function registration Host foo() Memory SQ CQ SSD SoC Interconnection Controller Function DRAM offloading Controller NAND SSD DRAM User Functions f#1 foo() 13

14 Summarizer Computation (Dynamic mode) Storage Interface (PCIe / NVMe) Request queue Response queue Host CPU User Applications / Operating Systems New NVMe SSD command Firmware decode I/O Controller (NVMe command decoder) Translation Layer (FTL) RD&PROC(PPA,foo) Summarizer Task Controller New NVMe command NVMe Host Driver RD&PROC( LBA,foo) TQ Host Memory SQ CQ SSD SoC Interconnection Controller DRAM Controller NAND SSD DRAM User Functions f#1 foo() f#2 goo() 14

15 Summarizer Computation (Dynamic mode) Storage Interface (PCIe / NVMe) Request queue Response queue Host CPU User Applications / Operating Systems SSD Firmware I/O Controller (NVMe command decoder) Translation Layer (FTL) RD&PROC(PPA,foo) Summarizer Task Controller NVMe Host Driver RD&P(PPA2,foo) RD&P(PPA1,foo) RD&P(PPA1,foo) TQ Host Memory SQ CQ SSD SoC Interconnection Controller DRAM Controller NAND Page data SSD DRAM User Functions f#1 foo() f#2 goo() 15

16 Summarizer Computation (Dynamic mode) Storage Interface (PCIe / NVMe) Request queue Response queue Host CPU User Applications / Operating Systems NVMe Host Driver SSD Firmware I/O Controller (NVMe command decoder) Translation Layer (FTL) RD&PROC(PPA,foo) RD&P(PPA1,foo) CC/Proc Summarizer Task Controller Register in TQ buf1, foo TQ Host Memory SQ CQ SSD SoC Interconnection Controller NAND DRAM Controller SSD DRAM Page data User Functions f#1 foo1() f#2 goo() 16

17 Summarizer Computation (Dynamic mode) Storage Interface (PCIe / NVMe) Request queue Response queue Host CPU User Applications / Operating Systems NVMe Host Driver SSD Firmware I/O Controller (NVMe command decoder) Translation Layer (FTL) RD&PROC(PPA,foo) RD&P(PPA1,foo) CC Summarizer Task Controller TQ is full TQ Host Memory SQ CQ SSD SoC Interconnection Controller NAND DRAM Controller SSD DRAM Page data User Functions f#1 foo() f#2 goo() 17

18 Summarizer Finalization Storage Interface (PCIe / NVMe) Request queue Response queue Host CPU User Applications / Operating Systems SSD Firmware I/O Controller (NVMe command decoder) Translation Layer (FTL) Summarizer Task Controller New NVMe command NVMe Host Driver FINAL ( foo) TQ Host Memory SQ CQ SSD SoC Interconnection Controller NAND DRAM Controller SSD DRAM Results User Functions f#1 foo() f#2 goo() 18

19 Summarizer API and NVMe commands Initialization NVMe command: INIT_TSKn Transfer a in-ssd procedure to SSD memory Initialize data structure and temporal variables for in-ssd computation Computation NVMe command: READ_PROC_TSKn Page read command is issued with the flag indicating the user procedure embedded in SSD memory Return the special code if the requested page is processed in SSD Page data is transferred to the host if the requested page is NOT computed in SSD Finalization NVMe command: FINAL_TSKn Gather final in-ssd computation results and transfer to the host 19

20 PCIe (host LS2085a) PCIe (LS2085a - FPGA) Evaluation Platform LS2085a intelligent SSD development platform ARM cores running FTL and Summarizerfirmware FPGA implementing NAND flash controller PCIe Gen. 3 4x lanes for host communication LS2085a CPU CPU CPU CPU L1D (32KB) L1I (48KB) L1D (32KB) L1I (48KB) L1D (32KB) L1I (48KB) L1D (32KB) L1I (48KB) L2 (1MB) Interconnection L2 (1MB) FPGA (ALTERA Stratix V) DDR4 Memory Controller DRAM DRAM NAND NAND flash flash DIMM DIMMs 20

21 Evaluation - Performance 4 SDD time TPC-H Query6 Host time Static workload offloading Static Dynamic 21

22 Evaluation - Performance 4 SDD time TPC-H Query6 Host time CPU only processing (baseline) SSD only processing Static Dynamic 22

23 Evaluation - Performance 4 SDD time TPC-H Query6 Host time Summarizer Dynamic Offloading Static Dynamic 23

24 Evaluation - Performance 4 SDD time TPC-H Query6 Host time SSD processing + transfer time 3 (internal + external + In-SSD processing) Host CPU processing time Static Dynamic 24

25 Evaluation - Performance 4 TPC-H Query6 Execution time normalized to baseline (CPU only) SDD time Host time Static Dynamic 25

26 Execution time (normalized to baseline) Evaluation - Performance 4 SDD time TPC-H Query6 Host time Static Dynamic 26

27 Evaluation - Performance Execution time (normalized to baseline) SDD time 0.30 Chart Title SDD TPC-H timequery6 Host time Host time CPU only Static Dynamic Dynamic 27

28 Evaluation - Performance W/O NDP With NDP SDD time Host time Chart Title SDD TPC-H timequery6 Host time Performance improved by 14% 0.24 External (host storage) Data host Data transfer from storage 0.62 Data storage Internal CPU only Static Dynamic Dynamic 28

29 Evaluation - Performance 4 SDD time TPC-H Query6 Host time 3 Performance degraded by static NDP Static Dynamic 29

30 Evaluation - Performance Execution time (normalized to baseline) Execution time (normalized to baseline) Execution time (normalized to baseline) Execution time (normalized to baseline) 16% 10% 20% 7% 30

31 Design Exploration Higher Internal Bandwidth Host Data transfer bottleneck CPU Storage interface Commercial SSD maintains internal bandwidth external bandwidth 31

32 Design Exploration Higher Internal Bandwidth Host Data transfer bottleneck CPU Storage interface SP Higher internal bandwidth without increasing external bandwidth 32

33 Design Exploration Higher Internal Bandwidth Speedup 100% 80% 60% 40% External : Internal bandwidth ratio 20% 0% 1:1 1:2 1:3 1:4 1:1 1:2 1:3 1:4 1:1 1:2 1:3 1:4 1:1 1:2 1:3 1:4 1:1 1:2 1:3 1:4 TPC-H Query 6 TPC-H Query 1 TPC-H Query 14 String Similarity Join Average 33

34 Design Exploration Higher Internal Bandwidth Speedup Summarizer is effective if an SSD platform has higher internal bandwidth 100% 80% 60% 40% 20% 0% 1:1 1:2 1:3 1:4 1:1 1:2 1:3 1:4 1:1 1:2 1:3 1:4 1:1 1:2 1:3 1:4 1:1 1:2 1:3 1:4 TPC-H Query 6 TPC-H Query 1 TPC-H Query 14 String Similarity Join Average 34

35 Design Exploration Better SSD Processor Host CPU Storage interface AP Better embedded processor is cost effective 35

36 Design Exploration Higher Internal Bandwidth Speedup 120% 100% 80% 60% 40% Embedded processor performance 20% 0% X1 X2 X4 X8 X16 X1 X2 X4 X8 X16 X1 X2 X4 X8 X16 X1 X2 X4 X8 X16 X1 X2 X4 X8 X16 TPC-H Query6 TPC-H Query1 TPC-H Query14 String Similarity Join Average 36

37 Design Exploration Higher Internal Bandwidth Speedup Summarizer is a cost effective NDP solution with powerful storage processors 120% 100% 80% 60% 40% 20% 0% X1 X2 X4 X8 X16 X1 X2 X4 X8 X16 X1 X2 X4 X8 X16 X1 X2 X4 X8 X16 X1 X2 X4 X8 X16 TPC-H Query6 TPC-H Query1 TPC-H Query14 String Similarity Join Average 37

38 Conclusion Dynamic computation offloading framework Dynamic NDP framework for SSDs Opportunistic in-ssd computation Page-level task control Optimal performance improvement Opportunistically enables in-ssd processing Page-level NDP control Automatic workload partitioning Summrizer programming model Summarizer programming model Evaluation on the real development platform Explored design space for future SSDs 38

39 Thank you Summarizer: Trading Communication with Computing Near Storage Gunjae Koo, Kiran Kumar Matam, Te I, H. V. Krishna Giri Nara, Jing Li, Hung-Wei Tseng, Steven Swanson, Murali Annavaram (We thank to Dell EMC for supporting the SSD development board)