Malacology. A Programmable Storage System [Sevilla et al. EuroSys '17]

Transcription

1 Malacology A Programmable Storage System [Sevilla et al. EuroSys '17] Michael A. Sevilla, Noah Watkins, Ivo Jimenez, Peter Alvaro, Shel Finkelstein, Jeff LeFevre, Carlos Maltzahn University of California, Santa Cruz

2 Narrow Waist to Storage Redundant Functionality My App Storage System 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 2

3 Narrow Waist to Storage Redundant Functionality My App consensus atomic operations batching migration Storage System 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 3

4 Problem Our solution Roadmap Implementation Evaluation Programmable Storage Systems expand narrow waist to storage 2. expose internal functionality as interfaces 3. simplify the construction of new services 4. facilitate app-specific stacks with less layers 4

5 Programmable Storage Systems expand narrow waist to storage Roadmap 2. expose internal functionality as interfaces 3. simplify the construction of new services 4. facilitate app-specific stacks with less layers 5

6 Motivation: Breaking the Narrow Waist Model User Defined Classes Developers willing to break layers and use non-standard APIs 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 6

7 Exposing Internal Functionality as Interfaces [Not a fixed set of interfaces ] Data I/O Service Metadata File Type Shared Resource Load Balancing durability Storage System 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 7

8 Exposing Internal Functionality as Interfaces [Not a fixed set of interfaces ] My App consensus atomic ops batching migration Data I/O Service Metadata File Type Shared Resource Load Balancing durability storage daemon M monitor daemon M metadata daemons M 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 8

9 Programmable Storage Systems expand narrow waist to storage Roadmap 2. expose internal functionality as interfaces 3. simplify the construction of new services 4. facilitate app-specific stacks with less layers 9

10 Composing Interfaces to Build Higher-Level Services 2X less Source code shipped with Ceph CORFU [Balakrishnan, NSDI'12] Mantle [Sevilla, SC'15] R W Data I/O Shared Resource File Type Service Metadata Load Balancing durability storage daemon M monitor daemon M metadata daemons M 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 10

11 Composing Interfaces to Build Higher-Level Services CORFU R W Data I/O Shared Resource File Type Service Metadata Load Balancing R W Sequencer 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 11

12 Programmable Storage Systems expand narrow waist to storage Roadmap 2. expose internal functionality as interfaces 3. simplify the construction of new services 4. facilitate app-specific stacks with less layers 12

13 Application-Specific Trade-offs for the Sequencer Higher is Better Lower is Better 8 requests have a latency of 700 ms Cost of throughput is latency Knob for configurable performance characteristics 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 13

14 Application-Specific Load Balancing for the Sequencer customized for sequencer workload Load balancing improves performance (free) Customized to be Less Aggressive Programmable load balancing app-specific software stacks 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 14

15 Conclusion Programmable storage less layers, more robust, app-specific Storage System My App consensus transactions batching Principled approach to using interfaces is effective: - CORFU: distributed shared commit log - Mantle: 2X less code, shipped in Ceph [Release] Future work - explore existing services; find proper set of interfaces Programmable Storage System

16 Thank you Questions? Malacology: A Programmable Storage System Michael A. Sevilla, Noah Watkins, Ivo Jimenez, Peter Alvaro, Shel Finkelstein, Jeff LeFevre, Carlos Maltzahn

17 Extra 17

18 Related Work Software defined networks (control/data plane) [Kumar SIGCOMM'13] Software-defined storage FAST'16; Thereska SOSP'13] Policy-based provisioning Whitepaper'15; Gracia-Tinedo IEEE'16] Operating System: application-specific stacks [SOSP'95, OSDI'96; SOSP'95] [Stefanovici, [Carlson, Programmable Storage: adding interfaces to expose functionality to get application-specific stacks; approach and applying it to storage 18

19 Conclusion: Programmable Storage Systems... expand the narrow waist to storage expose interfaces to internal functionality; we built: CORFU: shared commit log Mantle: file system metadata load balancer facilitate building application-specific software stacks

20 Customizable Trade-offs for the Sequencer Higher is Better Lower is Better 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 20

21 Re-Implementing Mantle on Malacology balancing policy.lua File Type Service Metadata Load Balancing durability Mantle New features: 2X src code reduction Shipped in Ceph [Luminous] 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 21

22 Building a Distributed Service ZLog R W Atomic TXNs Sequencer Versioning Load Bal. C Problem 2. Solution 3. Higher-Level Services 4. Evaluation 22

23 Traditional Storage APIs are Convenient but Narrow Traditional Storage Offerings Programmable Storage LIB OBJECT BLOCK FILE Mantle ZLog MALA COLOGY storage daemon M monitor daemon M other daemons M Storage System (e.g., ) 23

24 Storage Systems Common Internal Abstractions Functionality consensus transaction serialization data access migration persistence Ceph M monitor daemon object server daemon other daemons (metadata server in this case) 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 24

25 Storage Systems Common Internal Abstractions Functionality consensus transaction serialization data access migration persistence Ceph M Malacology service metadata data I/O shared resource file type load balancing durability 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 25

26 Example: Service Metadata Interface My App My App My App consensus Service Metadata Interface 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 26

27 Example: Service Metadata Interface authentication logging/debugging configs cluster maps metadata for higherlevel service (e.g., binaries, version numbers) M M M monitor cluster M M Higher-Level Service 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 27

28 Customizable Trade-offs for the Sequencer Good Performance Bad Performance Batch Size Diminishing Returns 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 28

29 Narrow Waist to Storage Redundant Functionality My App Storage System 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 29

30 Today's Applications File, Block, Object Data I/O Shared Resource File Type Service Metadata Load Balancing durability 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 30

31 CORFU Mantle R W File, Block, Object Data I/O Shared Resource File Type Service Metadata Load Balancing durability 1. Problem 2. Solution 3. Higher-Level Services 4. Evaluation 31

32 Narrow Waist to Storage Redundant Functionality My App Storage System 32