SCFS: A Shared Cloud-backed File System

Transcription

1 SCFS: A Shared Cloud-backed File System Alysson Bessani, Ricardo Mendes, Tiago Oliveira, Nuno Neves, Miguel Correia, Marcelo Pasin*, Paulo Veríssimo University of Lisbon - Portugal * now at University of Neuchatel - Switzerland 1

2 Cloud-backed Storage Personal Storage Services (e.g., DropBox, OneDrive) DropBox System Amazon EC2 and S3 Cloud-backed File Systems (e.g., BlueSky, S3FS) Proxy Cloud Storage 2

3 State of the Art 3

4 Shared Cloud-backed File System (SCFS) Client-based Use existing cloud services Cloud Storage DATA DATA Pay-per ownership Each client pays for its own files SCFS Agent SCFS Agent SCFS Agent Strong Consistency Controlled sharing Access control for security and concurrency Redundant Cloud Services 4

5 SCFS Design 5

6 Design Choices Data layout/access pattern Each file is an object (single-block file) Multiple versions of the files are maintained Always write, avoid reading (exploiting free writes) Cache Persistent file cache Local storage is used to hold copies of all/most client files Opened files are also maintained in main-memory Short lived main-memory metadata cache To deal with bursts of metadata requests 6

7 Design Choices Consistency Consistency-on-close semantics Control of durability and consistency Locks used to avoid write-write conflicts Modular coordination Separate data from metadata Metadata is stored in a coordination service E.g., Zookeeper [ATC 10], DepSpace [EuroSys 08] Also used for managing file locks 7

8 SCFS Architecture Coordination Service Lock Service Access Control Metadata Computing clouds SCFS Agent Cache SCFS Agent Cloud storage Storage clouds Cache SCFS Agent Cache 8

9 Consistency 9

10 Consistency Anchor Problem: How to provide strong consistency on top of weak consistent storage clouds? Composite Storage write Algorithm read strong consistency write read write weak consistency Storage Service strong consistency Consistency Anchor read Key property: the composite storage consistency is the same of the consistency anchor 10

11 Consistency Anchor in SCFS If x is cached, it is only validated! READ fd = open( x, );!! Local Storage Coordination read(fd, );! write(fd, );! 1 2 Service Main Memory! 2 fsync(fd);! Cloud(-of-Clouds)! write(fd, );!! close(fd);! Persistent Storage WRITE 1! 11

12 Implementation and Evaluation 12

13 SCFS Backends SCFS can use different backends i.e., different cloud storage and a coordination service plugin EC2( DS% DS% DS% DS% DS% RS( SCFS% Agent% AWS%Backend% S3( BFT$SMaRt* SCFS% Agent% DepSky* CoC%Backend% Operation: blocking, non-blocking and non-sharing WA( GS( S3( 13

14 The Cloud-of-Clouds Backend Does not require trust on any single cloud provider SCFS works correctly as long as less than a third of the providers misbehave DepSpace/BFT-SMaRt [EuroSys 08, DSN 14] SCFS Agent Cache DepSky CoC Storage [ACM ToS 13] 1. gen. key File Data Client% 2. encrypt 4. secret sharing 3. erasure coding Storage%Services% RS( 1 WA( Figure 6: A write in SCFS using the DepSky protocols. 2 GS( 3 S3( 4 14

15 Sharing Latency: SCFS vs DropBox DATA Rackspace Files #! Google Storage?$ DATA DATA Latency (s) Windows Azure Blob % Amazon S3 90% value 50% value CoC-B CoC-NB AWS-B AWS-NB Dropbox CoC-B CoC-NB AWS-B AWS-NB Dropbox DATA CoC-B CoC-NB AWS-B AWS-NB Dropbox 256K 1M 4M 16M Data Size CoC-B CoC-NB AWS-B AWS-NB Dropbox Non-blocking Blocking 15

16 Benchmarking (Unmodified) Desktop Applications Latency (s) OpenOffice Writer Open Action: 1 open(f,rw), 2 read(f), 3-5 open-write-close(lf1), 6-8 open-read-close(f), 9-11 open-read-close(lf1) Save Action: 1-3 open-read-close(f), 4 close(f), 5-7 open-readclose(lf1), 8 delete(lf1), 9-11 open-write-close(lf2), openread-close(lf2), 15 truncate(f,0), open-write-close(f), open-fsync-close(f), open-read-close(f), 25 open(f,rw) Close Action: 1 close(f), 2-4 open-read-close(lf2), 5 delete(lf2) Figure 7: File system operations invoked in the file synchro- 11 CoC read/writes Open Open Save Save Close Close. S3QL S3QL(L). CoC-NS CoC-NS(L). CoC-NB CoC-NB(L) AWS-NB(L) AWS-NB AWS CoC CoC (NS) Non-blocking S3QL Latency (s) CoC-B. AWS-B(L) AWS-B 55% 40% 80% S3FS. CoC-B(L) AWS CoC S3FS Blocking lock files S3FS(L) 16

17 Financial Evaluation VM Instance EC2 EC2 4 CoC Capacity Large $6.24 $24.96 $ M files Extra Large $12.96 $51.84 $ M files (a) Operation costs/day and expected coordination service capacity. Cost/op (microdollar) read CoC read AWS read CoC write AWS write write File size (MB) (b) Cost per operation (log scale). Cost/day (microdollar) CoC AWS File size (MB) (c) Cost per file per day. +50% 17

18 Wrap Up SCFS is a cloud-backed file systems that can be used for backup, disaster recovery and sharing data Key design principles: Always write, avoid reading (very cheap in terms of $$$) Strong consistency (despite storage cloud weak consistency) Experience so far Multi-cloud replication is feasible (CoC not slower!) This is a case for BFT crash-only solution will not make things better Being employed for sharing dataset metadata among Biobanks 18

19 Thanks! SCFS code available at DepSky and DepSpace/BFT-SMaRt also available