CSAL: A CLOUD STORAGE ABSTRACTION LAYER TO ENABLE PORTABLE CLOUD APPLICATIONS

Size: px

Start display at page:

Download "CSAL: A CLOUD STORAGE ABSTRACTION LAYER TO ENABLE PORTABLE CLOUD APPLICATIONS"

Noreen Rice
6 years ago
Views:

1 CSAL: A CLOUD STORAGE ABSTRACTION LAYER TO ENABLE PORTABLE CLOUD APPLICATIONS Zach Hill & Marty Humphrey Dept. of Computer Science, University of Virginia zjh5f@cs.virginia.edu

2 A Cloud Application User Requests Front-End Queue Service Table Service Object/ Blob Service 2

3 Many clouds, many code versions Storage Services Front-End Blobs Tables Queues Front-End Storage Services 3

4 Single code version, many clouds CSAL Storage Services Front-End CSAL CSAL Blobs Tables CSAL Queues Front-End CSAL Storage Services 4

getitem QueueStore createqueue deletequeue getmessage putmessage peekmessage deletemessage Blob Namespace

5 CSAL Overview Application CSAL Metadata Manager Service Manager BlobStore createcontainer listcontainers deletecontainer getblob putblob deleteblob TableStore createtable deletetable Insert Update Delete Query getitem QueueStore createqueue deletequeue getmessage putmessage peekmessage deletemessage Blob Namespace Table Namespace Q Namespace S3 Plugin SimpleDB Plugin Azure Queue Plugin Metadata Store table service(s) 5

6 CSAL Namespaces One namespace for each abstraction type Metadata only for containers Service endpoint, identifier, user credentials Each abstraction has an independent metadata store Metadata caching Container ops are not very common If data is stale, simply re-fetch and retry 6

CSAL Namespaces 2. Lookup X metadata cache first 1. Call getblob( X, foo ); CSAL BlobStore Metadata Manager Plugin Cache 5. Access foo in X in Service1 use 3.

7 CSAL Namespaces 2. Lookup X metadata cache first 1. Call getblob( X, foo ); CSAL BlobStore Metadata Manager Plugin Cache 5. Access foo in X in Service1 use 3. Retrieve from table if not in cache 4. Use metadata to determine plugin to X X foo Y Service1 Service3 Service2 ( X, testusr, testkey, ) 7

8 CSAL Implementation Client-side java library i.e. BlobStore.getBlob( Container, foo ); Metadata backing store in the cloud Currently supports Azure & AWS storage Both SOAP and REST 8

9 Performance of CSAL Adding software layers isn t free Compare CSAL to Azure s and Amazon s SDK APIs Set of micro-benchmarks to test operation latency Container Ops and Data Ops Expect a slowdown for container ops due to metadata 9

10 Performance Container Ops 1.6 CSAL in AWS 1.6 CSAL in Windows Azure Median Operation Time in Seconds Median Operation Time in Seconds Create Container Create Table Queue Create 0 Create Container Create Table Queue Create CSAL Total Time Native API CSAL Total Time Native API Core Op Time Metadata Op Time Core Op Time Metadata Op Time Note: Error Bars indicate 1 Standard Deviation 10

11 Performance Data Ops 0.4 CSAL in AWS 0.4 CSAL in Windows Azure Median Operation Time in Seconds Median Operation Time in Seconds CSAL Total Time Native API CSAL Total Time Native API Note: Error Bars indicate 1 Standard Deviation 11

12 What about Standards? Standards Efforts OCCI OVF Standards take time to develop and are resisted by vendors Multi-cloud APIs SimpleCloud, jclouds, DeltaCloud, LibCloud SAGA 12

13 Future work What if Cloud X doesn t have tables/blobs/queues? Map one abstraction to other (i.e. filesystem) 3 rd party services: Hbase, HyperTable, Cassandra Placing, replicating, and migrating data in real-time for performance and/or cost Real-world applications such as multi-cloud MR 13

14 Summary Application lock-in and portability are problems in clouds Standards are great, but don t hold your breath just yet CSAL provides storage abstractions to make the application code itself portable with little performance impact for common data operations 14

15 Questions? 15

HPSS Treefrog Introduction.

HPSS Treefrog Introduction. HPSS Treefrog Introduction Disclaimer Forward looking information including schedules and future software reflect current planning that may change and should not be taken as commitments by IBM or the other