Chelonia. a lightweight self-healing distributed storage

Transcription

1 Chelonia a lightweight self-healing distributed storage Zsombor Nagy (zsombor@niif.hu) Salman Toor (salman.toor@it.uu.se) Jon Kerr Nilsen (j.k.nilsen@fys.uio.no)

2 Motivation How to easily... Create a storage resource from available disk space on any computer Connect storage resources through the Internet Provide an easy-to-understand way for users to upload and share their files Use these files in a Grid environment

3 Design The system should be... self-healing easy to deploy easy to maintain lightweight no single point of failure

4 Decisions Files Replication Berkeley DB HA HTTP(S) file transfer SOAP messaging X509 certificates

5 NORDUGRID Grid Solution for Wide Area Computing and Data Handling Chelonia Developed by the KnowARC project Will be maintained by the NorduGrid Collaboration Running in NorduGrid ARC s web service container Written in Python Running on Linux and Mac OS X (and Windows soon)

6 The Chelonia Cloud Storage cloud for anyone Can be created by anyone Can be shared with anyone Can be used by anyone

7 Global namespace Global hierarchical namespace Files are organized in collections All users see the same tree Can use logical names Paths in Chelonia namespace Similar to paths in a regular local filesystem

8 Replication Files are replicated User specifies how many replicas are needed Chelonia itself takes care of the replication Easy to create a new storage node

9 Access Control Access policies to files and collections Can grant access to Individual users If in a Grid environment, to entire virtual organizations

10 Client tools Several ways to access Chelonia Command line interface FUSE module In an ARC enabled Grid environment Through job specification file With ARC client tools

11 FUSE module We have a FUSE module to mount Chelonia as a local filesystem You can upload and download files, create collections (directories) with your OS s CLI and GUI tools (e.g. drag and drop) Linux and Mac OS X

12 Components of Chelonia Consists of four services The Bartender The Shepherd The Librarian The A-Hash Can have multiple instances of all services

13 The Shepherd Manages the storage node with the actual file data Replication requires more storage nodes, each managed by a Shepherd Adding a Shepherd, replica will automatically be created (if needed)

14 The Librarian and A-Hash The Librarian manages the metadata Uses the A-Hash to store the data Both metadata and Librarians can be replicated More fault tolerant Better load balance

15 The Bartender Serves the users by negotiating with the Librarian to query files and collections the Shepherd to upload and download files More than one Bartender to eliminate single point of failure Users can contact any Bartender and get the same result

16 Downloading A-H A-H A-H 2. traverse(ln) B 1. getfile(ln) A-H L 3. file's metadata S 5. TURL 4. get(referenceid) 6. TURL 7. downloading the file USER S E S

17 Uploading A-H 2. traverse(ln) A-H A-H A-H L 3. parent's data 4. new(metadata) 5. file's GUID 6. add entry to parent 8. list of Shepherds 10. register new replica 9. put(...) 11. TURL B S 1. putfile(ln) 12. TURL 13. uploading the file USER 14. report state change S E S

18 Depth test

19 Width test

20 Multiple users

21 Number of replicas Five Shepherds were used for this test User Side ALIVE OFFLINE THIRDWHEEL CREATING 10 files of size 100MB have 3 replicas and now user: Increase the number replicas to 5. Decrease the number of replicas to 2 Number of files in the system Time in seconds taken by the system to achieve the needed replicas System Side 10 files with 4 replicas are in the system. One Shepherd went OFFLINE for sometime Became ONLINE Number of files in the system Time in seconds taken by the system to achieve the needed replicas

22 Memory usage 8 days running with lots of file uploads and downloads

23 Better clients? SRM? WebDAV? Encryption? AAI? Transfer protocols? Transfer security? Versioning? Resuming uploads? SOAP over HTTPS? Space reservation? Future?

24 ClusterGrid Maintained by NIIF, Hungary Since 2002 connecting a few hundred PCs located at several universities in Hungary Previously using NIIF s own grid middleware Soon it will use ARC and Chelonia

25 Chelonia and ClusterGrid Soon there will be a purchase of some storage solutions which will be deployed at chosen Hungarian universities We will deploy Chelonia storage nodes on these The ClusterGrid users will be able to upload and download (input/output) files to the Chelonia cloud and refer to these files in the job descriptions

26 Questions / Demo

27 Thank You! web: paper: video: Zsombor Nagy, NIIF (Hungary),