A Dropbox-like Personal Cloud for OpenStack Swift

Transcription

1 A Dropbox-like Personal Cloud for OpenStack Swift Pedro García López Adrián Moreno Martínez Cristian Cotes González OpenStack Summit May Atlanta

2 CloudSpaces project Open Service Platform for the Next Generation of Open Personal Clouds: Objectives: o Partners: o Interoperability and privacy URV, Eurecom, EPFL, eyeos (Web Desktop), TISSAT (cloud provider), NEC.

3 Personal Cloud We understand it as Storage Synchronization Sharing

4 Our motivations Not a cloud open-source solution available StackSync and OpenStack Users lack control of their data Privacy, Client-side encryption, secure sharing Vendor lock-in Interoperability

5 Big picture

6 First stage Metadata separated from data StackSync Server OpenStack Swift Metadata DB StackSync Server Metadata flow OpenStack Swift Data flow Desktop client

7 Second stage MOM (RabbitMQ) push notifications Elastic sync protocol Metadata DB StackSync StackSync Server StackSync Server Server RabbitMQ Metadata flow (Push) OpenStack Swift Data flow Desktop client

8 Current stage API as a Swift s proxy module Website Android app Metadata DB StackSync StackSync Server StackSync Server Server RabbitMQ Metadata flow OpenStack Swift StackSync API Metadata flow (push) Data flow Desktop client Mobile client

9 Lifecycle and architecture

10 Lifecycle StackSync StackSync Server StackSync Server Server RabbitMQ OpenStack Swift Client 3 Client 1 Client 2

11 Desktop client Metadata DB StackSync Server Metadata flow RabbitMQ OpenStack Swift StackSync API Metadata flow (push) Data flow Desktop client Mobile client

12 Tasks File system watcher Chunking, compression, and encryption

13 Features File versioning File sharing OS integration Overlay icons Notifications

14 StackSync server Metadata DB StackSync Server Metadata flow RabbitMQ OpenStack Swift StackSync API Metadata flow (push) Data flow Desktop client Mobile client

15 Operations getaccount getchanges commit { } "is_folder":false, "mimetype":"application/pdf", "status":"new", "parent_file_id": , "file_id": , "path":"/documents/", "filename":"stacksync_guide.pdf", "checksum": , "user":"john Doe", "client_modified":" :36:41.997", "size":775412, "version":1, "server_modified":" :36:41.997"

16 Syncing conflicts Two users modify the same file at the same time Whichever version is processed first wins The loser creates a conflicted copy of the file with his changes User 1 StackSync Server User 2

17 Workspace Represents the relation between files and users Every user has its own workspace Every shared folder has its workspace A workspace maps into a Swift container User 1 User 2 Workspace 1 Shared workspace Workspace 2 Container 1 Shared container Container 2 OpenStack Swift

18 Encryption settings Plain Server-side encryption Client-side encryption

19 Swift mappings StackSync OpenStack Swift installation user workspace tenant user container admin tenant admin file N chunks

20 Sharing process 1. Send share proposal 2. Notify the addressee 3. Proposal gets accepted 4. Create container and set up ACL X-Container-Read and X-Container-Write OpenStack Swift StackSync Server User 1 User 2

21 RabbitMQ Metadata DB StackSync Server Metadata flow RabbitMQ OpenStack Swift StackSync API Metadata flow (push) Data flow Desktop client Mobile client

22 RabbitMQ Communication between clients and server Push notifications Load-balancing Multicast messages per workspace

23 StackSync StackSync StackSync Server Server Server

24 StackSync API Metadata DB StackSync Server Metadata flow RabbitMQ OpenStack Swift StackSync API Metadata flow (push) Data flow Desktop client Mobile client

25 StackSync API Located in the proxy s pipeline Used by website & mobile clients Communicates with StackSync server to get metadata and authorization Activates when the X-StackSync-API header is set

26 StackSync API Auth Module located before the API OAuth 1.0a implementation Sets the user information in the wsgi environment for the API

27 GET /api/file/ Header: X-StackSync-API: v1

28 Performance

29 Traffic overhead

30 Push vs pull

31 Server elasticity Number of instances Number of requests Time (minutes)

32 Using StackSync

33 Deployment scenarios On premises On cloud provider Private Hybrid Public OpenStack Swift OpenStack Swift OpenStack Swift StackSync Server StackSync Server StackSync Server

34 Current deployments Universitat Rovira i Virgili Tissat (Data center) Rediris (Spanish university network)

35 Community Available for: Documentation available at stacksync.org Easy to deploy for swifters

36 Conclusions StackSync is a ready-to-use Personal Cloud for OpenStack Swift What s next Advanced privacy features Interoperability ios and OS X clients

37 Thanks for your attention! stacksync.org github.com/stacksync