National Data Store 2 Secure Storage Cloud with efficient and easy data access

Transcription

1 National Data Store 2 Secure Storage Cloud with efficient and easy data access Michał Jankowski, Maciej Brzeźniak Supercomputing Dept. of PSNC, Authors: NDS2 team at PSNC and partners full list of credits at the end of presentation Project funded by: NCBiR for under KMD2 project (no. NR /2011) FullPolishnameof theproject: System bezpiecznego przechowywania i współdzielenia danych oraz składowania kopii zapasowych i archiwalnych w Krajowym Magazynie Danych Project partners 10 Polish universities and supercomputing centres:

2 NDS2: Presentationplan Requirements: Secure, reliable and efficient data storage with easy and transparent interface Problem: Requirements are contradictory Our solution: Client side on-the fly encryption Key echange hierarchy for data sharing Demonstration: Software client screenshots End-user devices Discussion

3 NDS2: usecasesand requirements 3

4 NDS2: usecases 1. Individual user(scientist, researcher, student): Individual account on the remote storage Data to be available, persistent and safe Easyand efficientaccessto data fromvariousoss Transparent safety and security mechanisms Shareand publishthedata Data storage and access DATA REMOTE STORAGE Data sharing and publication DATA SECURED and REPLICATED DATA

5 NDS2: usecases 2. Institution, workgroup (digital library, scientific project) Use local, possibly existing accounts The data must be available, persistent and safe Local working space with simple and efficient access through typical LAN protocols(cifs, NFS) Local space should be extended by a remote space remotespaceisa backup of local,localiscacheof remote, etc. Local storage and access Remote backup and retrieval DATA CIFS/samba LOCAL STORAGE SFTP REMOTE STORAGE DATA SECURED and REPLICATED DATA

6 NDS2: problem definition 6

7 NDS2:secure, efficient and easy to use To provide in the same time: Data safety and security Efficiency of data storage and access Transparency of safety and security mechanisms Sharing support Support for data publication These are contradictory!

8 NDS1 and PLATON Archiving Services NDS project( ): User National Data Store Distributed, replicated storage Server side Virtual Filesystem in user space (Linux) Standard access protocols Metadata DB Users DB DB Node Accounting & limits DB Access Methods Servers (SSH, HTTPs, WebDAV...) VFS for data and meta-data NDS system logic Replica access methods servers (NFS, GridFTP) Replication Storage Node Access Node Storage Node Automatic replication FS with data migration (HSM) HSM system (NFS) NAS appliance PLATON ( ) PLATON Archiving Service Production deployment of NDS foracademicsin Poland connected to NREN 10 sites in Poland Tapes: 12+ PB in 5 sites Disks: 2+ PB in 10 sites

9 NDS2 a secure and extended NDS NDS and PLATON experience: Replication, data persistency etc. OK! Encryption (privacy) and integrity control needed! Manual implementation too complicated Existing tools not good enough System shoud better integrate with user s system (Win, Linux, mobiles ) and institution/ workgroup environment NDS2( ): National Data Store 2: End-to-end encryption& integrity control Easy and efficient data exchange Virtualdisksfor Windows, Linux Appliance for institutions Portable GUI client for individuals Mobile clients

10 NDS2 cryptography: encryption, integrity control and keys management 10

11 NDS2 cryptography: algorithms Overall concept: Data encrypted with AES-256 CTR AES - Strong and high performance algorithm for bulk data, resistant to brute force attacks Hardware supported: Intel Westmere and ARMv8 Performance: 1-2,5 GB/s on modern workstations CTR mode enables parallelism Integrity controlled using SHA-512: Resistant to collisions and attacks Calculated: User-side (per 64kB logical block) in order to enable users to detect manipulations or corruption on data or their digest System-side (per file) for replica integrity control

12 NDS2 cryptography: keysmanagement Proper key granulation is the clue! Separatekey for eachfile vsone key for the whole system Key hierarchies for easy and efficient sharing Separate key per file but: Exchange keys on directory or user group level The exchange is transparently done while managing dir access or group Small-scale sharing keys hierarchy Large-scale sharing keys hierarchy

13 NDS2 cryptography: moredetails More details: NDS2 paper submitted to TNC2013

14 NDS2 cryptographicclients: Features& concept: Implementation, 14

15 File system likeclients easy! Windows users CryptoFS 4Windows FS-like access (.net CallbackFS) Encryption & digests (.net crypto API) SFTP client (Rebex SFT library) Linux users CryptoFS 4Linux SSHFS extension FS-like access (FUSE/SSHS) Encryption & digests (OpenSSL) SFTP client (Built-into SSHFS) Workgroups Appliance LAN access through CIFS (Samba server) FS-like access (CryptoFS 4 Linux) Admin. interface (php+ perl) Local disk storage WAN (SFTP) NDS filesystem (extended with versioning, ACLs) SECURED and REPLICATED DATA

16 Dedicatedclients -evenmorefromnds2! Any platform Java GUI, CLI and API Browser-like access Drag & dropp Encryption & digests (Bouncy Castle) SFTP client (JSCH sftp) Metadata search Mobile platform mmm Android client Browser-like access Encryption & digests (Java API) SFTP client (Java API) Integration with Android shell WAN (SFTP) SECURED and REPLICATED DATA

17 NDS2 appliance concept LAN NDS2appliance Local disk space Users Data access & sharing (CIFS) SMB/CIFS server MGMT interface (web) Remote storage/ backup space Backup/ restore Data access + encryption WAN LDAP/ Active Directory server Appliance admin Private cloud Public cloud

18 NDS2: appliance implementation Physical appliance: Box for small groups/ instiutions Small and silent, green; fits below the desk! CPUs with AES-NI support Up to 2TB internal storage (2.5 HDDs) or green SSDs Cheap! up to 500EUR including storage Low power! Rack server for bigger institutions Rack server (1U, 19 ) with more disk capacity: CPUs with AES-NI Up 16 TB of RAW storage (4x 4TB HDD) Intermediate costs - ~2000EUR SC and SD cards for keys storage Virtual appliance: VM running on clients s hardware Hardware aid for keys protection Smart cards and SD cards for keys storage / backup SC and SD readers built into appliance Key backup/restore wizards / feature in appliance Virtual machine: Runnable on client virtualisation platform or workstation, no assumptions on hardware Need some local storage and CPU cores for encryption

19 NDS2 cryptographicclients: Demonstration 19

20 NDS2: Demonstration Here we show screenshots Full demonstration available at PSNC/PIONIER booth: Demonstration: Windows and Linux clients Android clients Appliance based on compact ASRock Vision HT box Location and time: PSNC boothbox nr. 14 During coffee breaks ontuesday15:30-16:00 and Wednesday 10:30-11:00 Please contact me at PSNC booth or by: You are welcome to try out our demo and discuss solution

21 NDS2: ndscryptofs4windows Login screen: Login name Login certificate containing a private key for authentication Server connection details Certificatecontaining4kB-long RSA key pair for data encryption Remote storage space visible and accessible as a local drive

22 NDS2: ndscryptofs4linux (1) Original directory content (user view) Encrypted directory content (server view)

23 NDS2: ndscryptofs4linux (2) Original file content (user view) Encrypted file content(server view)

24 NDS2: GUI(1) NDS2/SFTP Server connection details: Server name Server port Login screen: Login name Private RSA key for authentication Server connection details 4kB-long RSA keys pair for data encryption Needs localisation

25 NDS2: GUI (2) GUI client: supports Drag & Drop builds the upload jobs database if many files are dropped enables to monitor status of these jobs, pausing/resuming them etc. Metadata search XATTR, ACL support

26 NDS2: GUI (3) GUI client: Data are encrypted and integrity-controlled in the encrypted directory Remaining data are stored unencrypted Progress bars monitor upload/download status

27 NDS2: Android client

28 NDS2: Appliance (1) Appliance administration interface NDS2 (or SFTP server) connection configuration Network settings configuration Internal appliance disks/ RAIDs configuration

29 NDS2: Appliance (2) Appliance: end-user experience Data storedinnds2/sftp server Accessible through appliance and CIFS protocol Network share defined in appliance Access to data fromtheend-userworkstation remotestoragespaceaccessiblethroughcifs and NDS2 appliance

30 Discussion 30

31 NDS2: cryptographic filesystems &clients POSIX-like, local drive-like access: Natural interface for users Compatible with many applications Works on most popular OSs All versions of Windows Linux most distributions GUI clients: All NDS2 features Portable (Java, Android) Achieved High data security(persistency and privacy) Transparency and easy of use Efficiency(localspace, CPU) Possible improvements: Client-side caching Integration with sync & share applications (e.g. Dropbox)

32 NDS2: cryptographic appliance Network share interface Natural interface for regular users, workgroups etc. Compatible with many applications Different spaces: local/remote encrypted/unencrypted Achieved High data security(persistency and privacy) Transparency and easy of use Efficiency: lowoverhead on enduser resources Easy integration with existing customer infrastructure Possibility to offer the solution in various sizes and models (small-big appliance, virtual machine) Possible improvements: Intelligent caching HSM-like

33 National Data Store 2 crypto-clients - demonstration Thank you! Project funded by: NCBiR for under KMD2 project(no. NR /2011) Project partners 10 Polish universities and supercomputing centres:

34 National Data Store 2 crypto-clients - demonstration PSNC team: Maciej Brzeźniak, Gracjan Jankowski, MichałJankowski, Staszek Jankowski, Tomasz Jankowski Adam Zawada Sławomir Zdanowski Rafał Mikołajczak Credits: Partners: Tomasz Chmiel, Łukasz Kuczyński, Michał Major, Łukasz Redynk, Kamil Guryn, and others Project funded by: NCBiR for under KMD2 project(no. NR /2011) Project partners 10 Polish universities and supercomputing centres:

35 NDS2 crypto filesystems vs EncFS, Boxcryptor etc. Boxcryptor& EncFS similarities Boxcryptor(Win, ios, Android) supports EncFS data format Data in the cloud protected on-the-fly, local disk-like access Differences: Functionality: BC and EncFSstoredata inthelocalfilesystem, transfer to cloudout of scope EncFS+ SSHFS wouldrequirei/o to traversetwovfss, similarproblem for BC Security and sharing: Feature NDS2 Box cryptor/ EncFS File encryption algorithm / keytype Symmetric (AES 256 CTR) Symmetric (AES 256) Key usage Generated per-file Common for all files File name encryption Shared data encryption Symmetric(AES 256) keyderived from dir s asymmetric private key Per-directory asymmetric key,encrypted with private users key or group key Common for data and names Common key for every user no fine-grained keys management

36 National Data Store 2 crypto-clients - demonstration Backup slides Project funded by: NCBiR for under KMD2 project(no. NR /2011) Project partners 10 Polish universities and supercomputing centres:

37 Safe data sharing & publishing Secure sharing Sharing with other NDS2 users Very high level of security: symm. and asymm. Key handling combination (more elaboration elsewhere) Secure publishing and import/export from/to World Similar to get file link on Dropbox Works in both directions It s safer than with Dropbox Trust Safe key exchange Safe key exchange NO Trust Data access NO Trust Data access & storage Data access import/ export publication NDS2 sandbox

38 Requirements addressed in NDS and by PLATON Archiving Services Features Comments Data availability& persistency Replication, consistency checks Basic security Data sharing within a group Easy and efficient access Safery mechanisms transparent & scalable Transfer encryption, access control at rest, policies(media degaussing) Server-side Linux filesystems mechanisms for access control Access to data through number of protocols: SFTP, WebDAV, Web GUI, GridFTP First replica created by user using a convenientprotocol; then replicas created async. using GridFTP for efficiency in WAN

39 NDS2 Features End-to-end security: privacy & integrity Easy access, safety mechanisms transparency Rollback and versioning Easy, efficient and secure data exchange Mobile access Efficient & easy local storage & access + remote backup Comments AES-256 for data, RSA for key exchange, SHA digests Client-side cryptograpghy provided by easy to use clients: virtual filesystems, Java GUI Server-side versioning + support in clients Symm. and assym. key hierarchy, key exchange mechanisms Android application Appliance for institutions

40 NDS2 cryptography: encryption, integrity control and keys mgmt (2) Client-side encryption and integrity control: AES 256 CTRgenerated per file for data privacy Stored in the file header on the system side Protected with user s private RSA key => User takes care of only 1 pair of keys SHA-512 digests calculated per logical 64-byte block Stored with each block on the system side Protected by encryption using files symetric key => User application may access digest information using file s AES key

41 Clients for NDS2(3) Dropbox-like functionality Experimental data sync client 4 Windows: Based on CryptoFS4Win components MS Synchronisation Framework for synchronisation WAN (SFTP) SECURED and REPLICATED DATA