DataDirect Networks Japan, Inc.

Transcription

1 Workshop on Container Technologies for Convergence of HPC and AI/Big data Analysis Data resource and Security Management for Docker Container on Parallel Filesystem 2017/07/10 DataDirect Networks Japan, Inc. Senior Manger, Performance & Lustre Engineering Shuichi Ihara

2 2 About DDN History Founded in 98 World s Largest Private Storage Company Double Digit Growth, Profitable, Self Funded Headquarters Santa Clara and Chatsworth, CA Intersect360 Research s Top of All Things in HPC Survey World-Renowned & Award Winning

3 3 DDN Global Presence

4 4 DDN HPC & Big Data at Scale Supercomputing Web, Cloud & Telco Financial Services Weather/Climate Oil & Gas Life Sciences Analytics Manufacturing

5 5 UseCase DDN Leverages Machine Learning at Scale Largest customer as a single system Started 15PB deployment and already expanded ~200PB with multiple Lustre filesystem in 6 months Inject ~100TB+/day data for machine learning and accelerating Injecting of data

6 6 Storage System for Traditional HPC Infrastructure One of important components for data store Requirements Performance(Throughput, IOPS, Metadata, etc) Capacity Protocol (NFS CIFS, PFS) Data Flow(e.g. HSM) Storage Management for HPC is not simple Data Management for 100PB+ Huge amount of complete node Internal/External Storage Users More complex will comes in the future Cloud Storage Integration Virtualization

7 7 New Requirements for Storage/Parallel File System for HPC/AI Storage Enabling Virtualization Virtual Machine, OpenStack Docker Container Integration Data Security User/Node Authentication Storage Data Isolation Encryption Audit for data Multi-Level Security Data Resource management Cache Management (NVMe to HDD) QoS for Node/VM/Container Strict Network/Storage resource control

8 8 DDN Lustre - ExaScaler Lustre is an Open source parallel filesystem 70% system of Top100 systems uses Lustre Already proved scalability 25PB single namespace in Production ~100,000 clients 1M+ metadata operations with Distributed NameSpace DDN s Lustre evolution First large Lustre system in 2003 A ton of Lustre deployments in the world over 13 years Many deployments on BigData/Machine Learning system DDN Contributes Lustre community and pushed many enhancements and new features into upstream DDN Lustre development team is located in Tokyo and Shanghai

9 9 Docker Integration on Lustre Simple way, but not flexible and no Lustre functions are available in containers. Container can mount Lustre directly same as bare-metal clients. But, security inside container is challenge. Container-A Container-B Container-C Container-D /scratch/container-a /scratch/container-b Docker Linux Kernel Hardware Native Client loopback mount Lustre Mountpoint /scratch Lustre Servers /scratch /scratch Native Client Docker Linux Kernel Hardware Interconnect (Infiniband, OPA, Ethernet, etc) Storage

10 10 Security: Name space Isolation Sub Directory Mount DDN Contribution Provide Isolated Namespace(Sub Directory) to each Container. Available in Lustre-2.9 DDN Lustre-2.5 But still limitation if root account in container change the configuration. (e.g. umount/mount) Lustre Mountpoint /scratch/projecta Linux Kernel Bere-metal Client Container-C Docker Linux Kernel Container-D /scratch/containerc /scratch/containerd Hardware Lustre Servers Hardware Storage Native Client

11 11 Security: Name space Isolation with Authentication DDN Contribution Available in Lustre-2.10 Lustre has been supporting Kerberos authentication. DDN added Lustre Kerberos authentication inside Docker container. Container A Container B Lustre client 1. credentials(a) 2. creda mount(creda) /fileseta /filesetb Kerberos server nodemap ContainerA:/filesetA Lustre servers Registered restricted mount point in nodemap for each container.

12 12 MLS(Multi Level Security) with Container Multi-Level Security in SELinux Bell-LaPadula Model (no read up, no write down) Label:TopSecret FileA Label : Secret FileB Label:Confidential FileC Label:Public FileD Objects Append WriteOnly R/W ReadOnly MAC(Mandatory Access Control) Label :Secret Process Subjects TopSecret Container R/W Store Data in Lustre with Security Lavel Container with Security Label Read Only Available in Lustre-2.8 Confidential Container R/W TS C P Storage Read Only Public Container Lustre Servers R/W

13 13 Resource Management Binding Lustre Network for Containers Pin CPU/Network Interface for containers. Also, Isolated Network devices for containers DDN Contribution Available in Lustre-2.9 DDN Lustre-2.7 Container-A /scratch/containera LNET0 Docker Linux Kernel Container-D /scratch/containerd LNET1 Hardware GPU N/I GPU N/I GPU N/I GPU N/I Lustre Servers Storage

14 14 Resource Management Quota for Containers DDN Contribution User/group belongs different projects Available in Lustre-2.10 DDN Lustre-2.7 ProjectA ProjectB Quota Per User/Group Container-A Container-D Project Quota associated with container /scratch/containera Quota per Project (FileSet) /scratch/containerd Lustre Servers Storage

15 15 Resource Management I/O QoS for Containers DDN Contribution Add a TBF(Token Bucket Filter) policy into Lustre NRS(Network Request Scheduler) Available in Lustre-2.6+ DDN Lustre-2.5 Container-A Container-D Docker I/O QoS per Container Lustre Servers Lustre QoS works based on UID/GID, JOBID and NID, etc. Storage

16 16 DDN Lustre Ecosystem for Data/Security Management with Container Kerberos Server Authentication TopSecret Container /scratch/ts Project Quota Confidential Container /scratch/c Docker Linux Kernel Public Container /scratch/p MLS (Multi Level Security) NameSpace Isolation QoS Hardware Interface0 Interface1 Interface2 Network Isolation Encryption by Kerberos Lustre Servers Storage

17 17 Conclusions HPC/AI requires new Storage requirements for Docker Container Not only same Enterprise features, but also fusion of Enterprise and HPC features at scale DDN contributes Lustre community adding new features for Docker Container Authentication and Isolation inside docker container Storage resource management per container DDN keeps open Lustre development process Many collaboration developments and research with customers are undergoing Willing to hear feedback of missing/challenges on storage for Docker Container!

18 Thank You! Keep in touch with us 東京都千代田区四番町 6-2 東急番町ビル TEL: FAX: company/datadirect-networks