Above the clouds with container-native storage

Size: px

Start display at page:

Download "Above the clouds with container-native storage"

Allison Marshall
5 years ago
Views:

1 Above the clouds with container-native storage Ryan Cook, Senior Software Engineer Davis Phillips, Principal Software Engineer Glenn West, Principal Software Engineer Annette Clewett, Senior Storage Architect May 10, 2018

2 Introductions Davis Phillips - Principal Software Engineer Glenn West - Principal Software Engineer Annette Clewett - Storage Architect Ryan Cook - Senior Software Engineer

3 Container-native Storage

4 Openshift Persistent Storage Framework

5 Openshift Persistent Storage Framework submits PersistentVolumeClaim A request for storage Provider: ABC Capacity: 10 GiB Features: XYZ

6 Openshift Persistent Storage Framework sets up submits submitted to PersistentVolumeClaim StorageClass A request for storage A provider of storage Provider: ABC Capacity: 10 GiB Features: XYZ Provider URL: Credentials: Options:...

7 Openshift Persistent Storage Framework sets up submits submitted to PersistentVolumeClaim StorageClass A request for storage A provider of storage Provider: ABC Capacity: 10 GiB Features: XYZ Provider URL: Credentials: Options:... instructs Storage Backend

Openshift Persistent Storage Framework sets up submits submitted to creates PersistentVolumeClaim StorageClass PersistentVolume A request for storage A provider of

8 Openshift Persistent Storage Framework sets up submits submitted to creates PersistentVolumeClaim StorageClass PersistentVolume A request for storage A provider of storage Provisioned Storage Provider: ABC Capacity: 10 GiB Features: XYZ Provider URL: Credentials: Options:... Name: Size: AccessMode:... instructs provisions Storage Backend

Openshift Persistent Storage Framework sets up submits submitted to creates mounted by PersistentVolumeClaim StorageClass PersistentVolume A request for storage A provider of storage

9 Openshift Persistent Storage Framework sets up submits submitted to creates mounted by PersistentVolumeClaim StorageClass PersistentVolume A request for storage A provider of storage Provisioned Storage Provider: ABC Capacity: 10 GiB Features: XYZ Provider URL: Credentials: Options:... Name: Size: AccessMode:... APPLICATION (S) instructs provisions Storage Backend

GlusterFS - Distributed File Storage Scale Out Performance, Capacity & Availability RED HAT GLUSTER STORAGE FOR ON-PREMISE Deploys on Red Hat-supported servers and

10 GlusterFS - Distributed File Storage Scale Out Performance, Capacity & Availability RED HAT GLUSTER STORAGE FOR ON-PREMISE Deploys on Red Hat-supported servers and underlying storage: DAS, JBOD Scale-out linearly Replicate synchronously and asynchronous Scale Up Capacity Single, Global namespace SERVER (CPU/MEM) SERVER (CPU/MEM) SERVER (CPU/MEM)

11 Federating Local Storage GLUSTERFS VOLUME CLIENT MOUNT server1:/exp1 server2:/exp2 BRICK FILE 1 FILE 2 BRICK FILE 1 FILE 2

12 Container-Native Storage on Openshift CONTAINERIZED RED HAT GLUSTER STORAGE GLUSTER VOLUMES RED HAT GLUSTER HOST NETWORK PHYSICAL DISK RHEL HOST OS

13 Container-Native Storage on Openshift HEKETI RESTFUL GLUSTERFS MANAGEMENT RHGS RHGS REST API INFRA INFRA RED HAT HEKETI GLUSTERFS S RHGS RHGS

14 Container-Native Storage on Openshift RHGS INFRA INFRA RHGS RHGS

15 Container-Native Storage on Openshift RHGS 1.) PERSISTENT VOLUME CLAIM INFRA INFRA RHGS RHGS

16 Container-Native Storage on Openshift RHGS 1.) PERSISTENT VOLUME CLAIM INFRA INFRA RHGS 2.) RHGS

17 Container-Native Storage on Openshift GLUSTER VOLUME RHGS 3.) 1.) PERSISTENT VOLUME CLAIM INFRA INFRA RHGS 2.) RHGS

18 Container-Native Storage on Openshift 4.) PERSISTENT VOLUME GLUSTER VOLUME RHGS 1.) PERSISTENT VOLUME CLAIM 3.) 4.) INFRA INFRA RHGS 2.) RHGS PERSISTENT VOLUME

19 Container-Native Storage on Openshift

20 Container-Native Storage on Openshift Node Node AVAILABILITY ZONE A Node Node AVAILABILITY ZONE B Node Node AVAILABILITY ZONE C

21 VMware vsphere Platform

22 OpenShift Container Platform on VMware w/cns Storage types in vsphere NFS vsphere-volume CNS

23 vsphere Storage - NFS Pros Easy to use and setup Cons No ReadWriteMany or ReadOnlyMany support Limits scaling for services that would benefit from it Not supported for Registry, Metrics or Logging

24 vsphere Storage - vsphere-volume New in Kubernetes 1.6 using vsphere Cloud Provider Pros Makes provisioning storage a streamlined process Supports multiple datastores in vsphere Dynamic Provisioning using Storage Classes Cons No ReadWriteMany support Tied to Node Registry is not Supported

25 vsphere Storage - CNS Gluster File Storage Pros ReadWriteMany and ReadOnlyMany Support Replication Highly available Dynamic Provisioning using Storage Classes Support for Registry, Logging and Metrics Cons May require additional licenses

26 vsphere Plugin - Why CNS? Topic Dynamic Provisioning RWO (Block), RWX (File), Object CNS/CRS vsphere Plugin (All three in one) (Dev, deploy, use anywhere) Online Resize Support Volume Metrics/PV Consumption (Use business Rules) Hybrid Deployment VMware, Bare Metal, AWS,... Custom Volume Naming Integration with OCP High Availability Install, Management, Monitoring 3 Failure Domains Mounts ~7-25 seconds Failover Transparent Slow ~ 1 min 2 min Slow ~ 3-5 minutes (App outage)

27 Sample Architecture - CNS Registry only

28 Sample Architecture - CNS App & Registry

29 Azure

30 OCP on Azure w/cns Storage Options Existing Storage Choices Azure Disk CNS AZURE DISK Profile kind: StorageClass apiversion: storage.k8s.io/v1 metadata: name: fast provisioner: kubernetes.io/azure-disk parameters: skuname: Standard_LRS location: eastus storageaccount: azure_storage_account_name CNS Profile kind: StorageClass apiversion: storage.k8s.io/v1 metadata: name: slow provisioner: kubernetes.io/glusterfs parameters: resturl: " restuser: "admin" secretname: "heketi-secret" secretnamespace: "default" gidmin: "40000" gidmax: "50000"

31 Azure Disk Storage Pros Native to Azure Azure Underlying Redundancy Used Direct cost to usage per PV Direct use of Premium / SSD Drives Note Make sure vmname and hostname the same Cons ReadWriteOnce only supported model Number of drives per host severely limited (Limits containers per host) Very Slow PV Creation (Multiple Minutes)

32 CNS Gluster File Storage vs Azure Disk Pros Fast PV Create Time ReadWriteMany and ReadOnlyMany Support Highly available Dynamic Provisioning using Storage Classes Support for Registry, Logging and Metrics Ability to do very small to very large PV Cons May require additional licenses

33 Azure with CNS Storage CNS Registry CNS Nodes

34 Azure Why CNS?

35 AWS

36 OCP on AWS w/cns Storage Types on AWS EFS EBS S3

37 AWS Storage - EFS Pros Not AZ specific ReadWriteMany support Automatically scaled storage Cons No guaranteed performance Not suitable for all application workloads Cannot be used for logging and metrics Requires a specific playbook to be ran

38 AWS Storage - EBS Pros Guaranteed performance Cons AZ specific ReadWriteMany support requires all pods must run on a single node

39 AWS Storage - S3 Pros Automatically scaled Perfect for the image registry Cons Object based storage cannot be used for persistent container storage

40 AWS - CNS Gluster File Storage Pros ReadWriteMany and ReadOnlyMany Support Replication Highly available regardless of AZ All workloads supported (logging, metrics, prometheus) Cons May require additional licenses Requires large instances

41 AWS with CNS Application Storage CNS Nodes

42 AWS- Why CNS?

43 Q&A

44 Links Ref archs

45 THANK YOU plus.google.com/+redhat facebook.com/redhatinc linkedin.com/company/red-hat twitter.com/redhat youtube.com/user/redhatvideos

You Have Stateful Apps - What if Kubernetes Would Also Run Your Storage?

You Have Stateful Apps - What if Kubernetes Would Also Run Your Storage? Annette Clewett, Senior Architect, Red Hat Sudhir Prasad, Product Management Director, Red Hat Agenda Persistent Storage needs in