You Have Stateful Apps - What if Kubernetes Would Also Run Your Storage? Annette Clewett, Senior Architect, Red Hat Sudhir Prasad, Product Management Director, Red Hat
Agenda Persistent Storage needs in Containers world Different Container Storage Options and Patterns with Kubernetes Kubernetes/Orchestrated Storage Runtime Pattern How Does it work? Live Demo Example OpenShift (Kubernetes) + Red Hat Container Native Storage
Persistence Container Storage Requirement Infrastructure Application Local/Ephemeral Storage Registry Metrics Service 1 Logging Service 2 Stateless Applica,ons, Pod Images PERSISTENT STORAGE FOCUS
Persistence Container Storage Options Local NFS ScaleIO Red Hat CNS. NAS Gluster GCP. SAN Ceph NetApp w/ Trident Many Options! ISCSI AWS - EBS StorageOS Fiber Channel AWS EFS Portworx Flex Azure Quobyte Mul,ple Op,ons - What to choose? How to Decide? Differen,ate? Do they follow a Design PaCerns? If so what are they?
Container Storage Pattern 1 Adapters to Network File and Block Storage Pod iscsi NFS Cinder GCE Plug-ins/Adapters à Tradi,onal storage systems Fibre Channel EBS Leverage what you Have Flex Azure Disk Limited leverage of Containers or Kubernetes ScaleIO Gluster Quobyte vsphere Separate Subsystem/ Mgmt
Pattern 1 Evolution à Container Storage Interface NFS Cinder GCE iscsi EBS Fibre Channel Azure Disk Flex ScaleIO Other Container Orchestra,on PlaKorms? Make interface standard vsphere Gluster Quobyte Container Storage Interface (CSI)
Pattern 2 - Exposing Service Interfaces to Orchestrators Container Orchestrator Adapter Cluster 1 Service Interface Administrator not provisioning storage - UI or CLI Developer via API / Service Broker Two Subsystem, Two Cluster/System Limited Leverage of Container + Kubernetes Value Cluster 2 Storage Runtime Storage Nodes Storage Nodes Storage Nodes
Pattern 3 à Orchestrated Storage Runtime Container Native Storage, Let K8s run your Storage! Container Orchestrator One Cluster Apps, Software Defined Network, Software Defined Storage 1 APP Container 2 3 APP Container 4 APP Container APP Container APP Container Storage Runtime Service Interface Adapter Storage Nodes Storage Nodes Storage Nodes No separate Cluster, Storage like any other service on Kubernetes
Pattern 3 à Orchestrated Storage Runtime Example : Red Hat Container Native Storage, Let Kubernetes run your Storage! MASTER AUTHENTICATION SCHEDULER Kubernetes Cluster
Red Hat Container Native Storage, Let Kubernetes run your Storage! Kubernetes Platform + Container Value Proposition MASTER RED HAT CONTAINER NATIVE STORAGE AUTHENTICATION Storage runs as a Container on K8s One Cluster, avoid Cluster Sprawl SCHEDULER GLUSTER VOLUMES One Control and Management Plane Horizontal Scaling HOST NETWORK PHYSICAL DISK K8s maintains desired state RHEL HOST OS Kubernetes Cluster Runs everywhere Private, Public, Virtual, Bare Metal, Hybrid
Red Hat Container Native Storage, Let Kubernetes run your Storage! Kubernetes Platform + Container Value Proposition Container Value Prop MASTER RED HAT CONTAINER NATIVE STORAGE AUTHENTICATION SCHEDULER Faster Deployment Rolling Upgrade Isolation & Portability Versioning & Reusability Runs Everywhere RHEL HOST OS Kubernetes Cluster Option to Co Locate with App
Pattern 3 àorchestrated Storage Runtime Example: Red Hat Container Native Storage One Cluster, One Infrastructure - storage like any other service One control plane end to end Exploit full value proposition of Kubernetes + Containers + Software Defined Storage
EXAMPLE
Demo Red Hat Container Native Storage OpenShift à Kubernetes Container Platform Red Hat Container Native Storage (CNS) Pattern 3 Orchestrated Storage Runtimes Based on Gluster Kubernetes Open Source
OpenShift (Kubernetes) with Container Native Storage OPENSHIFT / KUBERNETES CLUSTER MASTER S OPENSHIFT S WITH STORAGE CNS FAST POOL CNS SLOW POOL OPENSHIFT CLUSTER Storage Class Examples: Fast Pool: 3x node w/ SSDs Slow Pool: 3x node w/ HDDs OPENSHIFT S WITHOUT STORAGE Pods on OCP nodes can mount volumes from Fast and Slow Storage Classes
Demo Configuration: Red Hat Container Native Storage, Let Kubernetes run your Storage! MASTER APP 1 APP 2 AUTHENTICATION SCHEDULER APP 3 CNS CNS INFRA CNS CNS Demo OpenShiH Cluster
Demo Configuration: Red Hat Container Native Storage, Let Kubernetes run your Storage! MASTER AUTHENTICATION APP 1 CNS APP 2 CNS MASTER RED HAT CONTAINER NATIVE STORAGE SCHEDULER APP 3 INFRA GLUSTER VOLUMES CNS CNS HOST NETWORK PHYSICAL DISK Demo OpenShiH Cluster
Storage Class Example heketi REST API # cat cns-fast-storageclass.yaml apiversion: storage.k8s.io/v1beta1 kind: StorageClass metadata: name: cns01-vmdk-gluster-fast provisioner: kubernetes.io/glusterfs parameters: resturl: http://heketi-storage.apps.syseng.com clusterid: d0a035dc9022343480fcb0ec9de307 restauthenabled: "true" restuser: "admin" secretnamespace: "default" secretname: "heketi-secret"
How does Kubernetes/OpenShift Apps get Persistent Volume? Steps: OpenShift calls heketi through REST heketi REST API API found in Storage Class object Heketi provisions volume on CNS 3x CNS PODs in CNS on 3x OCP nodes 3x minimum due to 3-way replication for Gluster volumes
How does Kubernetes/OpenShift Apps get Persistent Volume? sets up submits Persistent Volume Claim A request for storage submitted to StorageClass creates PersistentVolume provisions mounted by APPLICATION POD(S) Provider: glusterfs Capacity: 10 GiB Mode: RWX instructs Storage Backend
LIVE DEMO
Questions?