Deep Dive: Cluster File System 6.0 new Features & Capabilities

Deep Dive: Cluster File System 6.0 new Features & Capabilities Carlos Carrero Technical Product Manager SA B13 1

Agenda 1 Storage Foundation Cluster File System Architecture 2 Producer-Consumer Workload Use Case 3 Fast Failover for Critical Applications Use Case 4 Clustered NFS use Case 5 Scale-Out Applications Use Case SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 2

What is Cluster File System Node 1 Node 2 Node 64 App A VCS SW stack App B CFS Instance Virtualization App C VCS SW stack App D CFS Instance Virtualization Private Network Ethernet with VCS Protocols App E VCS SW stack App F CFS Instance Virtualization SAN (Fiber Channel, iscsi or SAS) /MNT1 Disk Arrays SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 3

Cluster File System Architectural Overview Veritas Cluster Server, agents for mounts/disk groups/volumes CFS/CVM agents RAC, NFS, Apps Node-instance Cluster File System Cluster Volume Manager File system instance Global Lock Manager Group Atomic Broadcast Low Latency Transport Cluster-wide SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 4

Producer and consumer workload Challenges Concurrent access for many clients Granular file IO Cache coherency Consumers constantly trying to read file File Financial data EOF Producer application periodically appending to file SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 6

Global lock management Locks held by this node GLM Instance Proxy Locks mastered by this node Master SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 7

Global lock management (continued) Name Node ID GAB membership Lock type. Algorithm Name Node ID GAB membership Lock type. Hash value Hash value Table Node ID Node ID Node ID Master Master SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 8

Node grants in Global Lock Manager GLM instance 2 GLM instance Proxy 1 Master 3 4 CFS thread 5 CFS thread Minimize communications overhead 6 CFS thread SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 9

Lock operation and cache coherency APP Node A Node B GLM proxy Node grant NULL EX GLM Proxy Node grant NULL PAGE-CACHE GLM master for that lock Node A has EX grant SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 10

Load cache and cache coherency 2 WRITE APP Node B READ APP GLM proxy A Node grant SH EX GLM proxy B Node grant NULL SH PAGE-CACHE PAGE-CACHE GLM master for that lock Node A has SH EX grant Node B has SH grant SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 11

Range locks Inode locking does not allow parallel access FILE File level read/write LOCK W1 W3 Serial Access W2 Serial Access SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 12

Range locks (continued) Range locks allow parallel access to the same file FILE R1 Range lock R2 Range lock W1 W3 Serial Access W2 Parallel Access SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 13

Quiz TRUE or FALSE: To maintain cache coherency, CFS sends the data blocks over the network to the requesting node FALSE: Just request a lock What mechanism does CFS use to minimize node to node communications overhead? Node Grants SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 14

Failover Fast-failover for critical applications Failover as fast as application restart Fast failovers INST1 INST1 Achieve sub-minute failovers - 90% faster failover times SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 16

Failover Service recovery time delays INST1 Detect failure Unmount file system Deport disks Database Start database processes Check file system Mount file system Import disks SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 17

Failover Service recovery time delays INST1 Cluster File System Detect failure Unmount file system Deport disks Database CFS mount points Start database processes Check file system Mount file system Import disks CVM shared disk groups GLM recovery SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 18

Membership arbitration and data fencing Kernel-based fencing algorithm Node 1: Lose INS1 Node 2: Win Immediate link failure detection Interconnect INS2 Data Protection Mechanism Race upon server failure to determine node survival SCSI3 coordination disk Winner ejects the other node via SCSI3 key on data disk Node that lost the race triggers a 19 kernel panic Data protection 19 SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities

I/O Shipping Mitigate Infrastructure Failure Protect services from storage failure Utilise alternate I/O paths Eliminate Data Loss During Failover APP I/O Shipping Cluster File System SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 20

Cluster Volume Manager Asymmetry Increase uptime A node can join CVM if disk is accessible from other nodes Storage is present through other node DiskGroup A Cluster File System DiskGroup B Node is able to join cluster Node lost visibility for DiskGroup B storage SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 21

Faster Mirror Resync Improved Performance Sequential logging based DRL Minimize DRL updates DRL used as staging for DCO update Rw1 Rw2 Rwn Circular Region Log Better Scalability Independent of volume size & workload Per node DRL map with asyncupdate to DCO SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 22

ESX ESX ESX ESX ESX ESX CFS in VMware Environments RDM-P SCSI3 PR for Data Protection vmotion and DRS are prohibited VMDK Take advantage of ESX multi-writer capability Provide cache consistency Allow vmotion and DRS Use Coordination Point Servers CPS CFS CFS VMDK VMFS + multi-writer SCSI3 Fencing SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 23

Quiz Name one feature that allows CFS to make data immediately available CFS Mount Points, CVM Shared Disk Groups, GLM Recovery Which technology provides membership arbitration & data protection Fencing SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 24

Clustered NFS Availability and Scale for NAS NFS NFS Better price/ performance vs. NAS arrays Cache Cache Cache Cache CFS lock management Scale NFS across multiple low-cost nodes Scale performance separately from storage capacity Complements NFS v3 advisory locks with CFS/GLM locks SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 26 26

Directory sharing with partitions Locking at directory level limits performance Directory Directory level read/write LOCK W1 W2 W3 W1 W2 t Serial Access W3 SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 27

Directory sharing with partitions (continued) Directory Lock Partitions support concurrent I/O Directory Directory Lock Partition W1 W2 W3 W1 W2 W3 T/3 Parallel Access SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 28

Delegation Node gets a share of resources to work with Inode Allocation Units Extent Allocation Units Inode Allocation Units Extent Allocation Units File System Inode Allocation Units Extent Allocation Units SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 29

Block locking Single request block lock GLM master GLM proxy Block allocation Block name inode inode inode inode SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 30

Memory Delayed allocation during writes Write Write Write File System Single request SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 31

Quiz How does CFS share resources among nodes? Delegation SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 32

Scale-out applications Performance for parallel apps Minimal application overhead File access scales linearly as additional nodes are added Single namespace for database copies Cluster File System Cluster File System SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 34

Scale-out applications (continued) Near Local Performance Decentralized Decision Making Dynamic Ownership Minimized Messaging Data & Metadata Partitioning Range Locking Node Grant Caching Per-Node Transaction Independence Delegation Partitioning SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 35

Quiz What is the primary performance design goal for CFS? Near Local Performance SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 36

Cluster File System Major Use Cases and Benefits Producer-Consumer Fast Failovers Clustered NFS Scale Out Applications Reduce Storage Costs & Improve Performance Improve Service Recovery Times Cost Effective and Scalable NAS solutions Scalable Architecture with Increasing Business Demands SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 37

Value Proposition Differentiator Integrated Solution Linear Scalability ASL/HW Support Avoid Data Migration Storage Optimization Subminute Failover Storage Foundation Cluster File System HA SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 38

Thank You! carlos_carrero@symantec.com +34 659 352 408 SA B13: Deep Dive: Cluster File System 6.x New Features and Capabilities 39