Tag line, tag line Challenges in Storage Systems: A NetApp perspective Deepak Kenchammana-Hosekote Advanced Technology Group NetApp
Agenda What we do (context) What we see happening (trends) What we are doing about it (initiatives) How you can help us.
NetApp Fact Sheet FY07 $2.8 billion 07 '03 '03 '04 '04 '05 '05 '06 '06 Founded in 1992 Fastest growing storage company 4 consecutive years of 30%+ growth #6 Best Company To Work For Headquartered in Sunnyvale Other engineering sites in: Pittsburgh, RTP, Boston, Bangalore India 75,000+ systems installed worldwide 111 Petabytes shipped Q1 FY08 #1 in NAS PB shipped #1 in NAS market share
Key NetApp Ideas Purpose-built appliance; does one thing well One management model & one system to learn Simplicity Simple on the outside simple on the inside Easy to use, easy to set up Combination of then new research ideas RAID [Patterson88] Log-structured file system [Rosenblum92] All built from commodity components Off-the-shelf x86 CPUs, memory, etc NVRAM is main exception
Scale from 1 TB to >6,000 TB Entry NAS Direct Attach/ Small Archive NAS Media Center Storage Storage Virtualization StoreVault Entry-level nearline archives under $3K FAS2000 Series Expandable NAS for small to medium nearline applications FAS6000 series FAS3000 series Modular enterprise-class storage; massive scalability V-Series Dynamic virtualization for heterogeneous storage Data ONTAP Operating System (FC SAN, IP SAN, NAS) Data Security Information Lifecycle Mgmt Virtual Tape Library NetApp DataFort Data encryption for audio, video, and image storage Information Server IS1200 Information classification and management for unstructured data NearStore VTL Disk-to-disk backup with tape library emulation 5
Data ONTAP from 30,000 feet Client Client Client Network Network Stack Protocols Filer WAFL RAID Storage Client Client NVRAM Disks looks a lot like an operating system, but Optimized for efficient data movement Specialized interfaces between components Robust HW error recovery
WAFL file system No fsck. Ever! Writes buffered in NVRAM Data never overwritten Metadata stored in files volinfo Inodefile File A File B
Writes Superblock I 0 Inode file I A I B Contains inodes for regular files and dirs File blocks A 1 A 2 B 1 B 2 File A File B
Writes (2) Superblock I 0 I 0 Inode file I A I B I B File blocks A 1 A 2 B 1 B 2 B 2
Snapshots Snapshot superblock S 0 I 0 I 0 Inode file I A I B I B File Blocks A 1 A 2 B 1 B 2 B 2
Snapshots enable many features Efficient application recovery (SnapDrive) MS Exchange Server & Oracle RDBMS Consistent image for backups (SnapMirror) Only changed data need be mirrored remotely Compliant data repository (SnapLock) Read-only online data Efficient creation of clones Copy-on-write snapshot; cloning DBs very useful Quickly tell what s changed (SnapDiff API) Compare inode files of two snapshots
Beyond a Single Controller Scaling up What if one controller is not big enough? Want a namespace that spans multiple controllers Scaling out Add capacity & performance with more controllers pay as you go model But what happens to manageability? Filers should only scale in performance and capacity; management should be just the same One filer is easy to use. 100 filers are a pain to use. Answer: Clustered ONTAP
Clustered ONTAP Architecture Clients NFS, CIFS iscsi, FC Distributed Volume Location Database N-blade N-blade N-blade WAFL SpinNP Protocol RAID D-blade D-blade D-blade Storage
Global Namespace in a Cluster Namespace Root R Clustered ONTAP System A B C D B A R C1 C2 D1 D2 C D1 D2 C2 D C1
Advanced Technology Group Under office of the CTO; 3yrs old 25 Members and growing; 4 sites worldwide Pursue long term projects Explore technology driving our strategic direction Create opportunities beyond current product horizon University research investments Coordination of research funding Leverage investments through hiring, internships Goal: Investigate new technologies, influence and create products
The Advanced Technology Group (2) University collaborations Wisconsin-Madison, Carnegie Mellon, MIT, UCSC, UCSD, Harvard, UIUC, Waterloo, Duke, Berkeley, Actively publish SIGMETRICS 2007 Best student paper 6 papers with NetApp authors in FAST 2008 incl. Best student paper 2 papers in USENIX 2008
Latent Sector Errors Study [Sigmetrics07] How individual sector errors affect data integrity Examined >70,000 systems with over 1.3 million disks Are our defense mechanisms good enough? Findings: 3.45% of 1.53 million disks with 1 LSE 8.5% of SATA vs. 1.9% of FC disks SATA On average 77% of Latent Sector Errors discovered by VERIFY
A Comprehensive Study of Failure Characteristics [FAST 2008] How to best improve resiliency to HW failures? Ex: RAID group layout Shelf enclosure model has strong impact on failures Failures are not independent The AFR for disks and storage subsystems does not increase with disk size.
Agenda What we do (context) What we see happening (trends) What we are doing about it (initiatives) How you can help us.
Three Categories of Innovation New invention Create an entirely new technology Launch new industries Monotonic improvement Technologies that ratchet ever upward Exponential improvement Monotonic improvement that follows an exponential curve E.g. DRAM, Processor Mips, Flash Memory, Disk size Observation 1: The steep part of the curve can be as enabling as a new invention Observation 2: The steep part of the curve can break old solutions E.g. disk size : disk IOPs ratio Observation 3: Suddenly becomes disruptive when applied to new areas, displace existing technology E.g. tape archive replaced by disk archive E.g. primary disk storage replaced by flash memory
Technology Drivers Performance Capacity Security Power Management and Complexity New applications and value-add Global regulatory requirements
Enabling New Applications and Invention Server virtualization Ubiquitous computing Network computing PDA s as clients Globally distributed data service Increasing modularity and scale Reducing visible complexity Self-diagnosing, self-repairing servers Federating services across administrative and organizational boundaries End-to-end integrity and security
The Storage Business Primarily about providing containers LUNs Files Directories Volumes Value-add primarily in: Container virtualization Container management Container reliability Container access performance Archive, backup Data security and integrity
Key Initiatives Building a better container Scaleout Data Management Data Protection and Retention The expanded vision
Building a better container Increased virtualization Fine-grained container hierarchy Further disassociation of logical from physical Dynamic, fine-grained data selection Alternative and parallel hierarchies of containers Simplified and improved manageability Manage relatively few large-scale datasets composed of many fine-grained logical containers
Building a better container (2) Make copies smart: Leverage archive, backup as active assets. Compression and Dedup Improve storage utilization In-place data encryption Robust and secure key management Challenge is to combine these
Scaleout Multi-controllers Tightly integrated hardware Embedded clustering Incrementally expandable Scales up and down Increasing modularity in the core software Separate key components of the data path from each other and from software infrastructure Exploit user space for value-added software Increase absolute numbers of containers of all types Striping file system
Scaleout (2) Exploit scaleout technology to provide Caching Heterogeneous storage Hybrid storage hierarchies Heterogeneous federations Heterogeneous storage backends Storage and System resiliency Self-diagnosing and self-repairing subsystems Policy-driven fault recovery
Data Management Powerful data management within the single system image cluster Layered on-box and off-box data and storage management tools Spans multiple clustered and unclustered systems Heterogeneous participants Abstract data set model simplifies management Pushes complexity into the underlying software layers
Data Protection and Retention Low (to zero) Recovery Point Objective (RPO), Recovery Time Objective (RTO) for mirrors Smart copies Multi-use mirrors Advanced server and virtual server level DR Integration with virtualized computing environments
The Expanded Vision Global Data Service: Federations Many systems all interoperating to provide DR Remote caching and Vserver presence Archive and backup A hierarchy of flexible virtualized containers independent of the underlying hardware Connected across wide geographies Weak interconnects Different administration and trust zones Ultimately multi-vendor solutions Requires standards
The Database Business Primarily about adding value to the contents of containers Indexing Query processing Imposes structure on data to facilitate analysis by reusable tools Storage systems do not generally impose any discernable structure aside from block or extent size Containers are blobs
Containers and Contents Add value to stored content Extracting structure and information from unstructured blobs of data Indexing Query processing Extended Attributes and Metadata E.g. file provenance Tighter integration of database and storage systems Add value beyond providing containers Produce information from data
Tooling to meet challenges Software engineering challenge is enormous Not unique to NetApp How can we build ever larger systems quickly enough to meet accelerating market needs? How can we make all this software reliable? How can we prove its reliability? Big challenges in: Software tools System test and validation Support infrastructure, diagnosis and resolution Globally distributed development Are the programming languages and tools up to the task? Facing a lot of inertia MP is now the norm Moving from C to C++ is not the answer We have many of the same problems we help our customers solve
How you can help us We are seeking collaboration and leverage in the academic community Equipment donation Funded research Internships Full-time hires We would like to share real world data System logs, traces, Hope to energize you to look at some of these very interesting problems
Tag line, tag line We re hiring! Questions?