INFINIDAT Storage Architecture White Paper
Abstract The INFINIDAT enterprise storage solution is based upon the unique and patented INFINIDAT Storage Architecture (ISA). The INFINIDAT Storage Architecture is a fully abstracted set of software-driven storage functions layered on top of off-the-shelf hardware. By shipping the software with a highly tested hardware reference platform, INFINIDAT can deliver the first true enterprise-class software-defined storage. This paper discusses how INFINIDAT achieves multi-petabyte capacity in a single rack, mainframe-class reliability with an unprecedented 99.99999% (7 nines) availability, and over 900K IOPS of performance. 2
Introduction The INFINIDAT enterprise storage solution is based upon the unique and patented INFINIDAT Storage Architecture (ISA). The INFINIDAT Storage Architecture is a fully abstracted set of software-driven storage functions layered on top of off-the-shelf hardware. The result is multi-petabyte capacity in a single rack, mainframe-class reliability with an unprecedented 99.99999% (7 nines) availability, and over 900K IOPS of performance. Automated provisioning, management, and application integration provide a system that is incredibly efficient and easy to use. By separating the storage innovation from the hardware, INFINIDAT allows for the rapid adoption of the latest and most cost-effective hardware. In addition, by shipping the software with a highly tested hardware reference platform, INFINIDAT can deliver the first true enterprise-class software-defined storage. INFINIDAT Hardware INFINIDAT is an N-way-active architecture. Our current-generation products use three nodes (N=3) per rack. These nodes represent the intelligence in the system. Each independent node contains a server, D, and flash cache: up to 86TB up to 3TB HOST CONNECTIVITY INFINIDAT includes three connectivity layers: Front end FC and Ethernet connectivity for client access, replication, and management High-speed InfiniBand Low latency node-to-node communication Back end SAS connectivity from each node to all the drive enclosures in the system ALL NODES ARE CONNECTED TO ALL DRIVES 3
Availability The architecture was designed to sustain multiple failures without disrupting client operations. At the hardware connectivity level, this is done with the following redundancy: All front-end connections are redundant, with floating IP addresses and software-defined FC targets, which can failover between physical ports/nodes InfiniBand disconnection is solved by routing the InfiniBand messages through another node SAS connectivity failures redirect IO to the InfiniBand link as a temporary path of accessing drives via another node Below the node layer are eight drive enclosures. The drive enclosures can be filled to include 480 drives of up to 8TB each. All nodes utilize all drives and disk drive failures are handled by our de-clustered RAID data layout. This architecture guarantees that all the nodes receive an even load. In the event of a drive failure, all the remaining drives work in parallel to support the restore process. H O S TStorage CO NNEC TI V I T Y INFINIDAT Architecture AL L N O D E S ARE CO NNEC TED TO ALL D RI V ES 4
Caching Each of the nodes in the system is equipped with multiple SSD drives, which act as a global cache for all the nodes (no data block is cached twice). A read IO sent to node 1 can get a cache hit from node 2 s SSDs to minimize latency. With up to 86TB of SSD cache per system, InfiniBox can cache almost all the active dataset and avoid getting reads from slow, spinning drives. Logical Architecture INFINIDAT s logical architecture leverages small sections of disks as RAID members (instead of the entire disk). This minimizes disk contention and allows each of the nodes in the system to use all the drives all the time for maximum performance. Incoming Data 14 data sections 2 parity sections As data comes into the system it is aggregated into 14 optimized sections, each with its own DIF (data integrity field) and lost-write protection field. These protect the data from logical corruptions as well as disk-level errors. InfiniBox then adds two parity sections to complete a RAID stripe. The data is then sent to a group of 16 disks, with each RAID stripe always landing in different disks. This minimizes the common data between any two drives, accelerating double-failure RAID rebuilds. This also guarantees no hot spots can be created, as all data is evenly spread across all the disks in the system. 5
As a result, if the system sustains a double drive failure, it can immediately calculate which RAID stripes were shared by these two drives and rebuild the RAID groups with the higher risk first. Since the recovery is performed from all the drives at the same time, the system can rapidly recover (typically less than 10 minutes) from a double disk failure to protected mode. By designing both the software and hardware stacks to be highly resilient, the INFINIDAT Storage Architecture is built to provide an unprecedented 7 nines availability (99.99999% uptime). Performance With almost all the reads coming from SSD, the spinning drives run at a low utilization, offering the little IO that doesn t hit the cache a low latency response. Writes are buffered in a large protected memory cache that sustains all the changes to the blocks before they cool down enough to be sent to disk. When writes are finally sent to the disks, they are written to all the disks simultaneously, giving the system the ability to write at full drive speed to all the drives. Writes that are being accessed (hot data) are also written to the SSD cache layer to accelerate reads. Conclusion The INFINIDAT Storage Architecture was designed for the most critical of business applications. The architecture supports an extremely high level of availability, has a very small footprint, and is extremely high performance. The architecture also allows for virtually unlimited scalability and carefully balances functionality and cost. The INFINIDAT Storage Architecture is at the heart of the InfiniBox family of storage arrays. Unlike traditional storage architectures, this new architecture is designed to eliminate storage trade-offs. www.infinidat.com info@infinidat.com WP-STORARC-160720-US INFINIDAT 2016 6