Comparing File (NAS) and Block (SAN) Storage January 2014
Contents Abstract... 3 Introduction... 3 Network-Attached Storage... 3 Storage Area Network... 4 Networks and Storage... 4 Network Roadmaps... 5 Fibre Channel Roadmap... 5 Ethernet Roadmap... 5 Comparing the Speeds... 6 Flexibility of File-Based Storage Over Block... 6 Data Access... 7 Expansion... 9 Other Benefits of NAS... 10 Conclusion... 11 Copyright 2015 Spectra Logic Corporation. BlueScale, Spectra, SpectraGuard, Spectra Logic, TeraPack, T Finity, and TranScale are registered trademarks of Spectra Logic Corporation. ArchiveGrade, BlackPearl, IntraCloud, and ntier Verde are trademarks of Spectra Logic Corporation. All rights reserved worldwide. All other trademarks and registered trademarks are the property of their respective owners. All library features and specifications listed in this white paper are subject to change at any time without notice. Comparing File (NAS) and Block (SAN) Storage 2
A B S T R A C T There are two primary types of shared storage used in the Data Center: file-based Network Attached Storage (NAS) and block-based Storage Area Networks (SAN). They both have their place in the data center, and many organizations utilize both for different uses. In this paper we will look at both for use in large scale data storage and data protection. These use cases are most commonly backup, archive, and digital parking lots. 1 Both types of shared storage can provide high levels of value to data center operations, but for the use cases outlined in this paper, we will see that file-based, or NAS, storage is preferable for most deployments. I N T R O D U C T I O N This paper provides an analysis of NAS and SAN, including basics of how they work as well speed comparisons. N E T W O R K-ATTACHED STORAGE Network Attached Storage (NAS) at its simplest is file-level computer data storage connected to a computer network. NAS: a class of systems that provide file services to host computers using file access protocols such as NFS or CIFS. 2 The storage system creates and manages the file systems needed to store data. That data is accessed over an IP network, typically via NFS or CIFS. NFS, or Network File System, is a distributed file system protocol allowing file access over the network. It is typically used in UNIX and Linux environments. CIFS, or Common Internet File System, is the protocol used primarily by Windows systems for file access over the network. The NAS system manages access control and allows multiple systems to share the same data set. NFS and CIFS are file-based access protocols. Cloud-based interfaces, such and S3 and DS3, are another way to access file systems in a NAS environment. These interfaces are object based instead of file based. Object-based storage has advantages as well as limitations compared to traditional file based storage. Object-based storage systems and file-based storage systems have the same function: to manage storage and create the storage structure. While the interface is 1 A digital parking lot is an easily accessible storage location. It typically stores large unstructured data sets and is often not managed. 2 SNIA Dictionary - http://snia.org/education/dictionary/n Comparing File (NAS) and Block (SAN) Storage 3
different, the system does the same work, letting multiple clients share data access and enjoy easy scaling. S T O R A G E AREA NE T W O R K The commonly used definition for a Storage Area Networks (SAN) is not technically accurate. Most of the time, when referring to a SAN, people mean a fibre channelattached block storage solution. A SAN is actually less strictly defined than that. SAN: A network whose primary purpose is the transfer of data between computer systems and storage devices, and among storage devices. A SAN consists of a communication infrastructure, which provides physical connections; and a management layer, which organizes the connections, storage devices, and computer systems so that data transfer is secure and robust. The term SAN is usually (but not necessarily) identified with block I/O services rather than file access services. 3 A SAN is simply a dedicated network setup for storage access. Most people use the term interchangeably with block storage. For the rest of this paper, we will focus on block storage specifically, as it relates to SANs. The defining difference between block storage and file-based storage is the lack of a file system or object management. The defining difference between block storage and file-based storage is the lack of a file system or object management. Block storage systems present unformatted storage space to their clients. The client servers then handle the file system duties. In the past, this has allowed block storage to offer a higher performance potential, but prevented multiple systems from sharing the same data sets. There are some very rare deployments for specialty uses that allow shared block access, but these are not deployed for the use cases evaluated in this paper due to cost and complexity. N E T W O R K S A N D STORAGE Both file and block storage are accessed via a network. Traditionally, IP-based networks were associated with NAS and Fibre Channel networks with SAN. Network-based storage types can be easily confused when they both use the IP network. The first technology that caused confusion was iscsi. iscsi (internet Small Computer System Interface) is a protocol that allows SCSI commands to be transported over IP based networks. Some people have confused this protocol and NAS, as they both use the IP network. iscsi continues to be a block interface to storage. The storage array does not provide a file system or easy sharing access to data. The advantage is flexibility in connecting servers to the array, as traditional IP networks can carry the data. 3 SNIA Dictionary - http://snia.org/education/dictionary/s#storage_area_network Comparing File (NAS) and Block (SAN) Storage 4
A second block protocol on traditional networks to be introduced is FCoE. This protocol pushed Fibre Channel commands over Ethernet networks. This takes advantage of both the high efficiency of the Fibre Channel protocol and the pervasive Ethernet networks. In both cases, block storage has moved onto IP or Ethernet networks. While there are many reasons for this, the end result is still block storage. N E T W O R K ROADMAPS Historically, Fibre Channel SAN systems have been considered higher performance than NAS systems over Ethernet. The published roadmaps show that not only has Ethernet caught up, it is moving gigabytes faster than Fibre Channel now. F I B R E C H A N N E L R O A D M A P 4 The following table shows the Fibre channel roadmap. Table 1: Fibre Channel Roadmap Name Speed T11 Spec Technically Completed 1GFC 200 MBps 1996 2GFC 400 MBps 2000 4GFC 800 MBps 2003 8GFC 1600 MBps 2006 16GFC 3200 MBps 2009 32GFC 6400 MBps 2013 64GFC 12800 MBps 2016 E T H E R N E T R O A D M A P The roadmap for Ethernet shows a much higher performance future than FC. 5 Table 2: Ethernet Roadmap Name Speed Data initial standard Ratified 10Mb/s Ethernet 10 Mb/s 1983 100Mb/s Ethernet 100 Mb/s 1995 Gigabit Ethernet 1 Gb/s 1998 10 Gigabit Ethernet 10 Gb/s 2002 40 Gigabit Ethernet 40 Gb/s 2010 100 Gigabit Ethernet 100 Gb/s 2010 400 Gigabit Ethernet 400 Gb/s 2017 (EST) At the time of this paper, network infrastructure can be deployed supporting up to 16 GFC on the SAN side and up to 100 Gigabit on the NAS. 4 http://www.fibrechannel.org/fibre-channel-roadmaps.html 5 http://www.ethernetalliance.org/wp-content/uploads/2013/04/ethernet-alliance-technology- Roadmap-03-28-14.pdf Comparing File (NAS) and Block (SAN) Storage 5
C O M P A R I N G T H E S P E E D S The standards groups for Ethernet and Fibre Channel use different metrics to specify the speed of their technologies. This can make it hard to compare them. 1 Gb is equal to 128 MB. Looking at the charts above, we can see that Gigabit Ethernet s 1 Gb/s is 128 MB/s. 1GFC is 200 MB/s, making it quite a bit faster than Ethernet. To make it easier to compare the technologies, we can consider how long it will take to move 1 TB of data across the different networks. Table 3: Speed comparison, Fibre Channel Name Speed Time to move 1 TB of data 8GFC 1600 MBps 12 Minutes 16GFC 3200 MBps 6 Minutes 32GFC 6400 MBps 3 Minutes 64GFC 12800 MBps 1 Minute 30 Seconds Table 4: Speed comparison, Ethernet Name Speed Time to move 1 TB of data 10 Gigabit Ethernet 10 Gb/s 15 Minutes 40 Gigabit Ethernet 40 Gb/s 3 Minutes 45 seconds 100 Gigabit Ethernet 100 Gb/s 1 Minute 30 seconds 400 Gigabit Ethernet 400 Gb/s 22 seconds The current performance roadmap shows Ethernet outpacing Fibre Channel in performance for the next several years. The current performance roadmap shows Ethernet outpacing Fibre Channel in performance for the next several years. While most organizations have not deployed 100 Gigabit yet, as they do, they will see significant performance improvements over the 8GFC and 16 GFC that they are running today for storage. Less than a decade ago, if an organization had a high-performance requirement for their storage system, the obvious and possibly only choice was a Fibre Channelbased SAN. Technology has advanced on the Ethernet side such that now the network can provide similar or faster speeds than Fibre Channel, with the future looking even better. Deciding between NAS and SAN is now more about use case than performance. F L E X I B I L I T Y O F F I L E-BASED STORAGE O V E R BLOCK Backup, archive, and similar use cases can utilize disk storage from either SAN or NAS. As we have seen, when looking at new deployments, the network performance today isn t the big differentiator that it once was, so we should look at other operating characteristics to determine what is best. NAS or file based storage systems have some unique properties that allow for great flexibility compared to SAN systems. These all originate from the fact that a NAS Comparing File (NAS) and Block (SAN) Storage 6
device has a file system and manages the sharing and access to data stored on it. SAN systems do not have this ability. Unlike block based storage, file based storage allows data to be easily shared. D A T A A C C E S S Data access is very different between NAS and SAN based systems. Block based system preset up raw storage space, requiring the server that mounts the space (often referred to as a LUN) to format it. This server becomes the only access point to that storage. Except for in clustered environments, everyone needing access to the data must go through this server. File based storage presents formatted storage through a network share. The network share is designed to provide access to data to multiple servers or other clients. Unlike block based storage, file based storage allows data to be easily shared. Let s look at an example of data access. Figure 1 shows how data is accessed in most backup and archive systems. The user connects to an application server, typically called a media server. This server manages data movement into and out of storage. The media server handles indexing of the data, and may allow for searches, scans and well are reading and writing. Figure 1: Typical data access As environments grow, systems scale and get more complex. This generally leads to more Media Servers to manage the data access and movement. A single media server ultimately can become a bottleneck for the system. By deploying multiple media servers, the workload can be spread across them. This is where NAS starts to show its strengths. Block based storage can provide space for each Media Server, but that space is unique to each server. Figure 2 shows logically how the media servers are connected to a unique piece of storage. Data written by media server 3 to LUN 3 can only be accessed by media server 3. If media server 3 fails, then any data stored in LUN 3 cannot be accessed. Operationally, there isn t any difference between using a block based SAN system than there is for direct connected storage. Comparing File (NAS) and Block (SAN) Storage 7
Figure 2: Media server connection to storage If the same three media servers were connected to a file based NAS instead, things are much different. Figure 3: Three media servers connected to same CIFS share Figure 3 shows the three media servers all connected to the same CIFS share. This CIFS share is presenting same amount of storage as the example in figure 2, but in this case, all three media servers have access to all of the storage. Media server 3 can write data to the share, and media server 1 can read it. If one of the media servers is lost, the rest of them still have access to the data. As environments grow, the ability to share the data amongst multiple media servers or possibly allow direct access from the client becomes more important. Comparing File (NAS) and Block (SAN) Storage 8
Inevitable data growth means more media servers to manage the data and more disk space to store the data. E X P A N S I O N Like death and taxes, data growth is one of the inevitable things everyone in technology has to deal with. When looking at backup and archive, this means more media servers to manage the data and more disk space to store the data. For both types of growth, a NAS system is almost always much easier to expand than a SAN system. Growing a disk array is more complex than simply adding hard drives. The additional storage space has to be presented to the servers or clients that need it. Figure 4: Expanding a block storage system Figure 4 shows a before and after illustration of expanding a block storage system. Adding additional capacity to a block system requires many steps. In the diagram, LUNs 1, 2 and 3 must each be expanded individually. This typically means adding several additional hard drives for each LUN to the array. Each LUN will have to be modified one at a time. Once the array is updated, each of the media servers must be updated as well. Since each media server manages the file system on the LUN it is attached to, it will have to be told about the new space. Depending on the operating system and file system in use, this could be very involved. An interesting side effect of having to add capacity to each LUN is often the capacity expansion will be larger than needed. If 10 TB of additional capacity is needed to support operations, it will most likely be difficult to allocate 3.33 TB to each LUN. Instead, LUN may have to grow by 4 or more drives to maintain RAID configurations. With 3 LUNS, that and a relatively small 2 TB drive, this 10 TB growth requirements requires 24 TB of storage. Expanding a NAS system in this use case is easier and more efficient than block. Figure 5: Expanding a NAS system Comparing File (NAS) and Block (SAN) Storage 9
Figure 5 shows the same before and after as the block example, but with a NAS system. Expanding a NAS system requires adding capacity to the volume hosting the share, in the case of example above, CIFS. This will normally require adding some number of hard drives to the system. Because a single volume is shared with all of the media servers, only one volume needs to be expanded. Once the space is added, the servers immediately have access, as they interact with the system through the share. The storage system manages the file system, relieving the need to make changes on the media servers. As environments get larger, this becomes a bigger benefit. Media servers are often added to backup and archive environments to increase processing capability and bandwidth. In a block storage system, adding an additional media agent means creating an additional LUN. This added expense and complexity can reduce the likelihood that a media server will be deployed as soon as needed. With a NAS based system, the new media server is pointed to the same share as the existing ones. If additional space isn t needed, then no changes need to be made on the storage system. If more space is needed, then any capacity added will be available to all media servers. O T H E R BENEF I T S O F NAS We have seen that the operations benefits of a NAS based storage system in backup and archive use cases are significant. There are some additional considerations as well to look at with using an IP network. These are true for any IP storage, NAS, Object or the new block over IP technologies like iscsi. 1. Wide Area Networks WANs are generally IP networks. IP storage as access to the WAN connections. This is useful for data access from remote data users as well as for offsite replication. 2. Common Staff Every data center has an IP network and administrators to manage it. IP storage uses the same technology, allowing the training current administrators have to support the connectivity to the NAS system as well. 3. Error Checking TCP/IP has integrated error checking and delivery notification. This helps ensure that data is sent correctly and received. 4. Network Components Investments already made in a data center infrastructure can be leveraged for IP based storage. Instead of installing new Fibre Channel switches and HBAs, exiting Ethernet switches and HBAs can be used. If additional hardware is needed, IP network components are substantially less expensive than Fibre Channel ones. Comparing File (NAS) and Block (SAN) Storage 10
5. Tools many larger organizations have diagnostic tools to troubleshoot network issues. These tools and techniques can be used on any IP network, including storage networks. 6. 10 Gigabit coming to chipsets the next generation of Intel (Broadwell) processors will have native dual 10 gigabit interfaces directly from the CPU. This will help accelerate the transition to IP based storage over the next 3 years. Reviewing these benefits makes it obvious why many new storage technologies are looking for ways to leverage IP networks, even if they use it for block storage. C O N C L U S I O N File based and block based disk storage systems are both mature technologies that have served data centers well for many years now. They both have their advantages and disadvantages. Advances in network technology have brought performance of the two storage systems together and show a more promising future for Ethernet based storage. The flexibility of NAS provides some advantages for data access and expansion for backup and archive deployments over block based storage. In the end, this is what gives NAS the edge. Comparing File (NAS) and Block (SAN) Storage 11
Deep Storage Experts Spectra Logic develops deep storage solutions that solve the problem of long term storage for business and technology professionals dealing with exponential data growth. Dedicated solely to storage innovation for more than 35 years, Spectra Logic s uncompromising product and customer focus is proven by the largest information users in multiple vertical markets globally. Spectra enables affordable, multi decade data storage and access by creating new methods of managing information in all forms of deep storage including archive, backup, cold storage, cloud, and private cloud. For more information, please visit http://www.spectralogic.com. 303-449-6400 800-833-1132 (Fax) 303-939-8844 6285 Lookout Road Boulder, CO 80301 USA spectralogic.com