Disk Storage Systems Module 2.5 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 1
Disk Storage Systems After completing this module, you will be able to: Describe the components of an intelligent storage system Describe the configuration of a logical disk Discuss the methods employed to ensure that a host can access a storage volume Discuss back end volume protection Discuss front end host configuration Describe the I/O flow from the back end to the physical disks 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 2 At this point, you have learned how disks work and how they can be combined to form arrays, otherwise known as RAID sets. Now we are going to build on those concepts and add intelligence to those arrays, making them even more powerful. Throughout this module we will refer to this as an intelligent storage system. Disk Storage Systems - 2
Lesson: Intelligent Storage System Overview After completing this lesson, you will be able to: List the benefits of intelligent storage systems Compare and contrast integrated and modular approaches to intelligent storage systems Describe the I/O flow through the storage system Describe the logical elements of an intelligent storage system 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 3 In this lesson, we will take a high level look at the components of a disk storage system as well as two approaches to implementing them: integrated and modular. Disk Storage Systems - 3
What is an Intelligent Storage System? A disk storage system which distributes data over several devices and manages access to that data. When implemented properly, it provides the following benefits over individual storage devices: Increased capacity Improved performance Easier data management Better data availability More robust backup/restore capabilities Improved flexibility and scalability 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 4 The benefits of an intelligent storage system are shown above. Performance is one of the most important benefits. Vendors are continually working to improve performance and heavily market their performance numbers. Disk Storage Systems - 4
Monolithic (Integrated) Storage Systems FC Ports Port Processors Monolithic Cache RAID Controllers 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 5 Arrays generally fall into one of two categories, monolithic and modular. Monolithic storage systems are generally aimed at the enterprise level, centralizing data in a powerful system with hundreds of drives. They have the following characteristics: Large storage capacity Large amounts of cache to temporarily store I/Os before writing to disk Redundant components for improved data protection and availability Many built in features to make them more robust and fault tolerant Usually connect to mainframes or very powerful open systems hosts Multiple front end ports to provide connectivity to multiple servers Multiple back end Fibre Channel or SCSI RAID controllers to manage disk processing. Expensive This system is contained within a single frame or interconnected frames (for expansion) and can scale to support increases in connectivity, performance, and capacity as required. Monolithic storage devices can handle large amounts of concurrent I/Os on very large data applications. However, they have high upfront costs limiting their applicability to only the most mission critical applications. They also take up a large amount of space in the data center. Note: Monolithic arrays are sometimes called integrated arrays, enterprise arrays, or cache centric arrays. Disk Storage Systems - 5
Modular Storage Systems Modular Host Interface Cache RAID Controller A Host Interface Cache RAID Controller B Rack Servers FC Switches Disk Modules Control Module with Disks 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 6 Modular storage systems provide storage to a smaller number of (typically) Windows or Unix servers than larger integrated storage systems. Modular storage systems are typically designed with two controllers, each of which contains host interfaces, cache, RAID processors, and disk drive interfaces. They have the following characteristics: Small companies/department level Smaller disk capacity and less global cache Takes up less floor space and costs less Can start with a smaller number of disks and scale as needed Fewer front end ports for connection to servers Performance can degrade as capacity increases Cannot connect to mainframes Limited redundancy and connectivity Usually have separate controllers from the disk array Note: Modular storage systems are sometimes called midrange or departmental storage systems. Disk Storage Systems - 6
Elements in an Intelligent Storage System Intelligent Storage System Front End Back End Physical Disks Host Connectivity Cache Cache 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 7 Intelligent storage systems are organized into the following areas: Front End Cache Back End Physical disks Disk Storage Systems - 7
Intelligent Storage System: Front End Intelligent Storage System Front End Back End Physical Disks Host Connectivity Cache Ports Controllers Note: Include redundancy in the channels to and from the ports. 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 8 The front end provides the communication from the storage system to the host. Front end ports connect to the hosts in the environment. Ports are the external interfaces for connectivity to the host. Each storage port has processing logic responsible for executing the appropriate transport protocol for storage connections. For example, it could use SCSI, Fibre Channel, or iscsi. Behind the storage ports are controllers which route the data to the cache via the internal data bus. As soon as the cache receives the data, the controller can send an acknowledgement message back to the host. Note: To maintain data availability, the front end of the storage systems generally have multiple ports. This provides redundancy in case of a failure, and also helps to balance the load when the system is experiencing heavy use. The number of front-end ports on a midrange storage system generally ranges from 1-8; 4 is typical. On a large monolithic array, port counts as high as 64 or 128 are common. Disk Storage Systems - 8
Front End Command Queuing Without Command Queuing Request 1 Request 2 Request 3 Request 4 F R O N T E N D 4 3 2 1 3 2 1 4 With Command Queuing Request 1 Request 2 Request 3 Request 4 F R O N T E N D 4 2 3 1 3 2 1 4 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 9 As we have seen earlier, command queuing processes multiple concurrent commands based on the organization of the data on disk, regardless of the order in which the commands were received. The command queuing software will reorder commands so as to make the execution more efficient, and will assign each command a tag. This tag identifies when the command will be executed, just as the number you take at the deli determines when you will be served. Some disk drives, particularly SCSI and Fibre Channel disks, are intelligent enough to manage their own command queuing. Intelligent storage systems may make use of this native disk intelligence, and may supplement it with queuing performed by the controller. There are several command queuing algorithms that can be used. Here are some of the common ones. First In, First Out commands are executed in the order in which they arrive. This is identical to having no queuing, and is therefore inefficient in terms of performance. Seek Time Optimization - faster than First In, First Out. However, two requests could be on cylinders that are very close to each other, but in very different places within the track. Meanwhile, there might be a third sector that is a few cylinders further away but much closer overall to the location of the first request. Optimizing seek times only, without regard for rotational latency, will not normally produce the best results. Access Time Optimization - combines seek time optimization with an analysis of rotational latency for optimal performance. Note: The Queue Depth Setting defines the number of outstanding requests that are active (in the queue) at the same time. Many manufactures have configurable queue depths. Disk Storage Systems - 9
Intelligent Storage System: Cache Intelligent Storage System Front End Back End Physical Disks Host Connectivity Cache 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 10 Cache is very fast memory that improves system performance by isolating the hosts from the mechanical delays associated with physical disks. You have already seen that accessing data from a physical disk usually takes several milliseconds, because of seek times and rotational latency; accessing data from high speed memory will typically take less than a millisecond. The performance of reads as well as writes may be improved by the use of cache. Cache is discussed in more detail in the next lesson. Disk Storage Systems - 10
Intelligent Storage System: Back End Intelligent Storage System Front End Back End Physical Disks Host Connectivity Cache Controllers Ports 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 11 When data gets transferred from the cache, it is sent through the I/O bus to the back end, where it is routed to the correct drive. The back end provides the communication with the disks for read and write operations. Back end functionality is generally provided by disk controllers. The disk controller: Manages the transfer of data between the I/O bus and the disks in the storage system Handles addressing for the device, translating logical blocks into physical locations on the disk Provides additional, but limited, temporary storage for data Provides error detection and correction often in conjunction with similar features on the disks Allows multiple devices to communicate to the Host Bus Adapter (HBA) on the host Facilitates performance enhancement Disk controllers are implemented as hardware (with firmware a built-in program) that communicates with the disks via the disk interface, sending commands to initiate the read/write process on the disks. The design of the controller is vendor specific. To provide maximum data protection and availability, dual controllers provide an alternative path in case of a failure. This reliability is enhanced if the disks used are dual-ported; each disk port can connect to a separate controller. Having multiple controllers will also facilitate load balancing. Having more than one port on each controller will provide additional protection in the event of certain types of failure. Disk Storage Systems - 11
Intelligent Storage System: Physical Disks Intelligent Storage System Front End Back End Physical Disks Host Connectivity Cache 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 12 Physical disks are where the storage actually takes place. This could be a single disk drive or a more complex RAID set. Drives are connected to the controller with either SCSI (SCSI interface and copper cable) or Fibre Channel (optical or copper) cables. When a storage system is used in environments where performance is not critical, ATA drives may be used. The connection to the drives will then be made via parallel ATA (PATA) or serial ATA (SATA) copper cables. Some storage systems allow a mixture of SCSI or Fibre Channel drives and ATA drives. The higher performing drives are used for application data storage, while the slower ATA drives are used for backup and archiving. Disk Storage Systems - 12
I/O Example: Read Requests Intelligent Storage System Front End Back End Physical Disks Host Connectivity Cache 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 13 To read data from disk: 1. The host sends a read request which is routed to the appropriate drive. 2. The disk drive locates the information by positioning the read/write head at the specified location on the disk platter. 3. The information is read from the disk and sent back to the host. If the data was not found, or if it was damaged, the drive will return the appropriate error status via a message sent to the controller. 4. The host verifies that the data arrived properly (without errors) meaning no bits lost to power spikes, line noise on the bus, or other hardware related failures, and makes use of the data. Since requests for data can occur much more quickly than data can be physically retrieved from disk, an intelligent storage system uses caching to predict what information on the disk will be needed in the near future, and load it into cache before the host requests it. This helps to reduce I/O bottlenecks in the system. Disk Storage Systems - 13
I/O Example: Write Requests Intelligent Storage System Front End Back End Physical Disks Host Connectivity Cache 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 14 To write data to disk: 1. The Host sends the data along with the instruction to write the data at a particular address location. 2. The Host waits for confirmation that the data has been stored, before it replaces the data in main memory with new data. Disk Storage Systems - 14
What the Host Sees Intelligent Storage System Host LUN 0 LUN 1 Cache Back End Physical Disks Host LUN 2 LUN 0 LUN 1 LUN 2 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 15 Since intelligent storage systems have multiple disk drives, they use the disks in various ways to provide optimal performance and capacity. For example: A large physical drive could be subdivided into multiple virtual disks of smaller capacity. Several small physical drives can be combined together into a RAID set and presented as one virtual disk. A combination of the above smaller physical drives may be combined into one RAID set, which can then be split into multiple virtual disks, or LUNs With a virtual disk, the physical storage does not change only the way it is represented to the hosts. A Logical Unit Number (LUN) represents the address of the physical disk partition, and is used by the host when accessing disks. The controller uses translation tables to map storage locations on the individual disk drives in the RAID set to locations on the virtual drives that the host sees. These virtual drives are known as logical volumes, or LUNs, and their addressing is independent of the physical disks. Disk Storage Systems - 15
The Host and Logical Device Names Host Volume Manager /dev/rdsk/c1t1d0 /dev/rdsk/c1t1d1 LUN 0 LUN 1 Intelligent Storage System Cache Back End Physical Disks Host Volume Manager \\.\PhysicalDrive0 LUN 2 LUN 0 LUN 1 LUN 2 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 16 This example shows a single physical disk divided into 3 LUNs: LUN 0, 1 and 2. The LUNs are presented separately to the host or hosts. A host will see a LUN as if it were a single disk device the physical configuration of the drives is hidden by the storage system. The host assigns logical device names to disk devices; these naming conventions vary by platform. Examples are shown for both Unix and Windows addressing. Disk Storage Systems - 16
Disk Organization in a Storage System Intelligent Storage System Host LUN 0 Cache Back End Physical Disks LUN 0 Host LUN 1 LUN 1 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 17 In the previous example, an individual disk was broken up into logical units. The same concept can be applied to entire RAID sets. Here a RAID set consisting of 5 disks has been sliced, or partitioned, into several LUNs. In this example, LUNs 0 and 1are shown. Note how a portion of each LUN resides on each disk in the RAID set. Disk Storage Systems - 17
Lesson Summary Key points covered in this lesson: An intelligent disk storage system: Distributes data over several devices and manages access to that data Has a front end, cache, a back end, and physical disks. Use the virtual disks to provide optimal performance and capacity. Individual disks within a RAID set can be divided into logical units. The same concept can be applied to entire RAID sets. 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 18 Please take a few moments to review the key points covered in this lesson. Disk Storage Systems - 18
Lesson: Cache A Closer Look After completing this lesson, you will be able to: Define cache Distinguish between multipurpose cache and configurable cache Describe cache hits and misses Describe algorithms to manage cache Trace the I/O flow from the cache to the back end to the physical disks 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 19 We already mentioned that cache plays a key role in an intelligent storage system. At this point, let s take a closer look at what cache is and how it works. Disk Storage Systems - 19
What is Cache in a Storage System A memory space used by a disk storage system to reduce the time required to read data/write data. It is usually made from very fast memory Write Request Read Request Cache Data Acknowledgment 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 20 Cache is very fast memory, which the processor can access more quickly than main memory. Caching is very effective because most programs access the same data repeatedly. In some implementations, multiple applications can hold data in cache simultaneously. Memory in a storage subsystem has two main functions: Staging - temporary buffering of current read and write data for each I/O Caching - repository for frequently accessed data Read cache holds data in anticipation that it will be requested by a client. Write cache holds data written by a client until it can be safely stored on more permanent storage media such as disk or tape. Because cache is memory, it is volatile loss of power will cause memory content, and therefore host data, to be lost. Storage system vendors solve this problem in various ways the memory may be powered by a battery until AC power is restored, or write cache content may be saved to non-volatile storage, such as disk. An additional complication arises on many midrange storage systems, where cache is mirrored (i.e., each controller has exclusive access to its own cache). It needs a mechanism to ensure that cache is coherent, in other words that the two copies of the data are identical. Failover from one controller to another is only reliable if this cache coherency is guaranteed by the design. Disk Storage Systems - 20
How Cache is Structured Data Store Tag RAM 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 21 The amount of data that the cache can hold is based on the cache s size and design of the cache. A cache will normally consist of two parts: Data store - the part of the cache that holds the data. Tag RAM the part of the cache that tracks the location of the data in the data store. Entries in this area indicate where the data is found in memory, and also where the data belongs on disk. Additional information found here will include a dirty bit a flag that indicates that data in cache has not yet been committed to disk. There may also be time-based information such as the time of last access. This information will be used to determine which cached information has not been accessed for a long period of time, and may be discarded. In some cases read and write cache reside within the same memory space. This type of cache implementation is referred to as a multipurpose cache. The storage system will determine how much space should be devoted to reads and how much should be devoted to writes. Another implementation allows the user to determine the ratio of read cache to write cache. This cache implementation generally occurs in modular storage systems, since the amount of memory is limited. When large amounts of memory are available, as there is in integrated storage systems, read and write caches often have their own dedicated memory spaces. This type of cache implementation is known as a dedicated cache. Note: You may see references to global cache. This term indicates that there is one cache for the storage system which is shared by all the controllers in that storage system, rather than a separate cache per controller. Disk Storage Systems - 21
Read Cache Hits and Misses Data found in cache = Hit Read Request Cache No data found = Miss Read Request Cache 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 22 The processor initiates a memory request that is sensed by the cache controller. The requested address is used to access Tag RAM to determine whether required data is available in cache. When the requested data is found in the cache, it is known as a cache hit. The data is sent directly to the host, with no disk operation required. Besides providing fast response from memory components, cache hits can also shorten the I/O path. I/O processing time will be mostly dependent upon the host, not the array as I/O response time is minimal. When the data is not found in cache, the operation is known as a cache miss. When there is a cache miss, the data must be read from disk. Once the data is retrieved from disk it is moved to the host. It will normally also be placed in cache, as shown in the slide The read cache hit ratio (or hit rate), usually expressed as a percentage, describes how well the read cache is performing. To determine the hit ratio, divide the number of read cache hits by the total number of read requests. Cache misses degrade performance by lengthening the I/O path cache must be searched before the storage system attempts to find the data on disk. The physical task of locating and reading the data from the physical disk takes appreciably longer than a cache hit 5ms or more is typical. Here the I/O response time comprises a larger proportion of the I/O processing time. Note: A read cache hit can take less than a millisecond, while a read cache miss can take many times longer (the actual time depends on many factors). Remember that average disk access times for reads are often in the 10 ms range. Disk Storage Systems - 22
Algorithms Used to Manage Read Cache New Data Least Recently Used (LRU) Determines which items are accessed frequently/infrequently Discards least recently used data Oldest Data Read Ahead (pre-fetch) Accesses data sequentially and puts it into cache before it is requested May assume that data recently accessed will not be needed again. 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 23 Cache has a limited amount of space. Once it is full, the cache controller needs to determine which data will be retained, and which data will be removed to make room for new incoming data. The algorithm used should be based on how the application accesses data for maximum efficiency. Two algorithms commonly used for read cache management are: Least Recently Used (LRU) determines access frequency and discards least recently used data. This algorithm is simple to implement, and will usually allow a high hit ratio. It reflects the access pattern by keeping data which is frequently accessed. Read Ahead (pre-fetch) reads data from disk and puts that data into read cache before it is requested by the host. This operation will be triggered when the cache controller detects data accesses which are sequential. Some implementations assume that data recently accessed will not be needed again, and will discard that data rather than pre-fetched data. Pre-fetching minimizes the time that the host waits for data to be fetched from the storage system. Note: It is inefficient to pre-fetch too much data, so some systems enable you to set a limit for the maximum amount of data which may be pre-fetched. Disk Storage Systems - 23
Write Algorithms Write-through Cache Write Request Cache Acknowledgement Write-back Write Request Cache Acknowledgement 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 24 Write-through cache immediately moves data to disk, while retaining a copy in cache. Low performance benefits for writing data; benefits for reads Low risk of data loss Write-back cache keeps data from several write operations in cache and writes the data to disk at some later time. Improves write performance, especially on a busy system Can be risky with regards to data loss. This is why data must be protected from power failures. Disk Storage Systems - 24
Write Cache: Performance Manage peak I/O requests bursts through flushing Least-recently used pages are flushed from cache to the drives For maximum performance: Provide headroom in write cache for I/O bursts Coalesce small host writes into larger disk writes Improve sequentiality at the disk 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 25 Some of the things that cache can do to improve performance include: Manage peak I/O requests by absorbing large groups of writes called bursts without becoming bottlenecked by the speed of a physical disk. This is known as burst smoothing. Merging several writes to the same area into a single operation Because write cache is a limited resource, it must be managed carefully, especially in busy environments. The algorithms that manage the flushing of cached data to the disks must be efficient, and should adapt to changing data access patterns. The actual algorithms used are vendor-specific. Disk Storage Systems - 25
Lesson Summary Key points covered in this lesson: Cache is a memory space used by a disk storage system to reduce the time required to read data/write data. It can speed up both read and write operations. Cache read algorithms include: Least Recently Used (LRU) Read Ahead (pre-fetch) Cache write algorithms include: Write-through Write-back 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 26 Take a few moments to review the key points covered in this lesson. Disk Storage Systems - 26
Module Summary Key points covered in this module: An intelligent disk storage system distributes data over several devices and manages access to that data. Monolithic storage systems are generally aimed at the enterprise level, centralizing data in a powerful system with hundreds of drives. Modular storage systems provide storage to a smaller number of (typically) Windows or Unix servers than larger integrated storage systems. Cache is an important part of intelligent disk storage systems as it can be used to improve performance. 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 27 Please take a few moments to review the key points covered in this module. Disk Storage Systems - 27
Check Your Knowledge What are the parts of an Intelligent Disk Subsystem? What is the difference between a monolithic and a modular array? What is the difference between cache hit and a cache miss? What is the difference between Least Recently Used and Read Ahead cache? What is the difference between Write-through and Writeback cache? 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 28 Disk Storage Systems - 28
Apply Your Knowledge Upon completion of this case study, you will be able to: Describe the basic architecture of the CLARiiON modular storage array. Describe the basic architecture of the Symmetrix integrated storage array. 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 29 At this point, we will apply what you learned in this lesson to some real world examples. In this case, we will look at the architecture of the EMC CLARiiON and EMC Symmetrix storage arrays. Disk Storage Systems - 29
CLARiiON CX3-80 Architecture UltraScale Storage Processor Fibre Channel Mirrored cache FC CPU CPU 1/2/4 Gb/s Fibre Channel Front End CLARiiON Messaging Interface (CMI) Multi-Lane PCI-Express bridge link Fan Power supply Fan Fan UltraScale Storage Processor Fibre Channel FC FC FC SPS Power supply FC FC FC FC Fan SPS Mirrored cache CPU CPU 2/4 Gb/s Fibre Channel Back End 4Gb/s LCC 4Gb/s LCC 2/4 Gb/s Fibre Channel Back End 4Gb/s LCC 4Gb/s LCC 4Gb/s LCC 4Gb/s LCC 4Gb/s LCC 4Gb/s LCC Up to 480 drives max per storage system (CX3-80) 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 30 The CLARiiON architecture includes fully redundant, hot swappable components meaning the system can survive the loss of a fan or a power supply, and the failed component can be replaced without powering down the system. The Standby Power Supplies (SPSs) will maintain power to the cache for long enough to allow its content to be copied to a dedicated disk area (called the vault) if a power failure should occur. Storage Processors communicate with each other over the CLARiiON Messaging Interface (CMI) channels. They transport commands, status information, and data for write cache mirroring between the Storage Processors. CMI is used for peer-to-peer communications in the SAN space and may be used for I/O expansion in the NAS space. The CX3-80 uses PCI-Express as the high-speed CMI path. PCI Express architecture delivers advance I/O technology delivering high bandwidth per pin, superior routing characteristics, and improved reliability. When more capacity is required, additional disk array enclosures containing disk modules can be easily added. Link Control Cards (LCC) connect shelves of disks. Disk Storage Systems - 30
Assigning CLARiiON LUNs to Hosts CLARiiON disks are grouped into RAID Groups Disks from any enclosure may be used in a RAID Group All disks in a RAID Group must be either Fibre Channel or ATA A RAID Group is the RAID set discussed earlier A RAID Group may be a single disk, or RAID Level 0, 1, 1/0, 3 or 5 The RAID Group is then partitioned into LUNs All LUNs in a RAID Group will be the same RAID Level The LUNs are then made accessible to hosts CLARiiON-resident software ensures that LUNs are seen only by the hosts that own them 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 31 Making LUNs available to a host is a 3-step process: 1. Create a RAID Group Choose which physical disks should be used for the RAID Group and assign those disks to the group. Each physical disk may be part of one RAID Group only. 2. Create LUNs on that RAID Group LUNs may be created (Note: The CLARiiON term is bound ) on that RAID Group. The first LUN that is bound will have a RAID Level selected by the user; all subsequent LUNs must be of the same RAID Level. 3. Assign those LUNs to hosts When LUNs have been bound, they are assigned to hosts. Normal host procedures, such as partitioning, formatting and labeling, will then be performed to make the LUN usable. The CLARiiON software that controls host access to LUNs, a process known as LUN masking, is called Access Logix. Disk Storage Systems - 31
Copyright 2006 EMC Corporation. Do not Copy - All Rights Reserved. EMC Symmetrix DMX Array y Direct Matrix Interconnect y Dynamic Global Memory y Enginuity Operating Environment y Processing Power y Flexible Back-End Configurations y Fault-tolerant Design 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 32 The Symmetrix DMX series arrays delivers the highest levels of performance and throughput for high-end storage. It incorporates the following features: y Direct Matrix Interconnect Up to 128 direct paths from directors and memory Up to 128 GB/s data bandwidth; up to 6.4 GB/s message bandwidth y Dynamic Global Memory Up to 512 GB Global Memory Intelligent Adaptive Pre-fetch Tag-based cache algorithms y Enginuity Operating Environment Foundation for powerful storage-based functionality Continuous availability and advanced data protection Performance optimization and self-tuning Advanced management Integrated SMI-S compliance y Advanced processing power Up to 130 PowerPC Processors Four or eight processors per director y High-performance back end Up to 64 2 Gb/s Fibre Channel paths (12.8 GB/s maximum bandwidth) RAID 0, 1, 1 + 0, 5 73, 146, and 300 GB 10,000 rpm disks; 73 and 146 GB 15,000 rpm disks; 500 GB 7,200 rpm disks y A fully fault-tolerant design Nondisruptive upgrades and operations Full component-level redundancy with hot-swappable replacements Support: Dual-ported disks and global-disk hot spares Redundant power supplies and integrated battery backups Remote support and proactive call-home capabilities Disk Storage Systems - 32
Symmetrix DMX Series Direct Matrix Architecture 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 33 This shows the logical representation of the Symmetrix DMX architecture. The Front-end (host connectivity directors and ports), Cache (Memory) and the Back-end (directors/ports which connect to the physical disks) are shown. Front-end: Hosts connect to the DMX via front-end ports (shown as Host Attach ) on Front-end directors. DMX supports ESCON, FICON, Fibre Channel and iscsi front-end connectivity. Back-end: The disk director ports (back-end) are connected to Disk Array Enclosures. The DMX back-end employs an arbitrated loop design and dual-ported disk drives. I/Os to the physical disks are handled by the backend. Cache: All front-end I/Os (reads and writes) to the Symmetrix have to pass through the cache, this is unlike some arrays which will allow I/Os to by pass cache altogether. Let us take a look at how the Symmetrix handles front-end read and write operations: Read: A read is issued by a server. The Symmetrix will look for the data in the cache, if the data is in cache it will be read from cache and sent to the server via the front-end port This is a read hit. If the data is not in cache, then the Symmetrix will go to the physical disks on the back-end, fetch the data into cache and then send the data from the cache to the requesting server This is a read miss. Write: A write is issued by a sever. The write will be received in cache and a write complete will be immediately issued to the server. Data will be de-staged from the cache to the back-end at a later time. Enhanced global memory technology supports multiple regions and sixteen connections on each global memory director, one to each director. Each director slot port is hard-wired point-to-point to one port on each global memory director board. If a director is removed from a system, the usable bandwidth is not reduced. If a memory board is removed, the usable bandwidth is dropped. Disk Storage Systems - 33
Symmetrix DMX: Dual-ported Disk and Redundant Directors Disk Director 1 Disk Director 16 S P S P S P S P P S P S P S P S P = Primary Connection to Drive S= Secondary Connection for Redundancy 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 34 Symmetrix DMX back-end employs an arbitrated loop design and dual-ported disk drives. Each drive connects to two paired Disk Directors through separate Fibre Channel loops. Port Bypass Cards prevent a Director failure or replacement from affecting the other drives on the loop. Directors have four primary loops for normal drive communication and four secondary loops to provide alternate path if the other director fails. Disk Storage Systems - 34
Configuring Symmetrix Logical Volumes (SLV) Physical Disk Physical Disk Physical Disk Physical Disk Physical Disk Symmetrix Service Processor Running SymmWin Application Initial configuration of Symmetrix Logical Volumes is done via the Symmetrix Service Processor and the SymmWin interface/application A configuration file (IMPL.BIN) is created and loaded on to the array Subsequent configuration changes can be performed online using EMC ControlCenter (GUI) or by using Solutions Enabler (CLI) 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 35 All Symmetrix Arrays have a Service Processor running the SymmWin application. Initial configuration of Symmetrix Arrays has to be performed by EMC personnel via the Symmetrix Service Processor. Physical disks (in the disk array enclosures) are sliced into hypers or disk slices and protection schemes (RAID1, RAID5 etc.) are then incorporated, creating the Symmetrix Logical Volumes (we will discuss this further in the next slide). A Symmetrix logical volume is the entity that is presented to a host via a Symmetrix front-end port. The host views the Symmetrix Logical volume as a physical drive. Do not confuse Symmetrix Logical Volumes with host-based logical volumes. Symmetrix Logical Volumes are defined by the Symmetrix configuration while Hostbased logical volumes are configured by Logical Volume Manager software. EMC ControlCenter and Solutions Enabler are software packages which are used to monitor and manage the Symmetrix. Solutions Enabler has a Command Line Interface, while ControlCenter provides a Graphical User Interface (GUI). ControlCenter is a very powerful storage management tool Managing the Symmetrix is one of the many things that it can do. Disk Storage Systems - 35
RAID1 Symmetrix Logical Volume RAID1 SLV Data is written to two hyper volumes on two different physical disks which are accessed via two different disk directors Host is unaware of data protection being applied Different Disk Disk Director Director Hyper Volumes Physical Drive Logical Volume 04B Host Address Target = 1 LUN = 0 Physical Drive LV 04B M2 LV 04B M1 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 36 Mirroring provides the highest level of performance and availability for all applications. Mirroring maintains a duplicate copy of a logical volume on two physical drives. The Symmetrix maintains these copies internally by writing all modified data to both physical locations. The mirroring function is transparent to attached hosts, as the hosts view the mirrored pair of hypers as a single Symmetrix logical volume. A RAID1 SLV: Two hyper volumes from two different disks on two different disk directors are logically presented as an RAID1 SLV. The hyper volumes are chosen from different disks on different disk directors to provide maximum redundancy. The SLV is given an Hexadecimal address - In the example SLV 04B is a RAID1 SLV whose hyper volumes exist on the physical disks in the back-end of the array. The SLV is then mapped to one or more Symmetrix front-end ports (a target and LUN ID is assigned at this time). The SLV can now be assigned to a server. The server will view the SLV as a physical drive. On a fully configured Symmetrix DMX3 array one can have up to 64,000 Symmetrix Logical Volumes. The maximum number of SLVs on a DMX is a function of the number of disks, disk directors and the protection scheme used. Disk Storage Systems - 36
Data Protection Mirroring (RAID 1) Highest performance, availability and functionality Two hyper mirrors form one Symmetrix Logical Volume located on separate physical drives Parity RAID (Not available on DMX3) 3 +1 (3 data and 1 parity volume) or 7 +1 (7 data and 1 parity volume) Raid 5 Striped RAID volumes Data blocks are striped horizontally across the members of the RAID group ( 4 or 8 member group); parity blocks rotate among the group members RAID 10 Mirrored Striped Mainframe Volumes Dynamic Sparing SRDF (Symmetrix Remote Data Facility) Mirror of Symmetrix logical Volume maintained in a separate Symmetrix 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 37 Data protection options are configured at the volume level and the same Symmetrix can employ a variety of protection schemes. Dynamic Sparing: Disks in the back-end of the Array which are reserved for use when a physical disk fails. When a physical disk fails the dynamic spare will be used as a replacement. SRDF is a remote replication solution and will be discussed later on in the Business Continuity section of this course. Disk Storage Systems - 37
Assigning Symmetrix Logical Volumes to Hosts Configure Symmetrix Logical Volumes Map Symmetrix Logical Volumes to Front-end ports Performed via EMC ControlCenter or Solutions Enabler Make Symmetrix Logical Volumes accessible to hosts SAN Environment Zone Hosts to Front-end ports Perform LUN Masking Can be performed via EMC ControlCenter or Solutions Enabler LUN Masking information is maintained on the Symmetrix in the VCM Database (VCMDB) LUN Masking information is also flashed to all the front-end directors 2006 EMC Corporation. All rights reserved. Disk Storage Systems - 38 Assigning Symmetrix Logical Volumes to hosts is a 3 step process: 1. Configure SLVs (discussed in the last few slides) 2. Map the SLVs to front-end ports When a SLV is created it is not assigned to any front-end port, thus one must assign SLVs to front-end ports before a host can access the same. Mapping is the task of assigning SLVs to front-end ports. For redundancy one would map a device to more that one front-end port. 3. Make SLVs accessible to hosts SAN Environment Zoning and LUN masking has to be performed. We will discuss Zoning and LUN Masking in the SAN lecture later on in this course. Disk Storage Systems - 38