Using IBM FlashSystem V9000 TM to virtualize existing storage Summary: The ability to virtualize external storage systems is a pivotal feature provided by IBM FlashSystem V9000. IBM FlashSystem V9000 integrates the IBM software-defined storage codebase which has been virtualizing storage since the year 2003. IBM FlashSystem V9000 benefits from the experience gained over those years. IBM FlashSystem V9000 can be deployed with and virtualize capacity from both new and existing storage systems. This document details the procedure for migrating existing storage systems to be virtualized by FlashSystem V9000. 1
Introduction to storage virtualization IBM FlashSystem V9000 functions as a feature rich software-defined storage layer that virtualizes and extends the functionality of all managed storage. A unique capability of the IBM FlashSystem V9000 platform is its ability virtualize a wide array of external storage systems. IBM FlashSystem V9000 acts as the virtualization layer between the host and other external storage systems, providing flexibility and extending functionality to the virtualized external storage capacity. Up to 32 PB of external storage can be managed by a single IBM FlashSystem V9000, and because the storage is virtualized, volumes can be nondisruptively moved between external and V9000 storage capacity. This functionality enables very easy integration of IBM FlashSystem V9000 into existing storage environments with seamless data migration. It is important to understand the terminology involved with storage virtualization in the IBM FlashSystem V9000. Managed disks (MDisks): RAID arrays that are presented to the IBM FlashSystem V9000 from internal and external storage are called MDisks. Storage pool: IBM FlashSystem V9000 takes a single MDisk or aggregates multiple MDisks and creates a storage pool. Volume: A volume is a storage unit that is presented and accessed by host systems. Volumes are created from the storage capacity present in storage pools. Figure 1 displays a representation of the storage virtualization layers. Figure 1 Storage virtualization Many advantages are gained by virtualizing external storage systems behind V9000. Some of them include better performance through the V9000 caching, centralized management of multiple storage systems, volume abstraction from the back-end storage, and the ability to blend high performance flash storage with traditional hard-drive based arrays. FlashSystem V9000 can also migrate non-virtualized workloads from existing storage systems to virtualized workloads on any storage managed by V9000. 2
Pre-requisite information IBM FlashSystem V9000 supports the virtualization of many different storage system makes and models. IBM maintains and publishes a supported hardware list for each version of the IBM FlashSystem V9000 software; the appendix and resource section of this paper includes a link to the latest list. Once an external storage system is virtualized, servers and applications will access their storage through the IBM FlashSystem V9000 virtualization layer, so it is also important to verify that the server operating system and hardware configuration is supported. Some host operating systems or applications provide their own methods for migrating data between storage volumes, and some can even do so nondisruptively. Virtualization environments, for example VMware vsphere, can migrate virtual machines between storage volumes using Storage vmotion. Storage vmotion can nondisruptively migrate virtual machine files between VMFS volumes. A viable option for migrating to new storage within VMware vsphere is to utilize this feature. New storage is provisioned and configured on the vsphere hosts and virtual machines are migrated over. The process can be time consuming because each virtual machine must be migrated individually and Storage vmotion is not supported with virtual machines that have physical raw device mappings, as in the case of virtual machines configured in a Microsoft cluster. On the other hand, the migration process using IBM FlashSystem V9000 is simple, and although a short downtime window is needed, all volumes and data can be migrated at one time. In the case of VMware vsphere, the entire vsphere environment can be migrated immediately, including clustered virtual machines. The steps for importing an existing storage system to IBM FlashSystem V9000 are the same regardless of the make or model of the storage system. From the host perspective, volumes presented from the original storage system are removed and then added back from IBM FlashSystem V9000. This means hosts must be set up with the proper multi-pathing for IBM FlashSystem V9000 and this can vary by the host operating system. The appendix and resource section of this paper includes a link to IBM FlashSystem V9000 online documentation, which includes host configuration instructions. VMware vsphere is the host operating system used as an example in this paper, and the steps for importing an existing storage system that contains a vsphere environment are outlined in the following sections. Preparation steps In this example, IBM Storwize V7000 is the existing storage system which will be migrated into and virtualized by IBM FlashSystem V9000. V9000 acts as a host initiator when communicating with external storage systems. V9000 must be zoned to the external storage system and host masking must be configured for it. 3
Before making any changes to the storage configuration, it is a good idea to document the host and storage relationships. For a VMware vsphere environment this includes documenting what virtual machines each datastore contains, as well as which virtual machines have Raw-Device Mappings (RDMs). The example used in this paper has the following configuration: Storage volumes in use by VMware V7K-VMware-Datastore (VMFS Datastore) V7K-VMware-RDM1 (RDM volume) V7K-VMware-RDM2 (RDM volume) Virtual machines on VMFS Datastore VM-NoRDMs VM-RDMs Virtual machines with RDMs VM-RDMs o Volume V7K-VMware-RDM1 o Volume V7K-VMware-RDM2 Although not required, it is also recommended to document the virtual SCSI adapter location for each RDM drive. As shown in Figure 2, virtual machine VM-RDMs have RDM volumes attached to SCSI locations SCSI 1:0 and SCSI 1:1. Figure 2 RDM SCSI locations 4
A review of the steps which should be taken prior to any changes of the existing storage configuration involves: 1. Setting up SAN zoning and configuring host multipath software for each server which will connect to the IBM FlashSystem V9000. 2. Setting up SAN zoning and host masking for the IBM FlashSystem V9000 and the external storage system. 3. Documenting the storage and server configuration. Import external storage With the preparation steps complete, a brief downtime is required for transitioning storage access from the external storage system to IBM FlashSystem V9000. After all of the virtual machines are taken offline, they should be unregistered from the vcenter server inventory. Because storage will be briefly unmapped from the vsphere hosts, this ensures that no virtual machines are orphaned. It is also recommended to remove any RDM volumes from virtual machines as shown in Figure 3; these will be added back after the storage import is complete Figure 3 Removing RDM volumes After the virtual machines are offline and unregistered, it is time to remap the storage within the external storage system. The procedure for remapping the storage on the Storwize V7000 system 5
involves changing the logical volume mapping from the flashse4-3.flashse-ad.ibm.local host to the FlashV9000-1 host. With this change, the logical volumes are removed from the vsphere hosts and mapped to IBM FlashSystem V9000. Figure 4 shows this change being made for logical volumes V7K- VMware-Datastore, V7K-VMware-RDM1, and V7K-VMware-RDM2. Figure 4 Create logical volume mappings on FlashSystem V9000 This step should be repeated for all logical volumes. After remapping all the logical volumes to FlashSystem V9000, you need to issue a command to the vsphere hosts to rescan all host bus adapters (HBAs). This ensures that the logical volumes are removed from the vsphere hosts. Now that the logical volumes have been removed from the vsphere hosts and remapped to V9000, they must be detected and set as managed MDisks. The Detect MDisks button on the external storage view within the V9000 management GUI is used to detect the storage. Figure 5 shows the discovery process. 6
Figure 5 Detecting external volumes as MDisks The author found it beneficial to rename the newly discovered MDisks at this point because the storage is being provided by the external storage system. MDisks are renamed to match the logical drive names found in the IBM Storwize V7000 or the application environment. MDisks can be renamed by a Right click on MDisk>Rename. Figure 6 Renaming MDisks You can access the Import wizard from the Actions menu. This wizard helps you import the data stored on the MDisks to image-mode volumes, which are volumes that map directly to the MDisk. Additionally, the wizard can migrate the MDisks to an existing storage pool. You can generate a new storage pool by not selecting an existing storage pool. Figure 7, Figure 8, and Figure 9 show the import wizard and image-mode volumes. 7
Figure 7 Import wizard Use caching Figure 8 Import wizard - Select target storage pool Figure 9 Imported MDisks in temporary storage pool 8
You can now map the storage back to the vsphere hosts. Mappings can be easily created and modified with the Storwize V7000. Figure 10 shows the mappings created for each volume and vsphere host. Figure 10 Mapping image-mode volumes to vsphere host The following steps must be completed within the VMware vcenter server to finalize the storage changes: 1. Run a rescan operation on the vsphere hosts to detect the volumes being presented from FlashSystem V9000. 2. Use the Add Storage wizard to reattach the existing VMFS volumes. 3. Register the virtual machines to the vcenter server inventory. 4. Re-attach RDM volumes back to the virtual machines. 5. Power on the virtual machines and restore services. 9
Migrate external storage volumes After completing the previous steps, host servers are accessing the logical volumes which reside on the external storage system (Storwize V7000 in this case) through FlashSystem V9000. This is a supported configuration, but typically the desire is to migrate the data to V9000 flash storage. BecauseFlashSystem V9000 utilizes storage virtualization, this migration can be completed nondisruptively. FlashSystem V9000 provides two methods for migrating volumes between storage pools. The first method is to perform a migration using the Migrate Volume Copy function as shown in Figure 11. Figure 11 Migrate Volume Copy Migrate Volume Copy provides the ability to migrate multiple source volumes to a destination storage pool. The copies proceed at a predetermined rate, and the destination volumes will be a generic volume type. The second, more flexible and recommended method for migrating volumes between storage pools is to use the Add Mirrored Copy function, as shown in Figure 12. 10
Figure 12 Add Mirrored Copy This creates a second copy of the logical volume data, stored on a selected destination storage pool. The benefits of using this method are that the destination volume type can be changed to thin provisioned or compressed, and the copy rate can be adjusted by modifying the Mirror Sync Rate property for the volume. This is useful when a large number of volumes need to be migrated and you wish to minimize the performance impact to the external storage system. Once the volume copy completes, FlashSystem V9000 will maintain two copies of the volume on the source and destination storage pools. The volume properties window will display the details for Copy 0 and Copy 1, as shown in Figure 13. 11
Figure 13 Volume details To complete the migration, the original source volume copy, in this example Copy 0, can be deleted. These steps must be completed for all volumes being migrated. Summary IBM FlashSystem V9000 is a full featured all-flash storage system with software features that provide flexibility in how it can be integrated into existing environments. The ability to virtualize external storage systems means that V9000 can easily migrate existing data to flash capacity and continue to virtualize and use existing storage investments. The process for virtualizing an external storage system with V9000 is simple and can be accomplished with minimal application downtime when proper planning is used. Applications that provide their own data migration technologies, such as VMware vmotion, can still benefit from the external storage migration abilities provided by FlashSystem V9000. 12
Appendix and Resources: IBM FlashSystem V9000 ibm.com/systems/storage/flash/v9000/overview.html IBM Systems on PartnerWorld ibm.com/partnerworld/systems VMware website.vmware.com V7.3.x Supported Hardware List, Device Driver, Firmware, and Recommended Software Levels.ibm.com/support/docview.wss?uid=ssg1S1004620 IBM FlashSystem V9000 Knowledge Center and Online Documentation.ibm.com/support/knowledgecenter/ST2HTZ/landing/Flashsystem_V9000.htm?lang=en About the Author: Rawley Burbridge is a Storage Solution Architect with the IBM FlashSystem Solution Engineering (SE) team specializing in server, desktop, and storage virtualization usage with IBM FlashSystem products. 13