Technical Report RapidData Migration Solution Deployment and Migration Considerations David Peterson, NetApp January 2015 TR-4337 Abstract This document is for system administrators, architects, and NetApp technical field representatives who use or are considering the NetApp RapidData migration solution. It provides extensive conceptual information about the RapidData migration solution, including migration processing, deployment scenarios, configuration options, and support. It also describes migration considerations and best practices as well as restrictions and limitations.
TABLE OF CONTENTS 1 Introduction... 5 1.1 Functionality and Characteristics...5 1.2 Key Features and Benefits...6 1.3 Hardware Components...8 1.4 Key Networking Concepts...9 1.5 Additional NetApp Technologies... 10 1.6 Requirements and Limitations... 11 2 Configuration Overview... 14 2.1 Flow Director Command Line Interface... 14 2.2 RapidView Web Application... 14 2.3 Data Server Support Shell... 15 2.4 RapidData Planner... 16 3 Migration Planning... 17 3.1 Required Skill Set... 17 3.2 Common Physical Deployment Topologies... 18 3.3 Common Migration Use Cases... 20 3.4 Formal Migration Plan... 21 4 Migration and System Operations... 23 4.1 Migration Session and Tasks... 23 4.2 Phases of Migration... 24 4.3 Flow Director Port Pairs... 26 4.4 RapidData Intrasystem Communication... 26 4.5 Flow Director Operation and State Management... 27 5 Integrating RapidData into the Network... 29 5.1 Review of LAG Technology... 30 5.2 Flow Director Configuration Overview... 31 5.3 Assessing Environment Prior to Network Integration... 31 5.4 Designing a Path Through Flow Directors... 32 5.5 Changing Flow Director Configuration... 33 5.6 Recabling Storage Network... 34 6 Defining Storage Systems and Migration... 35 6.1 Premigration Configuration Tasks... 35 6.2 Defining Storage Systems and Migration with RapidData Planner... 36 2 RapidData Migration Solution Deployment and Migration Considerations TR-4337
6.3 Defining Storage Systems and Migration with RapidView... 38 6.4 Storage Cut-In and Caching... 38 7 Executing the Migration... 39 7.1 Starting Migration... 39 7.2 Monitoring Migration and System... 39 7.3 Transferring Authority... 40 7.4 Remounting Clients... 40 7.5 Completing Final and Postmigration Tasks... 41 8 Migration Considerations... 41 8.1 Migration Planning and Preparation... 42 8.2 Administrative Tools... 42 8.3 File-Access Protocols... 43 8.4 Logical Interfaces... 43 8.5 Preparation and Use of Storage Systems... 44 8.6 Data Caching and Policies... 46 8.7 Qtrees and Subdirectories... 46 8.8 Snapshot and SnapMirror Copies... 48 8.9 Task Boundary and Migration Operation... 49 8.10 Additional Factors Affecting System Performance... 49 9 Common Failure Scenarios... 49 9.1 Typical Migration Environments... 49 9.2 System Recovery Options... 50 9.3 Failure of Data Server Platform... 51 9.4 Failure of Single Flow Director... 52 9.5 Loss of Connection Between Data Server and Flow Director... 52 9.6 Failure of Source Storage Node in HA Pair... 52 9.7 Link Failure in LAG to Source Storage System... 53 9.8 Failure of Single Link to Source Storage System... 53 9.9 Unavailable Remote Destination Storage System... 53 10 Troubleshooting and Support... 53 10.1 RapidView Environmental Page... 54 10.2 RapidData Event Messages... 54 10.3 Alert Notifications... 54 10.4 NetApp AutoSupport Tool... 55 10.5 Advanced Debugging Tools and Techniques... 55 3 RapidData Migration Solution Deployment and Migration Considerations TR-4337
10.6 System Maintenance and FRU Support... 56 11 Summary of Deployment and Implementation Tasks... 57 11.1 Technical Assessment and Migration Planning... 57 11.2 Appliance Installation and Initial Configuration... 57 11.3 Network Integration... 57 11.4 Data LIF Definition... 57 11.5 Storage System Preparation... 57 11.6 Storage System and Migration Preparation... 58 11.7 Migration Execution and Monitoring... 58 11.8 Final and Postmigration Tasks... 58 References... 59 Core RapidData Publications... 59 Technical Reports... 59 Version History... 59 LIST OF TABLES Table 1) LIFs that must be defined to the RapidData migration solution.... 10 Table 2) Functional components of the RapidData Planner.... 16 Table 3) Summary of Flow Director actions.... 28 Table 4) RapidData system states.... 29 Table 5) Formats for presenting analytical data.... 40 Table 6) Summary of migration expectations based on configuration of destination storage system and qtree replication option.... 47 Table 7) Configuration of the Data Server recovery-mode option.... 51 Table 8) Configuration of the Flow Director recovery option.... 51 Table 9) Types of alert destinations.... 55 LIST OF FIGURES Figure 1) HA controllers, each accessed through a single 10Gb link.... 18 Figure 2) HA controllers, each accessed through a LAG consisting of two 10Gb links.... 19 Figure 3) HA controllers, each accessed through a LAG consisting of four 1Gb links.... 19 Figure 4) Single controllers, each accessed through one 10Gb link.... 20 Figure 5) Sample deployment using a two-link LAG.... 50 Figure 6) Sample deployment using single links.... 50 4 RapidData Migration Solution Deployment and Migration Considerations TR-4337
1 Introduction The NetApp RapidData migration solution offers a feature-rich appliance designed to migrate data between different storage controllers. 1.1 Functionality and Characteristics RapidData uses standard file-access and networking protocols to efficiently migrate active datasets with minimal disruption. Storage Migration and Data Caching Based on NFS The primary function of the RapidData migration solution is to copy data between storage systems as part of a migration. Each migration must be defined and executed as a complete event, and each is composed of one or more discrete copy operations. The data being migrated always flows in one direction, from a source storage system to a destination storage system. Usually, data is copied from a system running NetApp Data ONTAP operating in 7-Mode to a system running clustered Data ONTAP. An underlying data-caching engine supports the migration process. As clients access the source storage system, the data is cached internally. The caching process can be optimized according to the policy definitions created by the administrator. Caching improves the performance of storage access during migrations as well as during normal client file system access. The Network File System (NFS) protocol is fundamental to both the RapidData migration process and the caching process. NFS is used in the following ways: During a migration, RapidData acts as an NFS client in order to read data from the exports on the source storage system and write it to the destination storage system exports. RapidData intercepts the normal NFS traffic flowing between the clients and the source storage system as part of implementing the data-caching feature, as well as to maintain the integrity of the data on the destination storage system during a migration. RapidData caches only data that is intercepted during NFS client activity. Although the migration process is cache aware and attempts to read data from the cache before reading from the source storage system, it does not insert or update the data in the cache. Multicomponent Appliance The RapidData appliance is a multicomponent or composite system made up of three hardware devices working together to support migration and caching activities. The appliance consists of the following hardware: Data Server (one instance) Flow Director (two instances) The Data Server integrates with the two Flow Directors, along with custom software bundled with the hardware, to form a single system image. Inline Insertion into Storage Network The RapidData appliance is integrated directly into the storage network between the NFS clients and the source storage systems. The appliance must be within the cable distance limitations of the source storage systems because it must connect directly to the storage controllers. After integration, RapidData acts as a bump in the wire, operating at wire speed to inspect and process the client traffic. 5 RapidData Migration Solution Deployment and Migration Considerations TR-4337
1.2 Key Features and Benefits This section describes the many benefits and the rich set of features offered by the RapidData migration solution. Support for Transition to Clustered Data ONTAP Although the RapidData migration solution can be deployed to achieve multiple migration goals, it is optimized to support the migration of data from storage systems running Data ONTAP operating in 7- Mode to systems running clustered Data ONTAP. Transparent and Minimally Disruptive Operation RapidData migration provides a data-transition solution for customers who have very strict storageaccess and uptime requirements. The appliance is inserted inline into the network in front of the source storage systems. Because RapidData installs nondisruptively into a live network and is transparent to both the clients and the storage systems, it can intercept and examine NFS traffic in real time while still allowing the storage clients to continue accessing the source data. Scalability and Performance The RapidData appliance can scale to provide optimum performance for the largest NetApp storage customers. As the system scales, some of the applicable metrics that are tuned for a migration include files and bytes migrated, number of active migrations, and copy rate during migration. For more information about the current capabilities, refer to the latest Release Notes for RapidData RFD2410 Software, RapidData RDS310 System Software, and RapidData Planner 1.0. Powerful Management and Administrative Software The RapidData migration solution includes specialized software for managing the system and administering migrations. The software, which is bundled with the hardware, includes two primary user interfaces: RapidView, an intuitive web-based application used for configuring RapidData as well as for managing and monitoring the migrations and caching activity Support shell, a text-based command line interface (CLI) shell that enables more complex support and maintenance procedures These embedded applications, as well as other tools, are described in section 2, Configuration Overview. Storage Objects and Subvolume Migrations RapidData supports the migration of the following types of storage containers: Volumes Qtrees Subdirectories The ability to migrate qtrees and subdirectories allows you to define partial or subvolume migrations. In addition, both the source qtrees and the exported subdirectories can be promoted to volumes on the destination storage system. Both flexible and traditional volumes are supported. This flexibility allows you to define a wide variety of migration scenarios. 6 RapidData Migration Solution Deployment and Migration Considerations TR-4337
Flexible Data Reorganization As part of defining and executing a migration, customer data can be reorganized as needed. Three primary types of data reorganization are possible: Storage consolidation (fan-in). Data from several different volumes or other storage sources can be collected at a single destination. Data redistribution (fan-out). Data from a single source can be redistributed to several different storage destinations. Change of namespace. In certain situations, it might be necessary to modify the namespace for reasons such as convenience or alignment with business processes. Data and Metadata Caching The internal data-caching functionality improves overall performance. During a migration, the cache is checked first, before data is read from the source storage system. This design improves the performance of the migrations. In addition to the migration processes, normal client traffic is also directed to the cache first. The caching offloads data and storage-access operations to create additional headroom. In addition to file data, the file system metadata is also cached, which not only helps to improve performance but also is used to control the frequency of cache updates and storage access as well as to maintain data integrity. Remote Destination Storage Systems Although the RapidData appliance must be connected directly to the source storage systems, the destination storage systems can be either local or remote. The destination storage can be accessed through a layer 2 (switched LAN) or layer 3 (IP network) connection. Integrated Snapshot Copy Capability By using RapidView, the administrator can create NetApp Snapshot copies at both the source and the destination storage systems. The copies can be created either while the migration is in the presync phase or as part of the storage-cutover procedure. These Snapshot copies are consistent between the source and destination storage systems. Furthermore, the existing Snapshot copy schedules are able to continue operating normally on both systems. Smart Restart The smart restart feature allows migrations to be resumed and completed more quickly after being paused or experiencing a temporary failure condition. During a migration, certain attributes of the files being migrated are saved to nonvolatile storage in the RapidData appliance. If a paused migration is subsequently restarted, the saved file attributes are retrieved and compared to the live data files during the new discovery phase. Any files that have not been modified since the failure are not recopied to the destination. Flexible Per-Client Cutover After the data has been copied from the source to the destination storage system, the NFS clients must be remounted to the destination system. In planning for client remounting, there is flexibility regarding the following decisions: The order in which the clients are remounted How many clients are remounted at a time How long the entire remounting process takes If the remount process is properly planned and executed, it should have minimal impact on the clients. 7 RapidData Migration Solution Deployment and Migration Considerations TR-4337
Extensive Analytics As a part of normal operation, RapidData collects an extensive amount of data related to performance and general system activity. This analytic data is accessed primarily through the RapidView user interface and is displayed in a variety of different formats. The performance of several system areas can be displayed. Collectively, the RapidData analytics capability can be used to monitor performance, to adjust system and migration configuration parameters, and to help detect and resolve problems. Integrated Planning Capability The NetApp RapidData Planner software tool simplifies the process of defining a migration and provisioning the associated storage systems. It augments and enhances the RapidView application. The Planner provides the following benefits and features: Allows planning activities to begin weeks or months prior to the start of migration and prior to the installation of the RapidData appliance Performs an extensive set of prechecks that help make the planned migration successful Dynamically discovers information from both the source and the destination storage systems Updates the exports at both the source and the destination storage systems to prepare the migration (read-write and root access required) Provisions the destination storage systems by replicating certain definitions found on the source storage system, which reduces the administrative work involved and the possibility of error (root access required) Downloads from the Data Server the logical interface (LIF) definitions that are needed for the storage definitions associated with the migration Note: The LIF definitions must have been defined previously through RapidView. Publishes and uploads the entire migration to the RapidData Data Server, including the storage definitions Includes a local log, which enhances support and debugging 1.3 Hardware Components The RapidData migration solution is composed of two types of specialized hardware devices: the Data Server and the Flow Director. For more information about the hardware specifications, refer to the RapidData Appliance User Guide and Reference. Data Server The RDS310 Data Server is a two-rack-unit server appliance that is shipped with a preinstalled proprietary operating system and tightly coupled application software. It contains the following components: Two microprocessors (16 cores total) 384GB of memory 22 solid-state drives (SSDs) used for migration and caching Two hard disk drives (HDDs) used for statistics data and boot configuration Four 10GbE ports dedicated to Flow Director connectivity Redundant power supplies and fans The Data Server provides the following core functions: Data caching to improve performance Data-copy services to support migrations 8 RapidData Migration Solution Deployment and Migration Considerations TR-4337
Internal configuration database Monitoring and control services Analytics Web server for access to the RapidView administrative application Flow Director The RFD2410 Flow Director is a one-rack-unit high-performance, low-latency Ethernet network device (similar to a switch) that includes a preinstalled operating system with customized application software. It contains the following components: 24 small form-factor pluggable (SFP) or enhanced SFP (SFP+) ports A dedicated Ethernet management port LED status lights Redundant power supplies and fans A Flow Director performs the following core functions: Supports the physical integration of the RapidData appliance into the customer network through a recabling and reconfiguration procedure Regulates traffic flow: Note: Redirects NFS traffic to the Data Server for examination and processing (based on the TCP port number and the source storage IP address) Forwards traffic directly between the source storage system and the NFS clients The way these tasks are handled depends on several factors, including system configuration, current operational mode, and type of network traffic. Provides redundancy and resiliency as part of a Flow Director high-availability (HA) pair Maintains a distinct operational role or identity, with one unit acting as the top Flow Director and the other as the bottom Flow Director Best Practice Although the Flow Director s top and bottom operational roles are not related to its physical location, NetApp recommends rack-mounting the Flow Director equipment during installation to correspond with these roles. That is, the Flow Director in the top role should be installed and located above the Flow Director in the bottom role. 1.4 Key Networking Concepts Because the RapidData appliance is inserted into the storage network, several related networking technologies are important to its implementation and operation. Local Network Switch The local ( top-of-rack ) network switch is the device that is normally connected directly to the source storage system. Because it provides direct layer 2 connectivity to the source storage, this network switch is an important focal point in the deployment and operation of RapidData. As part of the network integration process, a recabling procedure must be completed to insert RapidData between the local network switch and the source storage system. In addition, the destination storage systems must be accessible from the local network switch, over either a local layer 2 (switched LAN) or a layer 3 (routed IP) connection. 9 RapidData Migration Solution Deployment and Migration Considerations TR-4337
Link Aggregation Link aggregation (LAG) is an industry-standard technique for combining multiple network links between two nodes into what appears to be a single link. The NetApp storage systems provide three LAG operational modes that can be configured on an interface group: single mode, static multimode, and dynamic multimode, which uses the Link Aggregation Control Protocol (LACP). The RapidData migration solution supports LAG technology in two primary ways: The connection between the Data Server and each Flow Director is a separate LACP LAG consisting of two 10Gb links. Depending on the customer configuration, the connection between the local network switch and the source storage can be a LAG. This connection is recabled through the Flow Directors during network integration. When a LAG is recabled through the two Flow Directors as a part of the network integration process, it might be necessary to reconfigure each of the Flow Directors to support the LAG. By default, the Flow Directors are able to support two two-port LAGs. Network topology can differ according Flow Director reconfiguration. For more information about the network integration process and the use of LAGs, refer to section 5, Integrating RapidData into the Network. VLAN Technology Virtual LAN (VLAN) technology provides an industry-standard technique for tagging LAN frames to control frame forwarding and create virtual LAN segments. The RapidData migration solution uses VLAN technology to identify and direct traffic within the single system image that contains the Data Server and the two Flow Directors. In general, the RapidData tags are inserted into a Flow Director port at ingress and removed from the corresponding Flow Director port at egress. In addition, the appliance can be placed in a customer network that is already using VLAN-tagged traffic. In this case, a second tag is added to the LAN frames flowing within the RapidData single system image. RapidData Logical Interfaces The LIFs listed in Table 1 must be defined as part of configuring and deploying the RapidData migration solution. They are fundamental to the way RapidData connects to the network and operates. Through these LIFs, the RapidData migration solution also supports the typical division between data and management networks in customer networks. Table 1) LIFs that must be defined to the RapidData migration solution. Logical Interface Flow Director management LIF Data Server management LIF Data Server data LIFs Description This is the interface that is used to access the Flow Director shell. This LIF also provides a pass-through capability to connect to the Data Server applications. This is a dedicated interface at the Data Server. One or more data LIFs must be defined and associated with the Data Server. These LIFs are used to connect to the storage systems. For more information about RapidData LIFs, refer to section 6, Defining Storage Systems and Migration. 1.5 Additional NetApp Technologies The RapidData migration solution also uses or is affected by the NetApp technologies described in this section. 10 RapidData Migration Solution Deployment and Migration Considerations TR-4337
NetApp Manageability SDK The NetApp Manageability SDK is a proprietary systems-management interface that is implemented in the NetApp storage systems. It is integrated into RapidData and is used in the following ways: The RapidData migration solution uses it for storage system management, including discovery and provisioning. The RapidData Planner uses it for multiple tasks: Storage system management, including discovery and provisioning Downloading LIF definitions from the RapidData Data Server Uploading migration and storage definitions to the RapidData Data Server The NetApp Manageability SDK (formerly Zephyr API, or ZAPI) provides the environment for developing and deploying the management applications as well as the format of the information transmitted over the network. NetApp AutoSupport Tool The NetApp AutoSupport tool is a proprietary technology for collecting and recording the operational details of NetApp storage systems. The AutoSupport tool is integrated into the RapidData migration solution. It can be used in the following ways: Note: It can be configured to generate RapidData operational data for sending to NetApp. It can be used by the RapidData Planner to extract information about the storage systems included with a migration. The information-extraction feature is available only to NetApp field personnel. Negotiated Failover The negotiated failover (NFO) configuration option is applied to an interface group on a NetApp storage system. When it is enabled, it provides for the orderly transfer of control from one storage controller to the other within an HA pair during a failure. The NFO configuration at the source storage system determines the response of RapidData to certain network or storage failures. 1.6 Requirements and Limitations Before selecting and deploying the RapidData migration solution, be aware of the solution s core requirements and limitations. For more detailed information about the current operational limitations, refer to section 8, Migration Considerations, as well as to the latest Release Notes publications. File-Access Protocols and Clients This section describes the restrictions and requirements related to file-access protocols. NFS Version 3 The RapidData migration solution supports only the NFS version 3 (NFSv3) protocol. It does not support other file-access protocols, including any of the following: NFSv2 NFSv4 Common Internet File System (CIFS) iscsi SAN 11 RapidData Migration Solution Deployment and Migration Considerations TR-4337
In general, all file-access traffic except for NFSv3 traffic is forwarded through the Flow Directors, bypassing the migration and caching processes in the Data Server. NFS Clients In general, the RapidData migration solution supports any client complies with the NFSv3 standard, including 32-bit as well as 64-bit clients. Network Lock Manager The RapidData migration solution does not recognize or inspect Network Lock Manager (NLM) protocol traffic. Therefore, it does not support the transparent migration of data accessed by applications that use NLM. However, installations with NLM can still use RapidData. In this case, a simultaneous or big-bang remount of all clients (while in the cutover state) should be used. Storage Systems This section describes the characteristics and restrictions related to the storage systems used in the solution. Note: Always check the NetApp Interoperability Matrix (IMT) for the most up-to-date information about the currently supported configurations. Common Characteristics The following characteristics apply to both source and destination storage systems: Only NetApp storage systems are supported. Aggregates can be 32-bit, 64-bit, or hybrid. RapidData must be able to connect to the storage systems and issue calls through the NetApp Manageability SDK. All NFS exports participating in a migration must be accessible to the data LIFs defined to the RapidData appliance by using RapidView. Source Storage The following additional characteristics apply to the source storage systems: Only source storage systems with Data ONTAP 7.3.3 or later operating in 7-Mode are supported. The source storage system must be connected directly to the RapidData appliance. vfiler units are supported. RapidData must have access to all of the data to be migrated (read-write and root access required). Destination Storage The following additional characteristics apply to the destination storage systems: Only destination storage systems with Data ONTAP 8.1.2 or later are supported. Both clustered Data ONTAP systems and systems operating in 7-Mode are supported. The location is flexible and can be either physically local (switched) or geographically remote (routed IP). RapidData must have read-write and root access to the parent exports. Networking This section describes restrictions and requirements related to the networking and security technologies. 12 RapidData Migration Solution Deployment and Migration Considerations TR-4337
IP Version 4 Only IP version 4 is supported. IP version 6 and other network layer protocols are not supported. Jumbo Frames Jumbo Ethernet frames are not supported on the Data Server for NFS traffic. However, the RapidData migration solution can coexist in a jumbo TCP environment. In this case, the maximum transmission unit (MTU) size is negotiated down to 1,500 bytes. Other protocols (such as iscsi, HTTP, FCoE, and SSH) that bypass the Data Server maintain their native jumbo (or other) MTU size. Network Services and Security The following standard application-layer network services are required for installing and operating the RapidData migration solution: Note: Domain Name System (DNS) Network Time Protocol (NTP) Network Information Service (NIS) Kerberos security is not supported. VLAN Tagging As described in section 1.4, Key Networking Concepts, RapidData can be inserted into a network in which the traffic has already been VLAN-tagged once. However, RapidData cannot be inserted into a network that already contains double-tagged VLAN traffic. RapidData Planner The RapidData Planner can optionally be used to define a migration and the associated storage systems. The software has the following restrictions: It can be installed only on a supported version of Microsoft Windows. It requires a recent version of Microsoft Internet Explorer. For more information, refer to the RapidData Planner 1.0 Installation and User Guide and the Release Notes for RapidData Planner. Additional Restrictions Additional restrictions apply to the various capability metrics. These restrictions are related to the supported maximum number of the following items: Files across all active migrations Bytes across all active migrations Active source storage controllers Source storage systems associated with a migration session Destination storage systems associated with a migration session Defined and active migration sessions Configured tasks across all migrations Bytes per second (copy rate) IP addresses per source storage system Qtrees per volume Qtrees per migration session 13 RapidData Migration Solution Deployment and Migration Considerations TR-4337
For more information about these restrictions and about the supported versions of Data ONTAP, refer to the RapidData documentation, including the current Release Notes. 2 Configuration Overview This section describes the options provided by the RapidData migration solution for configuring both the system itself and the data migrations. 2.1 Flow Director Command Line Interface The Flow Director includes a text-based command line interface (CLI) that is used to perform configuration and diagnostic tasks. The Flow Director CLI is used during installation and initial configuration as well as during the network integration process. Accessing the Flow Director The Flow Director CLI can be accessed in two ways: Through a local physical or emulated terminal that is attached directly to the Flow Director by a serial cable By issuing an SSH command from another workstation in the network, using either the IP address or the fully qualified domain name of the Flow Director You should log in using the user account admin. You must use the password that was set during initial configuration of the RapidData appliance. Types of Commands and Tasks The categories of available commands include those used to perform the following tasks: Switching among the various command privilege levels within the CLI Displaying the current system and port configuration Modifying the system and port configuration and save changes to permanent storage Resetting the Flow Director configuration Accessing the internal operating system shell environment Updating the software Rebooting the machine Refer to section 5, Integrating RapidData into the Network, for an example of how the CLI can be used during the network integration process. For more complete descriptions of the commands, refer to the RapidData Migration Solution Deployment and Migration Guide and to the RapidData Appliance User Guide and Reference. 2.2 RapidView Web Application RapidView is the primary application used to administer the RapidData migration solution and its datamigration activities. This web application is bundled with the Data Server and can be accessed through a current-version web browser. Accessing RapidView You must open a browser to access RapidView. Using a Flow Director management LIF, access RapidView through one of the Flow Director web pages. 14 RapidData Migration Solution Deployment and Migration Considerations TR-4337
Functionality RapidView provides functionality in the categories of core system configuration, migrations and data caching, analytics, and support. Core System Configuration The following aspects of the RapidData system are configurable: Core Data Server settings, including the following: System name Recovery options for the Data Server and the Flow Directors Data Server management LIF Data LIFs Alert notifications AutoSupport tool Migrations and Data Caching The following configuration tasks support data migrations and caching: Creating storage and migration definitions Defining caching policies Executing migrations Performing final and postmigration tasks Analytics RapidData collects information about system and migration activity. This data, which can be displayed in RapidView in several different formats, includes the following categories: Several areas of overall system performance Specific data about the performance of migrations and caching Support Procedures RapidView provides access to the RapidData support and maintenance features. The following tasks can be performed: Displaying the current environmental information Updating the Data Server software Generating a debug package (normally sent to NetApp support) Viewing the local event log 2.3 Data Server Support Shell In addition to using RapidView, you can also access the Data Server through the support shell, which provides a CLI that can be used to perform certain diagnostic and maintenance procedures, normally with the assistance of NetApp support or a NetApp field representative. Accessing Support Shell The support shell provides restricted and nonroot access to the internal Data Server operating system shell. It can be accessed in several ways, including the following: 15 RapidData Migration Solution Deployment and Migration Considerations TR-4337
Through the SSH command from a network workstation by using one of the Flow Director management LIFs Through a local terminal connected to the Data Server by a serial cable When logging in to the support shell, you must use the same user account that is used to access RapidView. Specifically, the admin account is used along with the corresponding password. No general file or directory browsing is permitted after login; the user is restricted to the local directory of the admin user account. RapidData Commands The commands that are provided with the support shell can be used to perform tasks such as the following: Resetting the Data Server Initializing a local disk drive or RAID array Displaying certain system information Rebooting the Data Server For more information about the support shell, refer to the RapidData Appliance User Guide and Reference. 2.4 RapidData Planner The NetApp RapidData Planner is a Windows-based software tool that simplifies and enhances the process of defining a RapidData migration. The Planner connects to both the local storage systems and the Data Server as part of defining and publishing a migration. Installation and Initial Configuration In order to install the RapidData Planner, you must first download it to the local workstation. The software is distributed as a Windows Installer package and can be installed as a standard Windows application. After installation, you must perform the initial configuration, which includes selecting a user name and the corresponding password. After installation is complete, you can start the Planner application and log in. Functional Components The RapidData Planner functionality is organized into four main components, which are listed in Table 2. Each of these components can be accessed through a separate tab at the top of the Windows user interface. Table 2) Functional components of the RapidData Planner. Tab RapidData Config Storage Systems Migration Sessions Log Interactions Description Defines the RapidData appliance that will be configured Defines and discovers the source and destination storage systems Defines and publishes a migration, including performing the required prechecks and storage provisioning Displays informational and diagnostic messages 16 RapidData Migration Solution Deployment and Migration Considerations TR-4337
Usage Patterns The RapidData Planner can be used to define and publish a complete migration, or it can be used early in the migration process to define the storage systems before the RapidData appliance is installed. Define and Publish a Complete Migration After the RapidData migration solution has been installed and is available, a complete migration can be defined and published. The typical process of creating a migration involves completing the following tasks in order: 1. Define the target RapidData appliance that will perform the migration (this is the location where the configuration information will be published). 2. Define both the source and the destination storage systems that will participate in the migration. 3. Create the migration and publish it to the RapidData system. A multistep pipeline (shown in the user interface) guides the administrator through the entire process of creating the migration. At the end of the pipeline, a set of prechecks is run and the storage can be provisioned. The final step is to publish the migration, a process that uploads the definitions to the target RapidData appliance. After the migration has been successfully published, the definitions are available through the RapidView user interface. Begin Migration Planning Before RapidData Appliance Is Available It is possible to begin using the Planner early in the migration cycle, before the RapidData appliance has been installed. In this case, the storage systems can still be defined in the Planner and discovered. The migration definition can also be started; however, it cannot be completed and published until the RapidData appliance has been installed and is reachable in the network. 3 Migration Planning Planning is the essential first step in the process of deploying the RapidData migration solution. 3.1 Required Skill Set A successful RapidData deployment requires a broad set of skills. For clarity and planning purposes, these skills can be arranged into several distinct roles. In a large data center, one or more people might fulfill a single role. At smaller sites, however, multiple roles might be carried out by the same person. Network and Data Center Technician The network technician is responsible for the physical installation and the initial configuration of the RapidData appliance. This technician must be competent in the installation of rack-mounted devices and cabling as well as with basic network commands and configuration tasks. Migration Architect The migration architect should have a high-level understanding of the customer s storage products and implementation. This architect should understand what data must be migrated and be able to develop a plan, including the schedule. For transitioning to clustered Data ONTAP, an understanding of the architectural and functional differences from the 7-Mode system is also necessary to fully optimize the transition. 17 RapidData Migration Solution Deployment and Migration Considerations TR-4337
Storage Administrator The storage administrator should be familiar with the administration and use of the NetApp storage products in the customer organization, including concepts, terminology, and commands. This administrator is responsible for defining and executing the migration according to the migration plan by using the available tools, including RapidView and the RapidData Planner. NetApp Field Operations At many customer sites, a NetApp representative is available to assist with and in some cases to lead the RapidData implementation. The representative must acquire and maintain a complete understanding of both the solution s deployment and the customer s requirements. When needed, the representative can act as a liaison with the NetApp support team. 3.2 Common Physical Deployment Topologies This section describes some of the typical physical topologies that are commonly encountered in deployments of the RapidData migration solution. HA Controllers with 10Gb Link The deployment topology shown in Figure 1 includes two source storage controllers configured as an HA pair. Each storage controller is accessed through a single link. Only one link traverses each of the Flow Directors. Figure 1) HA controllers, each accessed through a single 10Gb link. HA Controllers with LAG Consisting of Two 10Gb Links The deployment topology shown in Figure 2 includes two source storage controllers configured as an HA pair. Each storage controller is accessed through a two-link LAG. Two links traverse each of the Flow Directors. 18 RapidData Migration Solution Deployment and Migration Considerations TR-4337
Figure 2) HA controllers, each accessed through a LAG consisting of two 10Gb links. HA Controllers with LAG Consisting of Four 1Gb Links The deployment topology shown in Figure 3 includes two source storage controllers configured as an HA pair. Each storage controller is accessed through a four-link LAG. Four links traverse each of the Flow Directors. Figure 3) HA controllers, each accessed through a LAG consisting of four 1Gb links. Single Controllers with 10Gb Link The deployment topology shown in Figure 4 includes two source storage controllers, each accessed through a single link. One link traverses each of the Flow Directors. 19 RapidData Migration Solution Deployment and Migration Considerations TR-4337
Figure 4) Single controllers, each accessed through one 10Gb link. 3.3 Common Migration Use Cases Four scenarios are commonly used to support the customer data-migration objectives: Simple volume-to-volume migration Qtree promotion and destination namespace preservation Qtree or subdirectory consolidation to a single destination volume Data redistribution to multiple volumes or storage virtual machines (SVMs, formerly called Vservers) Simple Volume-to-Volume Migration Prerequisites: There must be one or more volumes on the source storage system. The goal of the simple volume-to-volume migration scenario is to migrate the data on each source volume in its entirety to a corresponding volume on the destination storage system. Qtree Promotion and Destination Namespace Preservation Prerequisites: There must be one or more qtrees on the source storage system. In addition, you must create a volume for /vol and junction in the volumes at the destination storage system. The goal of this migration scenario is to promote one or more qtrees, each to a separate volume on the destination storage system, while preserving the namespace. Qtree or Subdirectory Consolidation to Single Destination Volume Prerequisites: There must be multiple qtrees, subdirectories, or both on several different volumes, on either the same or different source storage controllers. The goal of this scenario is to consolidate a qtree or a subdirectory by migrating all of the data to a single volume on the destination storage system. This fan-in reorganization allows you to use one migration task to copy an entire source volume, including all of the qtrees and subdirectories it contains. Alternatively, it might be necessary to define multiple tasks, one for each qtree or subdirectory. That decision must be based on the configuration of the customer s storage. 20 RapidData Migration Solution Deployment and Migration Considerations TR-4337
Data Redistribution to Multiple Volumes or SVMs Prerequisites: All of the data must be on a single volume on the source storage system. The goal of this scenario is to redistribute data by segregating it and migrating it to multiple destination volumes or SVMs. This fan-out reorganization results in a namespace change. 3.4 Formal Migration Plan For every migration, you should create a formal migration plan. Before developing the plan, you must perform a thorough technical assessment of the environment and establish clear goals for the migration. Technical Assessment of Migration Environment A technical assessment of the customer environment is required before the migration plan can be created. You must collect a variety of information during the technical assessment. Source and Destination Storage Systems Collect the following information about each storage system that will participate in a migration: Its role in the migration (either source or destination) Its Data ONTAP model and version Its operational mode (either 7-Mode or clustered Data ONTAP) Its location and accessibility with respect to: Local network switch RapidData Flow Directors Its management IP addresses and login credentials Network Connection Between Source Storage and Local Switch Collect the following information about the network connection between the local network switch and the source storage system: Total number of links LAG configuration (if applicable): Number of links Operational mode Media type Network Connection to Destination Storage Systems Collect the following information about the network connection to the destination storage systems: Whether it is accessible from the local network switch Whether the connection link operates at layer 2 (switched LAN) or layer 3 (routed IP) Whether the source storage data interface can reach the destination data interface (LIF) NFS Clients Collect the following information about the NFS clients that access the data on the source storage system participating in the migration: 21 RapidData Migration Solution Deployment and Migration Considerations TR-4337
File-access protocols (including NFSv3) Number of clients Operating systems Size (32-bit or 64-bit) Ability to access both source and destination storage systems Use of an automounter program Application Workloads In addition to the technical aspects of the network, such as storage and networking, you should identify the specific application workloads that are involved as well as any special requirements or considerations. For example, determine whether the data on the source storage system can easily be segregated according to application workload. Migration Plan Contents Completing the initial technical assessment and your business plans will help you to articulate the goals for the migration. Those goals can then determine the formal plan for the migration. Your migration plan should include the following information: A complete technical description of the migration environment (with graphics whenever possible) Specific goals for the migration, identifying: Source and destination storage systems Individual storage objects and their promotion (if applicable) Namespace changes (if applicable) Data reorganization, including consolidation or redistribution Related or dependent applications and workloads Source storage system cut-in, either manually before the migration begins (to activate caching) or automatically when the migration is started How the application workloads are partitioned A description of how the source-to-destination copy operations map to the RapidData migration sessions and tasks A summary of tools to be used, including: RapidView RapidData Planner Data validation tools (if applicable) Specific steps needed to prepare or evaluate the migration environment, including any additional prechecks to be performed by the migration team A strategy for remounting the clients to the destination storage systems Scheduling requirements for the migration, including: Start date Time allocated Requirements and time allowed for the client-remounting process Anticipated completion date Potential problems and tentative responses, including back-out steps if needed Plans to verify that the data has been migrated properly and that the related applications function properly after the migration 22 RapidData Migration Solution Deployment and Migration Considerations TR-4337
A list of the participants, both internal personnel and NetApp field support engineers How responsibility for each of the tasks is assigned to the participants 4 Migration and System Operations The RapidData migration and caching functionality is implemented as a set of core processes running on the Data Server and the Flow Directors. These processes interoperate to support the execution of the migrations as well as to manage the system as a whole, according to the definitions established by the administrator. 4.1 Migration Session and Tasks A migration session is the fundamental administrative definition used to describe a migration. Each session is composed of one or more separate data-copy tasks. The migration of the storage containers (volumes, qtrees, subdirectories, and files) identified in the migration plan must be mapped to the RapidData migration sessions and tasks. Session A migration session is a type of container or abstraction that defines and anchors the migration. Each session has the following characteristics: It has a unique name within the system. It contains and manages one or more tasks. It is associated with a single NetApp source storage system. It can be associated with one or more NetApp destination storage systems (up to a maximum allowed value). It contains other session-wide configuration options that affect the operation of the migration. At any given time, it has a single state associated with it. After the migration session is defined, the administrator can control it. A session can be started, paused, completed, or deleted as needed. Tasks The migration tasks are defined within a session, and it is these tasks that migrate the data. Each task has the following characteristics: It is associated with a single migration session. It identifies a single storage container to be copied along with all of its contents. It defines the complete source and destination paths. At any given time, it has a single state associated with it. Task Operation and Boundary Each task is a complete description of a single source-to-destination copy operation. In a task definition, the source path identifies the root or starting point at the source storage system, whereas the destination path identifies the target at the destination storage system. Data is copied from the source to the destination storage, beginning at the source root point. In general, everything at the root level of the source path and below in the directory hierarchy is copied to the destination. However, if another task in the same migration session identifies a more specific source directory path for certain data, that data is not copied. 23 RapidData Migration Solution Deployment and Migration Considerations TR-4337