EMC DiskXtender for NAS Release 3.1

Transcription

1 EMC DiskXtender for NAS Release 3.1 Theory of Operations P/N REV A01 EMC Corporation Corporate Headquarters: Hopkinton, MA

2 Copyright EMC Corporation. All rights reserved. Published December, 2007 EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date regulatory document for your product line, go to the Technical Documentation and Advisories section on EMC Powerlink. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. All other trademarks used herein are the property of their respective owners. 2 EMC DiskXtender for NAS Release 3.1 Theory of Operations

3 Contents Preface Chapter 1 Chapter 2 Chapter 3 EMC DiskXtender for NAS Overview DiskXtender for NAS Operating systems DX-NAS installation DX-NAS licensing DX-NAS server port usage DX-NAS migration process DX-NAS file recall Primary storage About the Celerra system About the NetApp filer Differences between the Celerra and NetApp filer source Secondary storage Differences between DX-NAS versions Configuration Planning and Performance Tuning Primary storage considerations Effects of source file size on DX-NAS Effects of volatile source files on DX-NAS Effect of the full readback method Primary storage sizing formula DX-NAS server local disk space Sizing secondary storage Effect of source file size on secondary storage Effect of volatile source on secondary storage Managing secondary storage with orphan file scans Data deduplication Secondary storage considerations Deploying multiple DX-NAS servers DX-NAS Administration Jobs Policies Previewing a job Running a job EMC DiskXtender for NAS Release 3.1 Theory of Operations 3

4 Contents Monitoring a job Stopping a job Reports Migration report Orphan file report Primary storage report Secondary storage report Data deduplication report Printing reports Alerts and log files Alerts Printing alerts Log files Chapter 4 Chapter 5 Chapter 6 Chapter 7 File Migration Migrating files Per-file information Directory structure on the secondary storage Recalling files Recalling all files from the Celerra system Recalling all files from a NetApp filer Orphan file management File recall Source scan Orphan file scan Orphan file previews Backup and Recovery for DX-NAS Full data backups Orphan file management with full data backups Forcing a full data backup to read stubs in passthrough mode Backing up stub files Preparing primary storage for stub backups on a Celerra system Managing orphan files without backup synchronization Managing orphan files with backup synchronization Backing up secondary storage Restoring stub files from backup media Determining what files to restore from backup media Using Celerra Checkpoints with DX-NAS About checkpoints Source directory checkpoints Copying stub files from checkpoint directories Restoring files to a checkpoint directory Using a checkpoint directory as a DX-NAS source Secondary storage checkpoints DX-NAS Integration with other EMC Products Integrating DX-NAS with EMC VisualSRM Configuring VisualSRM to work with DX-NAS Integrating DX-NAS with DiskXtender for Windows Integrating DX-NAS with File System Manager for UNIX/Linux EMC DiskXtender for NAS Release 3.1 Theory of Operations

5 Contents Integrating with DiskXtender UNIX/Linux Storage Manager Integrating DX-NAS with EMC AVALONidm Using EMC Celerra HighRoad Chapter 8 Chapter 9 Chapter 10 High Availability Solutions for DX-NAS HA configurations for DX-NAS HA for DX-NAS source systems Implementing HA for a Celerra source Implementing HA for a NetApp filer source HA for the DX-NAS software Implementing the AutoStart Module for DX-NAS Running DX-NAS in a two-node Microsoft cluster Running DX-NAS in a two-node Linux cluster HA for DX-NAS secondary storage Implementing HA for a Celerra destination Implementing HA for a NetApp filer destination Implementing HA for secondary storage on a cluster Implementing HA for an EMC Centera Disaster Tolerant Strategies for DX-NAS Disaster tolerant strategies Synchronous data mirroring Point-in-time data mirroring Asynchronous data mirroring Failing over a disaster tolerant configuration Configuration Options for DX-NAS Two Data Movers on the same Celerra One Data Mover on the same Celerra Celerra or NetApp source to a Celerra destination Celerra or NetApp filer to DX-NAS server local disk Celerra or NetApp filer to network disk destination Celerra or NetApp filer to EMC CLARiiON ATA Celerra system or NetApp filer direct to EMC Centera NAS server to other storage device migration Glossary Index EMC DiskXtender for NAS Release 3.1 Theory of Operations 5

6 Contents 6 EMC DiskXtender for NAS Release 3.1 Theory of Operations

7 Figures Title Page 1 EMC Celerra provides file sharing for clients Celerra perspective of file access on local disk arrays Celerra perspective on file access for a migrated or offline file NetApp filer provides file sharing for clients Sources and destinations in the DX-NAS GUI Policy Wizard combines DX-NAS elements and search behaviors File list for a migration job preview Orphan file job preview listing Details pane for a migration job in progress Details pane for an orphan file job in progress Source scan job in progress information Example of a migration report An example of an orphan file report Alerts tab Updates to per-file information in DX-NAS database for migration activity Example of directory structure on secondary storage Connection string listing from Celerra Connection status information while recall is in progress Connection status after files have been recalled Orphan file creation after modifying a file Stub file orphan Example of full data backup of migrated files on Celerra Stub backup with corresponding secondary storage backup DX-NAS database backup EMC NetWorker User Recover window Celerra file system before checkpoint creation Celerra file system after checkpoint creation VisualSRM filter set to find PDF files Settings tab in the VisualSRM Console Celerra or NetApp filer to DiskXtender for Windows configuration Celerra or NetApp filer to a DiskXtender FSM destination Celerra or NetApp to FSM to AVALONidm configuration HA configurations for DX-NAS components Standby Data Mover on a Celerra system Data Mover failover and replication on a Celerra system AutoStart Module for DX-NAS configured on Solaris or Red Hat Linux AutoStart Module for DX-NAS configured on Windows DX-NAS installed in a two-node cluster configuration DX-NAS using a Celerra to provide HA for secondary storage HA secondary storage on Microsoft Windows or Linux cluster EMC DiskXtender for NAS Release 3.1 Theory of Operations 7

8 Figures 41 AutoStart Module for FSM for UNIX/Linux configured for HA Point-in-time mirroring with DX-NAS Example of asynchronous mirroring with DX-NAS Two Data Movers configured on the same Celerra system One Data Mover configuration Celerra source system to destination Celerra system NetApp filer source to a Celerra destination Celerra or NetApp filer to a local disk destination Celerra system or NetApp filer to network disk destination Celerra or NetApp filer source to EMC CLARiiON ATA destination DX-NAS migrating files directly to an EMC Centera system EMC DiskXtender for NAS Release 3.1 Theory of Operations

9 Tables Title Page 1 Supported operating systems Ports used by DX-NAS Support matrix for Data ONTAP Celerra and NetApp filer source systems comparison Differences between DX-NAS versions Sizing secondary storage Attributes used as to build match criteria Primary storage report Secondary storage report Data deduplication report DX-NAS and cluster specifications Requirements for a Linux two-cluster EMC DiskXtender for NAS Release 3.1 Theory of Operations 9

10 Tables 10 EMC DiskXtender for NAS Release 3.1 Theory of Operations

11 Preface As part of an effort to improve and enhance the performance and capabilities of its product lines, EMC periodically releases revisions of its hardware and software. Therefore, some functions described in this document may not be supported by all versions of the software or hardware currently in use. For the most up-to-date information on product features, refer to your product release notes. If a product does not function properly or does not function as described in this document, please contact your EMC representative. Audience This document is part of the EMC DiskXtender for NAS (DX-NAS), Release 3.1, documentation set. The information in this guide is primarily intended to familiarize pre-sales, technical support engineers, product specialists, instructors, and course developers with DX-NAS concepts. Readers of this document are expected to be familiar with the following topics: EMC Celerra administration EMC Centera administration Celerra FileMover API (formerly known as DHSM) NetWork Appliance (NetApp) filer administration Solaris, Linux, or Microsoft Windows operating system configuration and management Related documentation Related documents include: EMC DiskXtender for NAS Release 3.1 Multiplatform Version Release Notes EMC DiskXtender for NAS Release 3.1 Microsoft Windows Version Installation and Configuration Guide EMC DiskXtender for NAS Release 3.1 UNIX/Linux Version Installation and Configuration Guide EMC DiskXtender for NAS Release 3.1 Multiplatform Version Administration Guide EMC DiskXtender for NAS online help EMC Autostart Module for DiskXtender for NAS Release 3.1 Installation Guide Using Celerra FileMover Technical Module EMC Centera System Operator s Guide EMC Centera Product Guide EMC VisualSRM Installation Guide EMC DiskXtender for NAS Release 3.1 Theory of Operations 11

12 Preface Conventions used in this document EMC uses the following conventions for special notices. Note: A note presents information that is important, but not hazard-related.! CAUTION A caution contains information essential to avoid data loss or damage to the system or equipment.! IMPORTANT An important notice contains information essential to operation of the software. Typographical conventions EMC uses the following type style conventions in this document: Normal Used in running (nonprocedural) text for: Names of interface elements (such as names of windows, dialog boxes, buttons, fields, and menus) Names of resources, attributes, pools, Boolean expressions, buttons, DQL statements, keywords, clauses, environment variables, functions, utilities URLs, pathnames, filenames, directory names, computer names, filenames, links, groups, service keys, file systems, notifications Bold Used in running (nonprocedural) text for: Names of commands, daemons, options, programs, processes, services, applications, utilities, kernels, notifications, system calls, man pages Used in procedures for: Names of interface elements (such as names of windows, dialog boxes, buttons, fields, and menus) What user specifically selects, clicks, presses, or types Italic Used in all text (including procedures) for: Full titles of publications referenced in text Emphasis (for example a new term) Variables Courier Courier bold Used for: System output, such as an error message or script URLs, complete paths, filenames, prompts, and syntax when shown outside of running text Used for: Specific user input (such as commands) Courier italic Used in procedures for: Variables on command line User input variables < > Angle brackets enclose parameter or variable values supplied by the user [ ] Square brackets enclose optional values Vertical bar indicates alternate selections - the bar means or { } Braces indicate content that you must specify (that is, x or y or z)... Ellipses indicate nonessential information omitted from the example 12 EMC DiskXtender for NAS Release 3.1 Theory of Operations

13 Preface Where to get help Your comments EMC support, product, and licensing information can be obtained as follows. Product information For documentation, release notes, software updates, or for information about EMC products, licensing, and service, go to the EMC Powerlink website (registration required) at: Technical support For technical support, go to EMC Customer Service on Powerlink. To open a service request through Powerlink, you must have a valid support agreement. Please contact your EMC sales representative for details about obtaining a valid support agreement or to answer any questions about your account. Comments and suggestions about our product documentation are always welcome. To provide feedback: 1. Go to: 2. Click the Feedback link. EMC DiskXtender for NAS Release 3.1 Theory of Operations 13

14 Preface 14 EMC DiskXtender for NAS Release 3.1 Theory of Operations

15 1 EMC DiskXtender for NAS Overview This chapter discusses the following topics: DiskXtender for NAS Primary storage Secondary storage Differences between DX-NAS versions EMC DiskXtender for NAS Overview 15

16 EMC DiskXtender for NAS Overview DiskXtender for NAS The EMC DiskXtender for NAS (DX-NAS) is a client/server archiving solution that automatically migrates files stored on either an EMC Celerra Network Server or a Network Appliance (NetApp) filer to secondary storage. DX-NAS software comprises two components: a server and client. The DX-NAS server includes functionality to migrate files from primary storage to secondary storage by using user-defined policies. This process is commonly referred to as a migration process. The DX-NAS client provides a graphical user interface and a command-line interface in which you configure and manage DX-NAS primary and secondary storage, migration jobs, polices and schedules, and reports. Operating systems Table 1 The DX-NAS software can be installed in heterogeneous environments. Table 1 on page 16 provides a list of all operating systems supported by the DX-NAS software. Supported operating systems DX-NAS software Server Operating system Solaris 9 and 10 with the latest patches applied (32-bit or 64-bit) Red Hat Linux 4.0 Advanced Server Edition Microsoft Windows 2000 Microsoft Windows Server 2003 Client Solaris 9 and 10 (32-bit or 64-bit) Red Hat Enterprise Linux version 4 Microsoft Windows 2000 Microsoft Windows Server 2003 Microsoft Windows XP Microsoft Windows Vista DX-NAS installation The DX-NAS server can be installed on Solaris, Red Hat Linux, or Microsoft Windows. On Solaris or Red Hat Linux, the server is installed as two programs: DxnasServer DxnasDb On Microsoft Windows, the server is installed as two services: EMC DX NAS Database EMC DX NAS Server The following guides provide information for installing the DX-NAS software: EMC DiskXtender for NAS Release 3.1 UNIX/Linux Version Installation and Configuration Guide EMC DiskXtender for NAS Release 3.1 Microsoft Windows Version Installation and Configuration Guide 16 EMC DiskXtender for NAS Release 3.1 Theory of Operations

17 EMC DiskXtender for NAS Overview DX-NAS licensing DX-NAS software supports the following types of licenses: EMC Celerra source only NetApp source only Celerra and NetApp sources Data Deduplication DX-NAS server port usage Table 2 Table 2 on page 17 provides a list of the ports used by the DX-NAS server. Ports used by DX-NAS Port number Description 139 NetApp callback (not configurable) Note: If DX-NAS is used to migrate files from a NetApp filer, you cannot run Samba on the same machine where the DX-NAS server is running. 445 CIFS 2049 NFS 1976 DX-NAS database 3682 EMC Centera CLI EMC Centera Viewer EMC Centera communication 8080 DX-NAS web server (configurable) DX-NAS server (configurable) If you configured a firewall on the network where the DX-NAS server and client are located, enable access to ports 1976, 11001, and DX-NAS migration process The DX-NAS migration process copies files from primary storage to secondary storage. Migrating files is also commonly referred to as archiving or moving files. Before you migrate files with DX-NAS you must first configure a source and a destination. The DX-NAS source is either a Celerra system or a NetApp filer. The DX-NAS destination is a NFS or CIFS mountable/shared file system or an EMC Centera system. DX-NAS can be configured to manage multiple sources and multiple destinations regardless of whether the multiple sources or destinations are located on the same system or different systems. A migration policy identifies the files to be migrated from the source to the destination. The migration policy specifies criteria such as file size, file creation date, or days that have passed since the file was last accessed. For example, a migration policy might specify all files over 128 KB in size that have not been accessed in over 90 days. A migration job that you create, then specifies the migration policy. When the job runs, files that match the criteria specified by the policy are migrated from the source to the destination. DiskXtender for NAS 17

18 SB14 SB12 SB10 SB8 SB6 SB4 SB2 SB0 PS0 PS1 PS2 PS3 PS4 SMB0 SMB1 SB15 SB13 SB11 SB9 SB7 SB5 SB3 SB1 EMC DiskXtender for NAS Overview When the migration job runs, the source is scanned. Scanning is a discovery process that determines what files meet the criteria specified by the migration policy. Once the scan process identifies files to be migrated, DX-NAS determines the status of the file. Is the file offline? Does the file require remigration? If the file has already been migrated, no further action is necessary. If the file has not been migrated or requires remigration, the file is copied from the source to the destination. After a successful copy of the file from the source to the destination, DX-NAS marks the original source file as offline. This source file is now referred to as a stub file, and is usually 8 KB. The stub file no longer contains the file data. The file data is now located on the destination. The stub file contains only a pointer to the migrated file s new location on the destination. Per-file information about the migrated file is also stored in the DX-NAS database. Note: In releases before 3.0, per-file information about migrated files was stored in a file. This file was called an infofile. DX-NAS file recall The retrieval of a file from the destination is referred to as a file recall. In some instances, recalling files from the destination does not require the DX-NAS server software to be running. For example, if files originally stored on a Celerra source are migrated to a Celerra file system and then later recalled, the recall process retrieves the files using the Celerra FileMover API. DX-NAS is not in the data path when previously migrated files are recalled. If files, however, are migrated from a Celerra source to an EMC Centera device, the file recall process requires the DX-NAS server to be running. DX-NAS must translate the offline HTTP address from the stub file into an EMC C-Clip to recall the file from an EMC Centera device. Primary storage Primary storage (the source) is either a Celerra system or NetApp filer. DX-NAS works with the Celerra FileMover API or the Network Appliance Data ONTAP API to move the files from the source to secondary storage. About the Celerra system The Celerra system provides network-attached storage (NAS) to Solaris, Red Hat Linux, and Microsoft Windows clients as shown in Figure 1 on page 18. Windows NAS Client Celerra UNIX NAS Client GEN Figure 1 EMC Celerra provides file sharing for clients Solaris or Red Hat Linux clients use NFS to connect to the Celerra system and Windows clients use the CIFS protocol. From the perspective of both of these clients, 18 EMC DiskXtender for NAS Release 3.1 Theory of Operations

19 EMC DiskXtender for NAS Overview they are simply mounting volumes on, or mapping drives to, a specialized server in the Celerra cabinet called a Data Mover. Generally, NAS clients are not concerned with the configuration of the file system on the back-end disk array nor with the physical location of their files. However, from the Celerra perspective, the Data Mover manages the client connection and functions as an intermediate layer between the NAS clients and the physical storage. The Data Mover moves data between the storage system and the client machine. Figure 2 on page 19 illustrates this relationship. Celerra NAS client Windows Data Mover G:\file.avi NAS client UNIX/Linux /home/file.mpeg File system Back-end disk array Figure 2 Celerra perspective of file access on local disk arrays As with many storage solutions there are physical limits to the amount of storage that a Celerra can manage. Consequently, functionality was included in the data access in real time (DART) operating system to allow files to be moved by the Data Mover from a NAS file system to another location while preserving NAS client access to those files. The DX-NAS software employs this functionality by moving files through the DART FileMover API. (In earlier releases of DART, this functionality was called DHSM.) FileMover functionality in DART When a file is moved to secondary storage by using the Celerra FileMover API, DX-NAS leaves a small stub file on primary storage, and sets the offline attribute on the stub file. This stub is one block (usually 8 KB) in size and contains file attribute information as well as the location of the data portion of the file. Note: If alternate datastreams are associated with the file, the alternate datastreams are left unaffected by the migration. Primary storage 19

20 EMC DiskXtender for NAS Overview When a NAS client attempts to read a file that is offline, the Data Mover reads the file from secondary storage. Figure 3 on page 20 illustrates this point. Celerra NAS client Windows Data Mover Secondary storage on remote host NAS client UNIX/Linux /home/file.mpeg Shared file system File system Back-end disk array Figure 3 Celerra perspective on file access for a migrated or offline file Note: Depending on the FileMover configuration option, the file may be either read directly from secondary storage, or partially or completely retrieved to satisfy the read request. Supported DART versions The DX-NAS software supports the following versions of data access in real time (DART): or later or later Note: Versions of DART that are before are not supported. DX-NAS release 3.1 will support the next version of DART ( ) as a best effort only. The EMC NAS Support Matrix provides more information on supported software and hardware, Fibre Channel switches, and applications for Celerra network-attached storage products. The matrix can be viewed at About the NetApp filer NetApp filers provide network-attached storage to Red Hat Linux and Microsoft Windows clients. 20 EMC DiskXtender for NAS Release 3.1 Theory of Operations

21 EMC DiskXtender for NAS Overview The NetApp filer and DX-NAS can be configured to move files to secondary storage while leaving a stub on the NetApp filer for NAS clients to access. Figure 4 on page 21 illustrates a NetApp filer as a source device for DX-NAS. Windows NAS client NetApp Linux NAS client Figure 4 NetApp filer provides file sharing for clients The stub file on the NetApp filer is also 8 KB in size. DX-NAS, however, cannot migrate a file smaller than 8 KB off the NetApp filer. All migrations off the NetApp filer use a readback method of Full. NetApp filer requirements DX-NAS release 3.0 and later support NetApp filers as primary storage. The following requirements for using NetApp filers apply: DX-NAS server software installed on either Red Hat Linux or Microsoft Windows. Note: You cannot use a NetApp filer as primary storage if you are using DX-NAS server software on Solaris. Data ONTAP version 7.0.x or later installed on the NetApp filer. Table 3 on page 21 provides a support matrix for ONTAP versions that can be used with DX-NAS. Table 3 Support matrix for Data ONTAP ONTAP version File systems DX-NAS server 7.0.x CIFS only Note: ONTAP 7.0.x does not support NFS. Microsoft Windows CIFS Microsoft Windows NFS Red Hat Linux 7.2.1, 7.2.2, and CIFS Microsoft Windows NFS Red Hat Linux Before using a NetApp filer as a DX-NAS source, you must configure an NFS share and then mount it. The EMC DiskXtender for NAS Release 3.1 Installation Guides provide more information about configuring NetApp filers to be used as primary storage. Primary storage 21

22 EMC DiskXtender for NAS Overview Differences between the Celerra and NetApp filer source Table 4 on page 22 compares how DX-NAS works with Celerra and NetApp filer source systems. Table 4 Celerra and NetApp filer source systems comparison Area of difference Celerra source NetApp filer source Supported DX-NAS server platforms Microsoft Windows Linux Solaris Microsoft Windows Linux Migration mechanisms Filemover API Must be configured on the Celerra Control Station NetApp APIs Configured automatically bydx-nas Recall mechanism Number of DX-NAS servers supported Readback methods Readback methods on page 33 provides more information about readback methods. Orphan file creation Orphan file management on page 50 provides more information about orphan files. Celerra recalls files from secondary storage. DX-NAS is not in recall path unless migrating direct to an EMC Centera. Multiple DX-NAS servers can connect to one Celerra. Multiple Celerras can be serviced by one DX-NAS server. Full Partial Passthrough Delete source stub Access a file that was migrated with the Full readback method Modify a migrated file DX-NAS moves file back from secondary storage. DX-NAS is always in the data path. Only one DX-NAS server can connect to a NetApp filer Multiple NetApp filers can be serviced by one DX-NAS server Full Delete source stub Secondary storage Secondary storage is a category of storage such as disk or tape, that is supplemental to, less expensive, and usually slower than primary storage. In a DX-NAS environment, secondary storage can be a file system or an EMC Centera. The DX-NAS server uses secondary storage as the target of a migration process. The DX-NAS server installed on Microsoft Windows accesses the file system being used as secondary storage by using UNC paths. On either Solaris or Red Hat Linux you must first mount the file system on secondary storage before DX-NAS can access the file system. DX-NAS must have full read and write privileges on the drives or mount points that are being used as secondary storage. When migrating from primary storage to an EMC Centera device: Configure the EMC Centera device to use single-instance storage. 22 EMC DiskXtender for NAS Release 3.1 Theory of Operations

23 EMC DiskXtender for NAS Overview Note: Single-instance storage ensures that if a file has not changed and is migrated more than once, only one copy of the file is stored on the EMC Centera device. By default, single-instance storage is enabled on EMC Centera regardless of the EMC Centera model (Basic, Governance, or Compliance). Use retention classes rather than retention periods. Where possible, use virtual pools. Chapter 10, Configuration Options for DX-NAS, provides more information about secondary storage configurations. Differences between DX-NAS versions The UNIX/Linux and Microsoft Windows versions for DX-NAS are created from the same source code, and the differences between the two products exist because of operating system differences or file system differences. Table 5 on page 23 identifies the areas where the UNIX/Linux and Microsoft Windows versions of DX-NAS differ. Table 5 Differences between DX-NAS versions (page 1 of 2) Area of difference Microsoft Windows UNIX/Linux NetApp filer support Configure NetApp filers as source hosts Configure NetApp filers as source hosts on Linux NetApp filers not supported as source hosts on Solaris Default installation location Default location of uninstall program Default GUI location Default CLI location DX-NAS server and database startup Source and destination paths C:\Program Files\EMC\ DXNAS C:\Program Files\EMC\DXNAS\ _uninst\uninstallwin32.exe Add/Remove Program from the Control Panel Start > All Programs > EMC DX NAS > DX NAS Client GUI or C:\Program Files\EMC\DXNAS\ Client\runClientGUI.exe Start > All Programs > EMC DX NAS > DX NAS Client CLI or C:\Program Files\EMC\DXNAS\ Client\runClientCLI.exe From the Services dialog box, right-click EMC DX NAS Server and click Start. Note: Starting the server also starts the database. From the command line: net start EMC DXNAS Server UNC paths /opt/emc/dxnas/ /opt/emc/dxnas/_uninstall/uninstaller.bin /opt/emc/dxnas/client/bin/ runclientgui.bin /opt/emc/dxnas/client/bin/ runclientcli.bin From the UNIX/Linux command line: /opt/emc/dxnas/server/bin/dxnasdb start /opt/emc/dxnas/server/bin/dxnasserv er start Mount points Differences between DX-NAS versions 23

24 EMC DiskXtender for NAS Overview Table 5 Differences between DX-NAS versions (page 2 of 2) Area of difference Microsoft Windows UNIX/Linux Logs C:\Program Files\EMC\DXNAS\ Server\logs\DXNASServerLog_x.log C:\Program Files\EMC\DXNAS\ Server\logs\DXNASLog.log opt/emc/dxnas/server/logs/ DXNASServerLog_x.log opt/emc/dxnas/server/logs /DXNASlog.log File and path length limitations The Celerra implementation of CIFS limits path and filename length to approximately 256 characters The path and filename length limit on UNIX/Linux is 2000 characters. Note: However, if you are using a NetApp filer source, the path limit is 495 characters. Offline file indicators Microsoft Windows Explorer displays an offline icon and an offline attribute on files that have been migrated. None 24 EMC DiskXtender for NAS Release 3.1 Theory of Operations

25 2 Configuration Planning and Performance Tuning This chapter discusses the following topics: Primary storage considerations DX-NAS server local disk space Sizing secondary storage Deploying multiple DX-NAS servers Configuration Planning and Performance Tuning 25

26 Configuration Planning and Performance Tuning Primary storage considerations There are a number of ways to organize source files on primary storage. To plan for DX-NAS migrations, consider the source file size and the frequency the source file changes. Effects of source file size on DX-NAS If the files on primary storage are predominantly small (less than 1 MB) migration speeds (measured in bytes per second) will be significantly slower and management overhead on secondary storage (measure in bytes used) will be significantly higher. Conversely, with large source files, migration throughput will be higher and the secondary storage overhead will use a smaller percentage of the space. Predominant size of files Effect on DX-NAS Small Lower B/s migration throughput. Higher percentage of space used for secondary storage management overhead. Possible inode shortage on secondary storage when using NFS. Large Higher MB/s migration throughput. Negligible percentage of space used for secondary storage management overhead. Effects of volatile source files on DX-NAS When planning DX-NAS migrations, consider how often the files to be migrated change. Static files usually are migrated once and use a fairly predictable amount of secondary storage. Note: On a Celerra system, this is accomplished by creating policies that use the passthrough readback method or by using the checksum for uniqueness option. Source files that change frequently, require more secondary storage because DX-NAS keeps copies of previously migrated files. Consequently, with dynamic source file systems, to ensure that migrated files and stub files are synchronized, perform orphan file scans of secondary storage on a regular basis. Volatility Effect on DX-NAS Files change frequently More files are migrated because each changed file must be remigrated. More space is consumed on secondary storage because, by default, each version of the file is saved. Regular orphan file scans of secondary storage are required. Files change infrequently Files migrate only once. Secondary storage usage more predictable. Effect of the full readback method If files are migrated by using the full readback method, each time a file is accessed the entire file is retrieved from secondary storage. The recalled file then replaces the stub file on the Celerra system. This type of a recall operation results in the following actions: Creates an orphan file on secondary storage. Uses additional storage on the Celerra system. 26 EMC DiskXtender for NAS Release 3.1 Theory of Operations

27 Configuration Planning and Performance Tuning Requires the recalled file to be remigrated to secondary storage (if the policy did not use the checksum for uniqueness option). Depending on the configuration, the full readback method can lead to problems. Consider the following scenario: 1. A large number of files are migrated to secondary storage by using the full readback method. Because the migration leaves just a small stub file on the Celerra system, additional disk space is available. 2. More source files are added to the Celerra system, and then migrated to secondary storage. 3. More source files are added to the Celerra system. This scenario can result in exhausting the space on primary storage when a large number of files are recalled from secondary storage. Note: If attempts to recall data produce an error related to insufficient space or quotas, the Celerra will attempt to reread the data by using the passthrough mode. Primary storage sizing formula When determining the size of primary storage in gigabytes, use the following general formula: ( A 8192) + ( B ( C + 10) ) 1024 where: A is the total number of files to be created or stored on primary storage. B is the average amount of new data in bytes that will likely be created on primary storage daily. C is the last access age in days at which point DX-NAS will migrate files to secondary storage. DX-NAS server local disk space The disk space requirements for DX-NAS vary depending on the components installed and on the number of files you plan to migrate. The minimum requirements for the disk where the DX-NAS server is installed requires the following amount of disk space: 5 GB for the DX-NAS installation. 1.5 additional gigabytes for every one million files to be migrated from a NetApp filer or migrated directly to EMC Centera. DX-NAS server local disk space 27

28 Configuration Planning and Performance Tuning Sizing secondary storage When determining the size of secondary storage there are four main considerations: Size of source files Volatility of source files Orphan file management Data deduplication Effect of source file size on secondary storage When DX-NAS migrates files to secondary storage, in addition to the space used by the migrated file, there is additional space and inodes (if using NFS) that are consumed as file management overhead. If you have large files (greater than 10 MB) this overhead is negligible. However, with small files this overhead can become a significant percentage of the storage that is used. Effect of volatile source on secondary storage If a file that has been migrated off primary storage is updated or changed by a NAS client, the file stub that was created when the file was migrated is deleted and a complete file is written in its place. This creates an orphan file on secondary storage, and, if the new file is remigrated, the orphan file is preserved. Consequently, if the files on primary storage are being changed or updated frequently, it can result in multiple versions of migrated files being stored on secondary storage. Managing secondary storage with orphan file scans To manage orphan files on secondary storage, you can run orphan file scan jobs. Orphan file scan jobs examine secondary storage to determine if orphan files exist. If orphan files exist, the orphan file job can delete the orphans immediately or at a later time. The delete action is specified by the orphan file policy. Orphan file management on page 50 provides more information about orphan file scan jobs. Data deduplication DX-NAS release 3.1 includes the data deduplication feature for Celerra source devices that migrate files to a CIFS or NFS file system. Data deduplication is not available if EMC Centera is the destination device. (EMC Centera provides a native data deduplication feature.) Data deduplication does not support multiple target devices. Data deduplication does not apply to files that were migrated with releases of DX-NAS earlier than 3.1, or before the feature was enabled. Data deduplication, when enabled, keeps only a single copy of unique files on secondary storage by using a hashing algorithm that computes a hash value for every file that is migrated. Files that generate a unique hash value are retained on the back-end file system. Files that generate a non-unique hash value are considered duplicates, and are not copied to the back-end system. Data deduplication requires a separate licence. You can enable the data deduplication option from both the DX-NAS GUI and CLI. 28 EMC DiskXtender for NAS Release 3.1 Theory of Operations

29 Configuration Planning and Performance Tuning Secondary storage considerations Table 6 When determining the size of secondary storage, consider the type of secondary storage to be used. Table 6 on page 29 shows how much space to provide on secondary storage. Sizing secondary storage Source system Secondary storage Primary to secondary storage ratio Celerra File system 1:1.5 For every megabyte of storage you want to migrate from the Celerra provide 1.5 times that amount of space on secondary storage. NetApp filer File system 1:1 Celerra or NetApp filer EMC Centera 1:1 Deploying multiple DX-NAS servers To migrate a large number of files, consider using two or more DX-NAS servers. To use multiple DX-NAS servers, install each server on a separate machine, and configure each DX-NAS server to migrate non-overlapping sets of files. When determining the effectiveness of deploying additional DX-NAS servers, the primary consideration is network bandwidth. If running migration jobs on a single DX-NAS server overloads the network, you will not gain any performance by adding another DX-NAS server. If one DX-NAS server does not overload the network, then you can improve total migration performance by adding additional servers. The amount of network traffic generated by the DX-NAS server, as well as the effect of this traffic on the network, varies from site to site. However, estimates of DX-NAS load indicate that two DX-NAS servers can overload a typical Gigabit Ethernet network. Note: If a NetApp filer is used as a source for DX-NAS, only one DX-NAS server should register with the NetApp filer. Although a single DX-NAS server can register with more than one NetApp filer, only one DX-NAS server can access a NetApp filer at any given time. Deploying multiple DX-NAS servers 29

30 Configuration Planning and Performance Tuning 30 EMC DiskXtender for NAS Release 3.1 Theory of Operations

31 3 DX-NAS Administration This chapter discusses the following topics: Jobs Reports Alerts and log files DX-NAS Administration 31

32 DX-NAS Administration Jobs DX-NAS uses jobs to migrate files and manage orphan files. You can create and manage jobs by using either the DX-NAS GUI or CLI. From the GUI, the job structure is implied by the layout of the tabs. A job comprises the following elements: Source The folder or directory on the Celerra system or NetApp filer where the policy engine looks for files that are candidates for migration. On Microsoft Windows, a source is specified as a UNC path, for example: \\server2\wip On Solaris or Linux, a source is specified as a directory or mount point, for example: /Celerra/server2/wip Destination The folder or directory that will hold the migrated files. The DX-NAS server must be able to access the destination in one of the following ways: As a local directory As a UNC path (for CIFS shares) As a mounted NFS share (for Solaris or Linux) Policy A description of what files to migrate and how to migrate them. When a policy is created, match criteria is established to identify which files are candidates for migration. This match criteria can be based on a variety of file attributes like filename, file size, last modification time, and so forth. In Figure 5 on page 32, the first three tabs correspond to the elements of a job: a host and source, a destination, and a policy. To create a job, therefore, you first create a source, a destination, and a policy. Figure 5 Sources and destinations in the DX-NAS GUI 32 EMC DiskXtender for NAS Release 3.1 Theory of Operations

33 DX-NAS Administration The fourth step the step of actually creating a job on the Jobs tab of the GUI consists of selecting a policy for the job. The DX-NAS online help provides more information on using the GUI to configure DX-NAS jobs. You can also create jobs from the CLI by using the host, source, dest, policy and job commands.the EMC DiskXtender for NAS Release 3.1 Multiplatform Version Administration Guide provides more information on the usage and syntax of CLI commands. Policies After creating the source and destination components, you must create a DX-NAS policy. A policy is used by the job that you create. There are three types of policies: Migration policies identify files to be migrated from primary storage to secondary storage. Orphan file policies identify and delete migrated files on secondary storage that no longer have a valid corresponding stub file on primary storage. Source scan policies identify stubs on primary storage that no longer have valid corresponding files on secondary storage, and ensure consistency between file names and locations on secondary storage with the stub file counterpart on primary storage. Each type of policy is created by using the Policy Wizard. The Policy Wizard shown in Figure 6 on page 33 is launched from the Policies tab in the DX-NAS GUI. Figure 6 Readback methods Policy Wizard combines DX-NAS elements and search behaviors For a migration policy, you must specify a readback method to be used when files are recalled from secondary storage. The readback method is an option in the Policy Wizard for migration policies only. The readback method option does not apply to orphan file or source scan policies. When a file is recalled from secondary storage, the Jobs 33

34 DX-NAS Administration recall process employs a readback method. DX-NAS supports three readback methods, which are defined as follows: Full The file on secondary storage is recalled, and it replaces the stub on the source. The full readback method can be used for both Celerra and NetApp filer sources. On the Celerra system, if attempts to recall data produce an error related to insufficient space or quotas, the Celerra system attempts to reread the data by using the passthrough mode. Full is the only readback method supported by a NetApp filer source. Note: Except when using a NetApp source, do not use the full readback method. However, if you do use the full readback method with Celerra, then also use the checksum feature. Partial The Celerra system reads the file on secondary storage and restores only the parts of the file that are being accessed by the requesting client. The partial readback method can be used only by a Celerra source. There may be performance advantages to using the partial readback method if using EMC Centera as secondary storage. The disadvantage of using the partial readback method is that the file is still considered offline unless the entire file is read. Passthrough The Celerra system reads the file on secondary storage and passes the data to the requesting client without affecting the stub. The passthrough readback method is can be used only by a Celerra source. Note: On the Celerra system, the read policy override parameter for the FileMover connection takes precedence over the value set in the policy. Match criteria A migration job determines what files to migrate by using the match criteria specified by the migration policy. The Policy Wizard provides a list of attributes that DX-NAS uses to search for files. Table 7 on page 34 provides a description of these attributes. Table 7 Attributes used as to build match criteria (page 1 of 2) Attribute name Description Search for files of a specific name or extension. Note: On Windows, filenames are not case-sensitive, but on Solaris or Red Hat Linux, filenames are case-sensitive. When using the UNIX or Linux version of DX-NAS, the case of the filename must be considered when scanning files that are dual mounted as both CIFS and NFS size user Search for files that are at or above a specified size. (If using KB, MB, or GB, note that these values are 1024, , and , respectively. Search for files that belong to a specified user. Note: The user attribute is not supported on CIFS file systems. group Search for files that belong to a specified group. Note: The group attribute is not supported on CIFS file systems. accessed date (atime) Search for files based on when they were last accessed. 34 EMC DiskXtender for NAS Release 3.1 Theory of Operations

35 DX-NAS Administration Table 7 Attributes used as to build match criteria (page 2 of 2) Attribute change time or create time (ctime) modified date (mtime) file permissions (perm) Description On NFS file systems, you can search for files based on when their attributes changed. On CIFS file systems, you can search for files based on when they were created. Search for files based on when they were modified. Search for file based on their file permissions. Note: The perm attribute is not supported pn CIFS file systems. In addition to the file attribute list, the Policy Wizard enables you to specify a match string. The match string is specified as a command in the Match String text box, for example: "-name *.mpeg" The -name command in this example finds all files with an mpeg extension. The EMC DiskXtender for NAS Administration Guide provides more information about match criteria and a set of sample commands that can be used for the Match String text box. When a DX-NAS migration job runs, it begins its search for files to migrate in the source directory and searches down through each subdirectory. Every file that matches the match criteria is processed according to the requirements of the migration policy. Previewing a job Before running a migration or orphan file job, you can preview the list of files that will be processed by the job. The preview feature is available from both the GUI and the CLI. In the GUI, the Jobs tab includes the Preview button. The preview feature provides the following two options: Validate job configuration only Checks the source and destination specified by the job to determine their validity. Display candidate files Scans the source specified by the job and then displays information about the files that meet the match criteria specified by the job. The GUI displays the summary results of the file preview on the Summary tab in the Preview Job window. The list of files that will be processed by the job appears on the Jobs 35

36 DX-NAS Administration File List tab. Figure 7 on page 36 show information that appears on the File List tab for a migration job. Figure 7 File list for a migration job preview Besides the Summary and File List tabs, the Preview Job window also includes a set tabs that correspond to file attributes: Type, Size, Access Date, Modify Date, and Create Date. Each of these tabs provide distribution data for the files to be migrated. A preview of an orphan file job also displays summary results on the Summary tab in Preview Job window. Unlike the Preview Job window for a migration job, the Preview Job window for an orphan file contains only the Summary tab and the File List tab. Figure 8 on page 36 illustrates information on the File List tab for an orphan file job. Figure 8 Orphan file job preview listing Note: Source scan jobs cannot be previewed. Clicking Preview on the Jobs tab for a source scan job returns an error message. Previewing a job enables you to fine tune the match criteria before running the actual job. 36 EMC DiskXtender for NAS Release 3.1 Theory of Operations

37 DX-NAS Administration Running a job After ensuring match criteria meets the expectations for a migration or an orphan file job, the next step is to run the job. To run a job from the GUI, select the job and click Run. Once a job is running, it appears in the Jobs in Progress folder in the Tree pane. Once a job is finished, it is removed from the Jobs in Progress folder. Running a job can be started from either the GUI or the CLI. More information about running a job is available from the EMC DiskXtender for NAS Release 3.1 Multiplatform Version Administration Guide (for the GUI and CLI) and online help (for just the GUI). Monitoring a job DX-NAS enables you to monitor the progress of jobs that are currently running. This feature is available from the Jobs tab in the GUI. (The the DX-NAS CLI does not provide a way to monitor jobs.) A list of jobs that are running appears under the Jobs in progress folder in the Tree pane. By clicking a jobs in the Jobs in progress folder, you can see details about that job in the Details pane. Figure 9 on page 37 illustrates information for a migration job that is currently running. Figure 9 Details pane for a migration job in progress The information provided for a migration job includes: Active Job Threads Current number of job threads running for the migration job. Active Migration Rate Current migration rate in KB per second. Byes Migrated Current amount of bytes migrated. Files Migrated Current number of files migrated. Last File Migrated The last file migrated by the job. Number of errors Current number of errors encountered. Start Time Time the migration job started. Jobs 37

38 DX-NAS Administration Figure 10 on page 38 illustrates information for an orphan file job that is currently running. Figure 10 Details pane for an orphan file job in progress Information about an orphan file job includes: Active Job Threads Current number of job threads running for the orphan file job. Files Scanned Current number of files scanned on the destination. Number of Errors Current number of errors encountered. Orphan Bytes Deleted Current number of bytes deleted for orphan files. Orphan Bytes Detected Current number of bytes detected for orphan files. Orphan Files Deleted Current number of orphan files deleted. Orphan Files Detected Current number of orphan files detected. Start Time Time the orphan file job started. 38 EMC DiskXtender for NAS Release 3.1 Theory of Operations

39 DX-NAS Administration Figure 11 on page 39 illustrates information for a source scan job that is currently running. Figure 11 Source scan job in progress information Information about a source scan job includes: Active Job Threads Current number of job threads running for the source scan job. Files Scanned Current number of files scanned on the source. Number of Errors Current number of errors encountered. Start Time Time the source scan job started. Stopping a job You can stop a job only by using the DX-NAS GUI. The CLI does not provide a command to stop a job. To stop a job that is running, select the job from the Tree pane, and click Stop. A confirmation dialog box appears. The job continues to run until you click Yes in the configuration dialog box. Reports The DX-NAS GUI provides statistics for DX-NAS processes in a report format. Five types of reports are available from the Reports tab: Migration Orphan file Primary storage Secondary storage Data deduplication (applies to only a Celerra source) Each report appears by default in a bar chart format. Reports can also be viewed in a table format and then exported to comma-separated value format (CSV) files on the local file system. More information on the formatting of reports is available in online help. Reports 39

40 DX-NAS Administration Migration report The migration report provides data about the number of files and the amount of storage that has been migrated. By default, the report appears as a bar chart and shows migration activity for the past seven days as shown in Figure 12 on page 40. Figure 12 Example of a migration report Orphan file report Whenever an orphan file job deletes files, the data is collected for an orphan file report. This report contains the number of orphan files that have been deleted from secondary storage and how much space these files used. 40 EMC DiskXtender for NAS Release 3.1 Theory of Operations

41 DX-NAS Administration By default the report appears as a bar chart and shows deletion activity for the past seven days as shown in Figure 13 on page 41. Figure 13 An example of an orphan file report Primary storage report Table 8 The primary storage report displays information on the amount of storage used and the amount of storage available on each DX-NAS source. Table 8 on page 41 provides a description for each column for the primary storage report. Primary storage report Column Source Source Path Total Bytes Bytes free Percent (%) used Description DX-NAS source. This is the name the DX-NAS source object was given when it was created. For Microsoft Windows, a UNC path that displays the hostname and file system where the source files are located. For Solaris and Linux, the path (on the machine where the DX-NAS server is running) where the source is mounted. Total number of bytes available on the source file system. Number bytes the Celerra is reporting as being free on the file system. Percentage of total available bytes that are in use on the source file system. Reports 41

42 DX-NAS Administration Secondary storage report The secondary storage report displays information on the amount of storage used and the amount of storage available on each DX-NAS destination. Table 9 on page 42 provides a description for each column for the secondary storage report. Table 9 Secondary storage report Column Destination Destination Path Total Bytes Bytes free Percent (%) used Description DX-NAS destination. This is the name the DX-NAS destination object was given when it was created. For Microsoft Windows, a UNC path that displays the hostname and file system where the destination files are located. For Solaris and Linux, the path (on the machine where the DX-NAS server is running) where the destination is mounted. Total number of bytes available on the secondary storage host. Number bytes the secondary storage host is reporting as being free on the volume where the secondary storage directory resides. Percentage of total available bytes that are in use on the volume where the secondary storage directory resides. Data deduplication report Table 10 The data deduplication report provides details about the number of duplicate instances that have been removed from primary storage. The data deduplication report appears as a table with five columns. Table 10 on page 42 provides a description of each column. Data deduplication report Column Source Host Storage Capacity Reclaimed Instances Eliminated Duplicate Storage % Duplicate File % Description Celerra system enabled for data deduplication. Amount of primary storage that has been freed by migrating files with the Data Deduplication option. Number of duplicate file instances that have been set offline by migrating files with the Data Deduplication option. The files have not actually been eliminated, rather they have been set offline and a stub has been left on primary storage. Percentage of migrated storage that is single-instanced data. Percentage of migrated files that were duplicate files. Printing reports The print function in the DX-NAS GUI prints only what is visible on the screen. Even if there are scroll bars on the screen indicating that there is additional information not shown, only the visible portion is printed. If a migration report is large (that is, the software has been running for many days), it may be more efficient to export the report as a CSV file and then use a spreadsheet or a word processing program to print the file. 42 EMC DiskXtender for NAS Release 3.1 Theory of Operations

43 DX-NAS Administration Alerts and log files DX-NAS provides information to users two ways: alerts and logs. Alerts All alerts are related to job activity. If you have no jobs or schedules configured, you will not have any alerts. Messages about the status of jobs are displayed on the Alerts tab in the DX-NAS GUI. There are three types of alerts: Information alerts appear when a job starts, completes, or stops. Warning alerts appear when minor problems occur. For example, a source scan job will report orphan stubs (that have the data file missing on secondary storage) as warnings. Critical alerts appear when a migration fails or when a scheduled job cannot be run. Alerts are only reported in the DX-NAS GUI. Figure 14 on page 43 shows the Alerts tab. Figure 14 Alerts tab Printing alerts The print function only prints what is visible on the screen. Consequently, if you have a large number of alerts to print, it may be more efficient to export the alerts as a CSV file and then use a spreadsheet or a word processing program to print the file. Alerts and log files 43

44 DX-NAS Administration Log files Setting the server log level DX-NAS uses the following log files: DXNASLog.log contains all messages or errors that are written to the default output stream. This log is beneficial for troubleshooting problems. DXNASServerLog_x.log contains all messages or errors generated by the DX-NAS server process. The value x is a number that is incremented each time a new log is started. The amount of information that is written to the log files can be set by using the DX-NAS GUI. The following list of log levels are ordered from the least verbose to the most verbose: Emergency Reports information only when a process becomes unusable. Alert Reports information when action is required and where lack of action could result in the process becoming unusable. Critical Reports conditions that prevent the proper operation of a process. Error Reports conditions which cause an operation to fail. Warning Reports conditions that warn of impending operation failures. Notice Reports normal, but significant, conditions. Informational Reports informational messages only. Debug Reports most server activity. A selection of one of the more verbose log levels can cause the log files to grow in size very quickly. A large log file could result in system performance problems. Use Debug, Informational and Notice log levels only long enough to complete troubleshooting or testing. Afterwards, copy the log files to a different location, and reset the logs by starting and stopping the DX-NAS console server. The log files are located, by default, in the following directories: For Microsoft Windows, the server logs are located in the C:\Program Files\EMC\DXNAS\Server\logs directory. For Solaris or Linux, the server logs are located in the /opt/emc/dxnas/server/logs directory. Instructions for resetting log files are available in the EMC DiskXtender for NAS Release 3.1 Multiplatform Version Administration Guide. 44 EMC DiskXtender for NAS Release 3.1 Theory of Operations

45 4 File Migration This chapter discusses the following topics: Migrating files Recalling files Orphan file management File Migration 45

46 File Migration Migrating files The primary objective of DX-NAS is to move inactive or infrequently accessed files from primary storage to secondary storage. DX-NAS uses migration policies and migration jobs to move these files. File migration is accomplished by creating policies and jobs by using either the DX-NAS GUI or CLI. A migration job scans the primary file system and identifies files that match specific criteria and then copies those files to secondary storage. Chapter 3, DX-NAS Administration, provides more information about jobs and policies. Per-file information When a file is migrated from primary storage to secondary storage, DX-NAS enters metadata about the migration into the DX-NAS database. The term per-file information refers to this metadata. Per-file information applies to both the stub file on the source and the data file on the destination. The DX-NAS database contains three tables for the per-file information. The following list includes some types of per-file information stored in these tables: Metadata for the stub file including attributes such as access time and modified time. Metadata for the migrated data file on secondary storage. Source system for the stub file. Destination system for the data file. The stub file also contains information about the migration, however, the database contains more information. DX-NAS uses the per-file information whenever an orphan file preview or orphan file job runs. A source scan job uses per-file information in the database keeps the database up-to-date with the current status of the source and destination systems. 46 EMC DiskXtender for NAS Release 3.1 Theory of Operations

47 File Migration Figure 15 on page 47 illustrates how migrating, recalling, and remigrating a file updates the per-file information in the DX-NAS database. Primary storage 1. Migrate Secondary storage 2. Update per-file information DX-NAS database records.txt (stub offline) 3. Recall Data file records.txt (online) 4. Remigrate Orphaned data file 5. Update per-file information records.txt offline Data file Orphaned data file Figure 15 Updates to per-file information in DX-NAS database for migration activity Directory structure on the secondary storage DX-NAS creates a directory structure on the back-end file system the first time files are migrated. The directory structure comprises three levels and starts one level below the mount point. The first level of the directory structure and each subsequent level can contain a total of 128 entries (directories). DX-NAS employs a two-digit hexadecimal number starting with 00 and ending with 7f to name the directories. A migration job stores files in the third-level directory. Figure 16 on page 47 illustrates the directory structure after migrating one file from server299.lab.local, a Celerra source. Figure 16 Example of directory structure on secondary storage Migrating files 47

48 File Migration The maximum number third-level directories can be as high as 2,097,152 (128 * 128 * 128). A directory structure that contains 40 million files, therefore, could store an average of 20 files per entry. Directory names created for files processed with data deduplication are generated based on the message digest. Files that generate the same message digest are easily checked for the existence of duplicate files. For files that are not processed with data deduplication, a random algorithm generates the directory structure. The random algorithm distributes files among the all the directories in the directory structure. This distribution method ensures that files are not all written to a few directories. Recalling files When a file is migrated, the file is copied to secondary storage, and a stub file is created on the Celerra or NetApp source. At this point the file is considered offline, and as long as the relationship between the stub file and the data file is maintained or remains valid, the file can be accessed normally. However, if the file is migrated with the full readback method, then when any NAS client accesses the stub file, the Celerra or NetApp source replaces the stub file with the file from secondary storage. This file recall is transparent to the NAS client. It may take longer to open the file, but it is seen and accessed from the same location with the same name. The only difference may be that, in Microsoft Windows, the user may see that the file has an offline icon overlay, offline attribute, or both. Stub files on the Celerra or NetApp source contain the information necessary to recall migrated files even if DX-NAS is uninstalled. However, there are two exceptions: If DX-NAS was used to migrate directly to EMC Centera, you must recall all files from EMC Centera before uninstalling DX-NAS. If you have migrated files from a NetApp source, the DX-NAS server is required to recall files from secondary storage on NetApp. Similarly, in all NetApp configurations, the DX-NAS server retrieves the data portion of migrated files. DX-NAS must be present for NetApp to recall any files that have been migrated. Recalling all files from the Celerra system On the Celerra system, a recall is performed by the Data Mover. When the file was originally migrated, DX-NAS wrote the location of the data file into the stub, and the Data Mover uses this information to retrieve the file. Consequently, even if DX-NAS is not present, the Data Mover on the Celerra can retrieve the data portion of migrated files. If you want to recall all the files from secondary storage that is, unmigrate all your files from secondary storage you can accomplish this by deleting the secondary storage connection string from the Control Station on the Celerra system. The connection string is created on the Control Station when you configure the FileMover protocol. This connection string contains the information about where to retrieve data for migrated stubs. For example, consider the following connection strings: fs_dhsm -connection CIFSFilesystem1 -create -type cifs -admin administrator -secondary '\\server227.lab.local\winstor' -local_server server227.lab.local 48 EMC DiskXtender for NAS Release 3.1 Theory of Operations

49 File Migration fs_dhsm -connection NFSFilesystem1 -create -type nfsv3 -secondary lava2113:/solstor fs_dhsm -connection myfilesystem -create -type http -secondary -httpport cgi y Connections created by using these commands are assigned connection IDs (CID) as shown in Figure 17 on page 49. cid = 2 type = HTTP secondary = state = enabled read policy override = none write policy = full user = options = httpport=8080 cgi=y cid = 3 type = CIFS secondary = \\server227.lab.local\winstor state = enabled read policy override = none write policy = full local_server = server229.lab.local admin = lab.local\wtaylor wins = cid = 4 type = NFSV3 secondary = lava2113:/solstor state = enabled read policy override = none write policy = full options = userootcred=true proto=udp Figure 17 Connection string listing from Celerra You can force a recall of all migrated files from secondary storage by deleting the connection string. For example, to force a recall of all files from the CIFS connection shown in Figure 17 on page 49, you would use the following fs_dhsm -connection command with the -recall_policy option: fs_dhsm -connection CIFSFilesystem1 -delete 8 -recall_policy yes When yes is specified with the -recall_policy option, the Celerra system attempts to recall all the files on secondary storage. Depending on how many files have been migrated, this can take some time. While the recall process is in progress, you can query the status of the file system by using the following command: fs_dhsm -connection CIFSFilesystem1 -info The state of the connection returns screen output similar to that shown in Figure 18 on page 49. cid = 8 type = CIFS secondary = \\server227.lab.local\winstor\ state = recallonly[ Migration: ON_GOING ] read policy override = none write policy = full local_server = server227.lab.local admin = administrator wins = Figure 18 Connection status information while recall is in progress Recalling files 49

50 File Migration The state of the connection in Figure 18 on page 49 is Migration: ON_GOING. When the files have all been recalled, the connection is deleted or the state of the connection changes to recallonly as illustrated in Figure 19 on page 50. cid = 8 type = CIFS secondary = \\server227.lab.local\winstor\ state = recallonly read policy override = none write policy = full local_server = server227.lab.local admin = administrator wins = Figure 19 Connection status after files have been recalled You can delete recallonly policies by deleting the connection ID again by specifying no with the -recall_policy option. For example, the following command permanently deletes the secondary storage connection ID. fs_dhsm -connection CIFSFilesystem1 -delete 8 -recall_policy no When deleting an EMC Centera connection string, the Celerra system must first associate a user with the CID before using the -recall_policy yes options to the delete the connection string. For example, in Figure 17 on page 49, you would need to change CID 2 to user administrator by using the following command: fs_dhsm -connection myfilesystem -modify 2 -user administrator Supply the administrator password, which is usually the password used to log in to the Control Station. The Celerra will update the connection string. Then you can delete the connection string by using -recall_policy yes as described above. Recalling files from EMC Centera In the case of a Celerra recalling files directly from EMC Centera, the Celerra Data Mover contacts the DX-NAS server, and the DX-NAS server retrieves the requested data from EMC Centera and passes it back to the Celerra. Note: DX-NAS must be present for a Celerra to recall files that have been migrated directly to EMC Centera. Recalling all files from a NetApp filer Unlike Celerra, the NetApp filer does not have a built-in mechanism for recalling all migrated files. To recall all files from a NetApp filer, you must write a script that opens and reads at least one byte from each file in the directory and subdirectories where you have migrated files. The full readback method is the only readback method supported by the NetApp filer, therefore, the use of this script will recall all migrated files. Orphan file management Orphan files are files on the back-end destination that no longer have valid stub files on the source system. For example, if a stub file is deleted from the source system (either a Celerra system or NetApp filer), the relationship between the stub file and the data file on secondary storage is severed. A broken relationship between a stub file and its corresponding data file creates an orphan file on the back-end destination. The creation of orphan files on the destination is a normal occurrence in a DX-NAS environment. An orphan file is created whenever you rename, delete, modify, move, 50 EMC DiskXtender for NAS Release 3.1 Theory of Operations

51 File Migration or recall a file on primary storage. For example, if you migrate a file to secondary storage, recall the file, and then modify file, the next migration job that runs creates an orphan on secondary storage. At this point, two versions of the file exist on the back end. Figure 20 on page 51 illustrates how an orphan file is created after modifying a file. Primary storage 1. Migrate Secondary storage 5. Orphan file records.txt (offline) 2. Recall V2.MD5.4beec8da2477bc5e866bece9eacee records.txt (online) 3. Modify 4. Remigrate records.txt (offline) V2.MD5.59eeec0da3688bd3e977bece4ceaae Figure 20 Orphan file creation after modifying a file Note: By design, the filename on the back-end destination is not the same as the filename on the source. Directory structure on the secondary storage on page 47 provides more information on the naming convention used on secondary storage. Moving a file on primary storage also creates an orphan file, which exists only until you run a source scan. The source scan rediscovers where the file currently resides on secondary storage. A source scan manages orphans on the source system. Files that no longer have corresponding stub files are candidates to consider for deletion. To manage orphan files, DX-NAS enables you to run source and orphan file (destination) scan jobs. Both jobs are created by using the GUI or from the command line. Both types of jobs specify a policy. Orphan file policies identify and delete migrated files on secondary storage that no longer have a valid corresponding stub file on primary storage. Source scan policies identify stub files on primary storage that no longer have valid corresponding files on secondary storage, and ensure consistency between filenames and locations on secondary storage with the stub file counterpart on primary storage. File recall The type of readback method for a file recall produces different outcomes. When Celerra recalls a file that was migrated with the full readback method, the stub file is overwritten with the contents from secondary storage. At this point the file on secondary storage becomes an orphan file. When NetApp recalls a file, the stub file is overwritten with the contents from secondary storage. The file on secondary storage, however, does not become an orphan file. Orphan file management 51

52 File Migration Source scan If you move a file (stub file) on the source system, you must run a source scan to update the database with the correct location of the stub file. A source scan job performs the following tasks: Searches the source file system. Ensures the DX-NAS database is consistent with the files in the file system. Keeps the per-file information in the database up-to-date. DX-NAS reports a warning alert if a source scan job runs and discovers that a stub file exists, but the back-end data file is missing. The stub file contains a pointer to the data file on secondary storage. A source scan job checks for the existence of the data file on the back end by using the pointer from the stub file. If the data file is missing, the source scan generates a warning alert. The stub file then becomes an orphan file. Figure 21 on page 52 illustrates how a stub file becomes an orphan file after the data file on secondary storage is deleted. Primary storage 1. Migrate Secondary storage Stub file (offline) Data file Stub file (orphan) 2. Delete data file X Data file Figure 21 Stub file orphan Orphan file scan The purpose of an orphan file scan is to remove files on secondary storage that are no longer needed. The removal of unneeded files also frees space on secondary storage. An orphan file scan checks the files on the back-end system (destination) to determine if any orphan files exist. A file becomes an orphan file on secondary storage when its source stub file is deleted, modified, moved, or renamed. If orphan files exist, the orphan file scan job takes one of the following actions: Deletes the orphan files immediately. Deletes the orphan files at a later time. Specify the delete action when you configure the orphan file scan policy. Usually you will not delete orphan files immediately, but will retain them for a period of time that meets your data retention policy. Orphan file jobs that delete orphans at a later time, only delete the files the next time the orphan file job runs. In most cases, a source scan should be run before an orphan file scan. When stub file is deleted, the orphan file scan job updates the per-file information in the database. If on a subsequent orphan file scan the stub file is located (because the stub file is restored or network access to a source is reestablished) the per-file information in the database is updated to reflect the new status. The status of the data file on secondary storage is updated to reflect that the file is no longer an orphan file. 52 EMC DiskXtender for NAS Release 3.1 Theory of Operations

53 File Migration Orphan file previews The primary purpose of an orphan file preview is to determine the orphan status of data files on secondary storage. An orphan file preview performs that same tasks as an orphan file scan, except for deleting orphan files. The orphan file preview does the following: Updates per-file information if files are found to be orphans. Remove orphan status from the per-file information for files that are no longer orphans. After restoring file stubs, you must run an orphan file preview twice to display the correct number of orphan files. Orphan file management 53

54 File Migration 54 EMC DiskXtender for NAS Release 3.1 Theory of Operations

55 5 Backup and Recovery for DX-NAS This chapter discusses the following topics: Full data backups Backing up stub files Backing up secondary storage Restoring stub files from backup media Backup and Recovery for DX-NAS 55

56 Backup and Recovery for DX-NAS Full data backups When files are migrated from primary storage, they are split into two parts: the stub files, which remain on primary storage, and the data files, which are copied to secondary storage. Backup and recovery strategies must address how both of these parts are backed up. When you perform a full data backup, the backup software can treat stubs as if they were the regular files. Depending on the readback method that was set when the files were migrated, primary storage does one of two things: Pulls the files back from secondary storage. Passes the files data through the stub, leaving the stub intact on primary storage. On a Celerra system, if files are migrated with the full readback method, a backup of the primary storage effectively reverses the migration. The entire file is recalled back to primary storage and an orphan file remains on secondary storage. To avoid this situation, do not use the full readback method to migrate files from the Celerra system. On a Celerra system, if files are migrated with the passthrough readback method, the backup software reads the files data as it is passed through the stub. Figure 22 on page 56 illustrates a pass-through backup. Backup media Celerra Secondary storage Backup Data file Stub file (offline with passthrough readback method) Data file Restore Data file replaces stub file Now-orphaned data file Figure 22 Example of full data backup of migrated files on Celerra During a restore operation, the Celerra system overwrites the stub file on primary storage with the entire file, which creates an orphan file on secondary storage. Backing up the full data of stub files that have been migrated with the passthrough readback method has both advantages and disadvantages. The advantages are as follows: Entire data file resides on the backup media. Migrated files remains offline on primary storage and do not need to be migrated again. 56 EMC DiskXtender for NAS Release 3.1 Theory of Operations

57 Backup and Recovery for DX-NAS The disadvantages are as follows: Longer back up times because the data for each stub file must be retrieved from secondary storage. Increased network traffic because the data for each stub files must be retrieved from secondary storage. Creation of orphan files on secondary storage during a restore operation. Orphan file management with full data backups When a full data file is restored from the backup media to primary storage, the stub file on primary storage is overwritten. The NAS clients can now access the restored file, however, an orphan file on secondary storage is the result of this restore. Figure 22 on page 56 illustrates this action. Because the full data file resides now on both primary storage and backup media, orphan files can be deleted immediately from secondary storage. With full data backups there is no need to keep orphan files. Forcing a full data backup to read stubs in passthrough mode If files on the Celerra source are migrated by using the full readback method, it is still possible to back up the stub files in passthrough mode, therefore eliminating the need to remigrate files. To set the passthrough mode, type the following command on the Celerra Control Station: fs_dhsm -modify filesystem -backup passthrough Where filesystem is the name of the primary storage file system that you want to back up as passthrough. This command overrides the readback mode that was specified when the file was migrated. Backing up stub files When you back up the stubs of migrated files, the backup application must be able to distinguish that the stub files represent offline files. If the backup application determines that a file is offline (a stub file) it can then back up just the 8 KB stub file without recalling the file from secondary storage. However, in most cases the data portions of the stubs on secondary storage should also be backed up. Figure 23 on page 58 illustrates a backup and restore of a stub file on a Celerra primary storage Backing up stub files 57

58 Backup and Recovery for DX-NAS system, as well as a backup and restore of the stub file s corresponding data file on secondary storage. Backup media Celerra Secondary storage Backup media Backup Stub file Stub file Data file Data file Restore Stub file Data file Figure 23 Stub backup with corresponding secondary storage backup A backup of a NetApp filer source system also requires backing up the DX-NAS database. Figure 24 on page 58 illustrates a backup and restore of a stub file on a NetApp filer source, as well as a backup and restore of the stub file s corresponding data file on secondary storage. DX-NAS DB Backup media NetApp Secondary storage Backup media Backup Stub file Stub file Data file Data file Restore Stub file Now-orphaned data file DX-NAS DB Figure 24 DX-NAS database backup If you migrate files from either a NetApp filer or a Celerra system to an EMC Centera, you must back up the DX-NAS database at the same time you back up secondary storage. Note: Do not run any migration jobs while backing up the DX-NAS database and secondary storage. 58 EMC DiskXtender for NAS Release 3.1 Theory of Operations

59 Backup and Recovery for DX-NAS A backup of stub files on primary storage has the following advantages: Shorter backup times because only the 8 KB stub file is backed up (instead of the entire file). Reduced network traffic. Stub backups, however, also have disadvantages: Only the stub resides on the backup media. The data portion of the stub that resides on secondary storage must be backed up separately. If orphan file management policies are in use, backups of both stub files and the associated data on secondary storage must be synchronized. Note: You cannot perform a stub-only backup on primary storage file systems that are only exported as NFS. Stub-only backups are available only if the CIFS protocol is used. Preparing primary storage for stub backups on a Celerra system To back up just the stub files on a Celerra system you must first enable the offline backup feature on the file system. To enable offline backup, log in to the Celerra Control Station and type the following command: fs_dhsm -modify filesystem -backup offline where filesystem is the name of the primary storage file system that you want to back up as passthrough. The offline backup feature enables backup applications, such as EMC NetWorker to back up only the stub without recalling data from secondary storage. Figure 25 on page 59 shows an example of how the stub file takes up very little space on the backup media. Figure 25 EMC NetWorker User Recover window Managing orphan files without backup synchronization As long as files on secondary storage are not deleted, synchronizing the stub backup and the secondary storage backup is not necessary. Likewise, if the stub files on Backing up stub files 59

60 Backup and Recovery for DX-NAS primary storage are relatively static that is, they are not modified or deleted very often the stub files and associated data files on secondary storage require minimal synchronization. If, however, stub files are relatively dynamic that is, they change or are deleted often the associated data file on secondary storage becomes an orphan. To address the orphan files on secondary storage requires an orphan file management policy. Also, backups of both the stub files and data files must be synchronized. To avoid the need for synchronization in this instance, create orphan file policies that delete files only after the source file backup has expired. For example, consider a company that backs up primary storage stub files on a weekly basis and keeps those backups for 18 months before reusing the backup media. This company could create orphan file policies that flag orphan files to be deleted 18 months after they are migrated. In this way, not only can any restored stub find its data portion, but also no orphan files are retained beyond the point where a backed-up stub might need it. Managing orphan files with backup synchronization Deleting a stub file from primary storage changes the status of the data file on secondary storage to an orphan file. If the deleted stub file is never restored, then the orphan file on secondary storage is most likely wasting space. However, if the stub file is restored from a backup at some point, then the data file on secondary storage will no longer be an orphan. When the restored stub file is accessed, the data file on secondary storage will be recalled. Environments in which many orphans are created and deleted, a backup process must be implemented to insure that when a stub file is restored from a backup, the corresponding data file is available either directly from secondary storage or from a secondary storage backup. Consider the following rules in planning a backup process: 1. Always back up secondary storage before running orphan file jobs. 2. Select the appropriate Delete later option when creating orphan file polices. Set the orphan policy to delay deleting orphan files until sometime after the Orphan date (the data on which a file becomes an orphan). Set the policy to delay deleting orphan files for a period of time after the Migrate date (the data on which the file is migrated) that is longer than the total time it takes to fully back up secondary storage and complete the orphan file scan. Orphan file jobs that use the Delete later option only delete files the next time the orphan file job runs. Consequently, if you always back up secondary storage before running an orphan file job, all orphan files will be preserved on backup media before they are deleted. Backing up secondary storage There are no stub files or special files on DX-NAS secondary storage. Secondary storage contains regular files on a file system. You back up secondary storage file systems the same as you would back up any other the file system. 60 EMC DiskXtender for NAS Release 3.1 Theory of Operations

61 Backup and Recovery for DX-NAS Restoring stub files from backup media A typical stub file restore procedure includes the following steps: 1. Restore the stub file from the backup media, noting the date the file was backed up. 2. Use the DX-NAS GUI or CLI to run a source scan on the directories where you restored the stub files. 3. If the newly restored stub file cannot access its correct data portion from secondary storage, query the stub file by using the File Attributes Wizard from the DX-NAS GUI or the fileattrs command from the DX-NAS CLI. 4. Restore the file identified in the offline path by using the backup of secondary storage that immediately follows the date when the stub file was backed up. 5. After restoring file stubs, run an orphan file preview twice to display the correct number of orphan files. Secondary storage files must be restored to their original location, which is identified by the offline path. Restoring files to a different location disables the stub files ability to access its associated data files. Determining what files to restore from backup media When files are deleted from primary storage the administrator identifies these files for a restore operation by using the original filename. Files that are migrated by DX-NAS, however, are stored on secondary storage by using a different filename than the original filename. If for some reason a stub file can no longer retrieve the data portion of the file from secondary storage, identification of the file on secondary storage for a restore may not be readily apparent. Because of the naming convention used by DX-NAS for files stored on secondary storage, you must use the File Attributes Wizard or the fileattrs command to identify the file to be restore. The EMC DiskXtender for NAS Release 3.1 Multiplatform Version Administration Guide provides more information for using the File Attributes Wizard and the fileattrs command. Restoring stub files from backup media 61

62 Backup and Recovery for DX-NAS 62 EMC DiskXtender for NAS Release 3.1 Theory of Operations

63 6 Using Celerra Checkpoints with DX-NAS This chapter discusses the following topics: About checkpoints Source directory checkpoints Secondary storage checkpoints Using Celerra Checkpoints with DX-NAS 63

64 Using Celerra Checkpoints with DX-NAS About checkpoints The SnapSure technology of the Celerra enables you to create point-in-time snapshots or checkpoints of a file system. Checkpoints enable you access data, including any migrated file stubs, from earlier time periods from the Celerra system To illustrate how checkpoints work on the Celerra, consider the following scenario in which a checkpoint is created for a file system. 1. A file system named pfs12 is created on the Celerra system. 2. The pfs12 file system is exported as CIFS and NFS. 3. A directory called WorkingData is created. 4. A set of files are copied to pfs12. From the Windows Explorer, the directory structure appears similar to that shown in Figure 26 on page 64. Figure 26 Celerra file system before checkpoint creation Note: Because pfs12 is exported as NFS, the Celerra system creates the.etc and lost+found directories on the file system. 5. A checkpoint of the file system is created by issuing the fs_ckpt command from the Celerra Control Station. 64 EMC DiskXtender for NAS Release 3.1 Theory of Operations

65 Using Celerra Checkpoints with DX-NAS The fs_ckpt command creates a hidden directory called.ckpt and a directory with.ckpt that is named by using the date and time the checkpoint was created. Below the data and time directory, the Celerra system presents a read-only copy of the pfs12 file system as of the date the snapshot was created. Figure 27 on page 65 illustrates the file system structure after a checkpoint is created. Figure 27 Celerra file system after checkpoint creation Any stub files of migrated files that are present on the pfs12 file system are available in the checkpoint. Additionally, stub files in the checkpoint can be used to recall the data file from secondary storage if the data portion is still available on secondary storage. After the checkpoint has been created, a file stub can be deleted from the file system causing an orphan file to be created, which could be deleted by an orphan file job. If an orphan job deletes the data portion of the file, the checkpoint version of the stub will no longer be able to access it.! IMPORTANT The key point to remember about checkpoints is that all the files under the.ckpt directory are read-only which can affect the behavior of DX-NAS. Source directory checkpoints The use of checkpoints for a file system that contains DX-NAS source directories has no effect on the performance of DX-NAS. Accessing files in a checkpoint directory, however, has limitations. Copying stub files from checkpoint directories You can copy any stub from a checkpoint directory back to the your file system, but be aware that the complete file will be copied. Source directory checkpoints 65

66 Using Celerra Checkpoints with DX-NAS If you want to copy just the stub back to the file system you need a special tool like EMCopy. Restoring files to a checkpoint directory If the full readback method was used for a migration, the Celerra system will return an error when an attempt to store the files to the read-only checkpoint directory is made. When this failure occurs, the Celerra system overrides the full readback method with the passthrough option to retrieve the information. Using a checkpoint directory as a DX-NAS source Checkpoints present a copy of a file system as it appeared at a specific time in the past. If a stub file is deleted from the primary source after the checkpoint is created, the associated data file on secondary storage might be flagged as an orphan by an orphan file job. If, however, the orphan file job does not immediately delete the orphan, you can fix the orphan file by running a source file scan on the checkpoint directory. The source file scan job does the following: Determines the location of the data file for each stub file in the checkpoint directory. Uses the per-file information for the data file to: 1. Update the database with the checkpointed stub as the parent of the data file. 2. Remove the orphan status. Running a source scan job on checkpoint directories ensures that data files referenced by stub files have their orphan status removed and are not inadvertently deleted by orphan file jobs. Secondary storage checkpoints You can migrate files from source directories on the Celerra system to secondary storage that is also on the Celerra system. Secondary storage can be on another Celerra system, another Data Mover, another file system managed by the same Data Mover, or a different directory on the same file system. You can also configure checkpoints on secondary storage that exist on a Celerra system. Use of checkpoints on Celerra secondary storage enables you to copy any data file from a (read-only) checkpoint directory back to the principal (read-write) file system. Note: The.ckpt directory and all its subdirectories are read-only, therefore, do not use any of these directories as a destination. You cannot run an orphan file job on a checkpoint directory. 66 EMC DiskXtender for NAS Release 3.1 Theory of Operations

67 73 DX-NAS Integration with other EMC Products This chapter discusses the following topics: Integrating DX-NAS with EMC VisualSRM Integrating DX-NAS with DiskXtender for Windows Integrating DX-NAS with File System Manager for UNIX/Linux Integrating with DiskXtender UNIX/Linux Storage Manager Integrating DX-NAS with EMC AVALONidm Using EMC Celerra HighRoad DX-NAS Integration with other EMC Products 67

68 DX-NAS Integration with other EMC Products Integrating DX-NAS with EMC VisualSRM EMC VisualSRM is a policies-based storage management application that provides a consolidated view of your company's storage infrastructure, which enables you to manage your storage resources. Policies, which you configure, run across platforms to not only identify problem areas, but also to automatically act on that information. VisualSRM with DX-NAS provides an Information Lifecycle Management (ILM) solution for NAS storage with the added benefit of powerful reporting and analysis capabilities. The VisualSRM agent accesses both the primary and secondary storage from the DX-NAS server. The focus of DX-NAS is to create and run migration jobs, whereas, VisualSRM creates reports on storage usage including historical data usage and applies intelligent actions to manage storage volumes. For example, VisualSRM can monitor disk usage, and, at a pre-defined threshold, start compressing files that have not been accessed for three months. Another intelligent action that VisualSRM can take is to run a script that calls DX-NAS and migrates any files on the Celerra file system that match its filtering criteria. Figure 28 on page 68 shows a VisualSRM filter that has been set to find any PDF files on a Celerra volume named lava2176. Any files that meet this criteria will be sent to DX-NAS to be migrated to secondary storage. Figure 28 VisualSRM filter set to find PDF files 68 EMC DiskXtender for NAS Release 3.1 Theory of Operations

69 DX-NAS Integration with other EMC Products Configuring VisualSRM to work with DX-NAS The first phase of configuring VisualSRM to work with DX-NAS is to install each product separately. The following documentation provides installation instructions: EMC DiskXtender for NAS Release 3.1 Microsoft Windows Installation and Configuration Guide EMC DiskXtender for NAS Release 3.1 UNIX/Linux Installation and Configuration Guide EMC VisualSRM Installation Guide After installing both DX-NAS and VisualSRM, the following high-level steps are required: 1. From the DX-NAS GUI: a. Configure a source for the Celerra file system that will contain files to be migrated. This source directory is same directory that VisualSRM will scan. b. Configure a destination where files are to be migrated. c. Configure a policy that uses the source and destination created in step a and step b, respectively. The policy identifies the migration mode and readback method. Leave the Match Criteria field blank. Match criteria by default is set to -type f. This default setting enables DX-NAS to use the files sent to it by VisualSRM. d. Create a job to migrate the files. Specify the policy created in step c for this job. A VisualSRM batch file will run this job. 2. From the VisualSRM Console: a. Create a resource. This resource can be thought of as a container for disk drives or file systems. b. Add the Celerra file system created in step 1a to the resource. c. Create a scan policy for the resource. d. Create an intelligent action that will run the following script: C:\Program Files\EMC\DXNAS\Server\bin\runVSRMJob.bat Note: The runvsrmjob.bat file is distributed with the DX-NAS software. The runvsrmjob.bat script calls the DX-NAS job you created in step d. Integrating DX-NAS with EMC VisualSRM 69

70 DX-NAS Integration with other EMC Products e. Set the filtering criteria, such as filename and file size, on this intelligent action. Figure 29 on page 70 illustrates the Setting tab in the VisualSRM Console. Figure 29 Settings tab in the VisualSRM Console After you have completed these steps you can run the intelligent action from the VisualSRM Console. Note: Migrating more than 4,000 files at a time can negatively affect the performance of VisualSRM. To determine if migrating more than 4,000 files will affect the system, preview the results of the VisualSRM scan policy before running the intelligent action. Integrating DX-NAS with DiskXtender for Windows To migrate files to a device such as a tape library, the migration must be done indirectly through an intermediary product such as EMC DiskXtender for Windows. Secondary storage in such a configuration is a DiskXtender for Windows extended drive. An extended drive is simply a Microsoft Windows NT file system (NTFS). Files that are migrated to the extended drive are then managed by the DiskXtender for Windows component called the File System Manager. The process of moving files from the source device (either Celerra or a NetApp filer) is as follows: 1. A DX-NAS migration job moves files from the source system to the extended drive on the DiskXtender for Windows system. 2. The DiskXtender for Windows system examines the files on the extended drive by running a background scan process. 70 EMC DiskXtender for NAS Release 3.1 Theory of Operations