Ready-to-use Virtual Appliance for Hands-on Spectrum Archive Evaluation

Transcription

1 Front cover Ready-to-use Virtual Appliance for Hands-on Spectrum Archive Evaluation Takeshi Ishimoto Hiroshi Araki Hiroyuki Miyoshi Redpaper

2

3 IBM REDBOOKS PROMOTIONS IBM Redbooks promotions Find and read thousands of IBM Redbooks publications Search, bookmark, save and organize favorites Get personalized notifications of new content Link to the latest Redbooks blogs and videos Get the latest version of the Redbooks Mobile App ios Download Now Android Promote your business in an IBM Redbooks publication Place a Sponsorship Promotion in an IBM Redbooks publication, featuring your business or solution with a link to your web site. Qualified IBM Business Partners may place a full page promotion in the most popular Redbooks publications. Imagine the power of being seen by users who download millions of Redbooks publications each year! ibm.com/redbooks About Redbooks Business Partner Programs

4 THIS PAGE INTENTIONALLY LEFT BLANK

5 Introduction IBM Spectrum Archive Enterprise Edition for the IBM TS4500, IBM TS3500, and IBM TS3310 tape libraries provides seamless integration of IBM Linear Tape File System (LTFS with IBM Spectrum Scale by creating an LTFS tape tier. You can run any application that is designed for disk files on tape by using Spectrum Archive. Spectrum Archive can play an important role in reducing the cost of storage for data that does not need the access performance of primary disk. For more information about Spectrum Archive, see Service Management Connect - IBM Spectrum Archive - Overview, which is available at this website: This Spectrum Archive Virtual Appliance can be deployed in minutes and you can try key features of Spectrum Archive Enterprise Edition easily along with this user guide. The virtual machine has a pre-configured Spectrum Scale and a virtual tape library that allows to quickly test the Spectrum Archive features without connecting to a physical tape library. The trial VM is provided as a VirtualBox template image and you can create your own instance of the trial VM by importing the.ova file. Copyright IBM Corp All rights reserved. ibm.com/redbooks 1

6 Figure 1 shows an overview of Spectrum Archive Virtual Appliance. Client Applications Multiple Protocol Support Spectrum Archive EE 1.2 Virtual Machine POSIX HDFS NFS SMB Object Spectrum Scale Spectrum Archive Global Namespace Hybrid Storage Flash/Disk LTFS Figure 1 Spectrum Archive Virtual Appliance overview In this IBM Redpaper publication, we show you how to set up a virtual appliance and include typical use cases with instructions so that you can try the use cases yourself. This publication includes the following topics: How to migrate files from disk to tape manually and automatically. How to recall files from tape to disk. How to perform tape management operations, such as reconcile, reclaim, export, and import. Best practices for defining migration policy rules in Spectrum Archive solution. This version of the trial VM includes the following software: IBM Spectrum Archive Enterprise Edition 1.2 IBM Spectrum Scale Ready-to-use Virtual Appliance for Hands-on Spectrum Archive Evaluation

7 Note: Consider the following points: Because this virtual appliance uses the emulated tape library, some of the command output can be slightly different from the physical tape library. The capacity of each virtual tape cartridge is 2.5 GB, which is significantly smaller than the physical tape cartridges. The virtual appliance is not an appropriate environment for performance testing. Spectrum Scale protocol related service (NFS/CIFS/Object is currently disabled by default. Requirements The requirements to run the Spectrum Archive Virtual Appliance are listed in Table 1. 3

8 Table 1 Spectrum Archive Virtual Appliance requirements VM host platform Windows 7 Professional SP1 Mac OS RHEL 7.1 Note: Supports 64-bit only Software Tested with VirtualBox CPU Virtual Appliance.OVA file temporary download space Virtual Appliance.OVA file 1 Core 2.5 GB SpectrumArchive_Trial_VM_GA_1.2.ova Note: Download the Virtual Appliance.OVA file from: source=swerpzsw-spectrial-3 The web page will guide you through registering for an IBM user ID to download the.ova file. Minimum system memory allocated to the VM System disk capacity used by VM 4 GB 7 GB - 40 GB Note: 40 GB is used when all disk space (including IBM GPFS and GB tapes are full. For test cases that are described in this document, 10 GB should be enough. Machine virtualization acceleration setup Must be set to ENABLED IP addresses Note: You can connect the VM from the host via ssh -p 2022 root@localhost. Installation Complete the following steps to install the Spectrum Archive Virtual Appliance: 1. Start VirtualBox. 2. Configure a Host-Only network by clicking Preferences Network Host-only Networks. 3. Ensure that there is at least one host-only network with the default settings that are shown in Figure 2. 4 Ready-to-use Virtual Appliance for Hands-on Spectrum Archive Evaluation

9 Figure 2 Host-only Networks settings If no such network exists, click Add to create a network (as shown in Figure 3 and make note of the adapter name, which is used later. Figure 3 Creating a host-only network 5

10 4. Create your own instance of the trial virtual machine (VM by clicking File Import Appliance, as shown in Figure 4. Figure 4 Importing an appliance 5. Select the SpectrumArchive_Trial_VM_GA_1.2.ova file. Note: During the import process, do not select the Reinitialize the MAC address of all network cards option. 6. In the Network settings of the trial VM after the import process is complete, ensure that adapter 2 and 3 are attached to the adapters that were selected (see Figure 2 on page 5. The.ova file was exported in our example on Mac OS and its adapter names match with Mac OS/Linux version VirtualBox. Because Windows uses a different naming rule for adapter names, Windows users must change the adapter settings that are pre-configured in the.ova file. Complete the following steps to change the adapter names: a. Select the adapter names that are highlighted in Figure 5. 6 Ready-to-use Virtual Appliance for Hands-on Spectrum Archive Evaluation

11 Figure 5 Host-only networks connected to adapters 7

12 b. If changes must be made, click Settings, as shown in Figure 6. Figure 6 Clicking the Settings icon 8 Ready-to-use Virtual Appliance for Hands-on Spectrum Archive Evaluation

13 c. Click the Network tab and select the correct adapter names for adapters 2 and 3, as shown in Figure Figure 7 Correct adapter names Complete the following steps to change the adapter names: 1. Click Network. 2. Click the Adapter 2 tab. 3. Change adapter name, if needed. 4. Select the Adapter 3 tab to change the Adapter 3 name. 5. Click OK. 6. Ensure that Adapter 2 and Adapter 3 names are correct in the main window, as shown in Figure 8 on page 10. 9

14 Figure 8 Correct adapter names are shown Setting up Spectrum Archive Virtual Appliance In this section, we describe how to set up Spectrum Archive Virtual Appliance after the installation is complete. When the virtual machine boots, as described in Spectrum Scale 4.2 Trial VM Quick Start Guide that is included in the Spectrum Scale Trial VM, login by using the following credentials: User: root Password: ibm Spectrum Archive then can be configured by completing the initial setup process that is described next. This set up must be completed whenever the virtual machine starts. Complete the following steps: 1. Ensure that the VM is a member of a Spectrum Scale cluster, as shown in Example 1. Example 1 Ensuring VM is member of a Spectrum Scale [root@spectrumscale ~]# mmlscluster GPFS cluster information ======================== GPFS cluster name: democluster.spectrumscale GPFS cluster id: GPFS UID domain: democluster.spectrumscale Remote shell command: /usr/bin/ssh 10 Ready-to-use Virtual Appliance for Hands-on Spectrum Archive Evaluation

15 Remote file copy command: /usr/bin/scp Repository type: CCR Node Daemon node name IP address Admin node name Designation spectrumscale spectrumscale quorum-manager-perfmon 2. Ensure that the Spectrum Scale service started and /gpfs/fs1 is mounted, as shown in Example 2. It might take approximately 5 minutes for fs1 to be mounted. Example 2 Spectrum Scale service started [root@spectrumscale ~]# mmgetstate Node number Node name GPFS state spectrumscale active [root@spectrumscale ~]# df /gpfs/fs1 Filesystem 1K-blocks Used Available Use% Mounted on /dev/fs % /gpfs/fs1 3. Configure and start Spectrum Archive by issuing the commands that are shown in Example 3. Example 3 Configuring and starting Spectrum Archive [root@spectrumscale ~]# ltfs -o tape_backend=file -o changer_backend=file -o changer_devname=/ltfs_vtape/library /ltfs 6562 LTFS14000I LTFS starting, LTFS version (9601, log level LTFS14058I LTFS Format Specification version LTFS14104I Launched by "ltfs -o tape_backend=file -o changer_backend=file -o changer_devname=/ltfs_vtape/library /ltfs" 6562 LTFS14105I This binary is built for Linux (x86_ LTFS14106I GCC version is (Red Hat LTFS17087I Kernel version: Linux version el7.x86_64 (builder@kbuilder.dev.centos.org (gcc version (Red Hat (GCC #1 SMP Fri Mar 6 11:36:42 UTC 2015 i LTFS17089I Distribution: Red Hat Enterprise Linux Server release 7.1 (Maipo 6562 LTFS17089I Distribution: NAME="CentOS Linux" 6562 LTFS17089I Distribution: Red Hat Enterprise Linux Server release 7.1 (Maipo 6562 LTFS17089I Distribution: Red Hat Enterprise Linux Server release 7.1 (Maipo 6562 LTFS14064I Sync type is "unmount" 6562 LTFS17085I Plugin: Loading "file" driver 6562 LTFS17085I Plugin: Loading "unified" iosched 6562 LTFS17085I Plugin: Loading "file" changer 6562 LTFS17085I Plugin: Loading "ondemand" dcache 6562 LTFS17085I Plugin: Loading "memory" crepos 6562 LTFS11593I LTFS starts with a product license version ( _ LTFS12267I Getting the device directory (/tmp/ltfs LTFS12269I Scanning the devices directory (/ltfs_vtape/library 6562 LTFS11578I Inactivating all drives (Skip to scan devices 6562 LTFS16500I Cartridge repository plugin is initialized (memory, /gpfs/fs1/.ltfsee/meta/0000vlibrary_ll LTFS11545I Rebuilding the cartridge inventory 6562 LTFS11627I Getting Inventory VLIBRARY_LL LTFS11629I Aqcuireing MoveLock VLIBRARY_LL0 11

16 6562 LTFS11630I Aqcuired Move Lock (9-0000VLIBRARY_LL LTFS11628I Got Inventory VLIBRARY_LL LTFS11720I Built the cartridge inventory ( LTFS14708I LTFS admin server version 2 is starting on port LTFS14111I Initial setup completed successfully 6562 LTFS14112I Invoke 'mount' command to check the result of final setup 6562 LTFS14113I Specified mount point is listed if succeeded [root@spectrumscale ~]# ltfsee start Library name: lib1, library id: 0000VLIBRARY_LL0, control node (MMM IP address: GLESM401I(00253: Loaded the global configuration. GLESM402I(00264: Created the Global Resource Manager. GLESM403I(00279: Fetched the node groups from the Global Resource Manager. GLESM404I(00287: Detected the IP address of the MMM ( GLESM405I(00298: Configured the node group (G0. GLESM406I(00307: Created the unassigned list of the library resources. GLESL536I(00074: Started the Spectrum Archive EE service (MMM for library lib1. 4. Verify the status of Spectrum Archive by issuing the commands that are shown in Example 4. Example 4 Verifying status of Spectrum Archive [root@spectrumscale ~]# df /ltfs Filesystem 1K-blocks Used Available Use% Mounted on ltfs:/ltfs_vtape/library % /ltfs [root@spectrumscale ~]# ltfsee status Ctrl Node Status PID Library Active lib1 [root@spectrumscale ~]# ltfsee info libraries Library Name Status Model Serial Number Ctrl Node lib1 Inactive 0000VLIBRARY_LL [root@spectrumscale ~]# ltfsee info drives Drive S/N Status Type Role Library Address Node ID Tape Node Group VDRIVE0000 Not mounted LTO5 mrg lib G0 VDRIVE0001 Not mounted LTO5 mrg lib G0 VDRIVE0002 Not mounted LTO5 mrg lib G0 VDRIVE0003 Not mounted LTO5 mrg lib G0 [root@spectrumscale ~]# ltfsee info tapes Tape ID Status Type Capacity(GiB Free(GiB Unref(GiB Pool Library Address Drive VTAPE9L5 Unavailable LTO lib VTAPE8L5 Unavailable LTO lib VTAPE7L5 Unavailable LTO lib VTAPE6L5 Unavailable LTO lib VTAPE5L5 Unavailable LTO lib VTAPE4L5 Unavailable LTO lib VTAPE3L5 Unavailable LTO lib VTAPE2L5 Unavailable LTO lib VTAPE1L5 Unavailable LTO lib VTAPE0L5 Unavailable LTO lib Ready-to-use Virtual Appliance for Hands-on Spectrum Archive Evaluation

17 A virtual tape library (lib1 with four virtual tape drives (VDRIVE VDRIVE0003 and 10 virtual tape cartridges (2.5 GB capacity for each VTAPE0L5 - VTAPE9L5 are attached. The drives are pre-configured. 5. Create a tape pool for data migration, as shown in Example 5. This process is required only once when the virtual appliance is first created and set up. Example 5 Creating a tape pool for data migration [root@spectrumscale ~]# ltfsee pool create pool1 [root@spectrumscale ~]# ltfsee pool add -p pool1 -t VTAPE0L5 -t VTAPE1L5 -t VTAPE2L5 -e GLESL042I(00640: Adding tape VTAPE0L5 to storage pool pool1. GLESL042I(00640: Adding tape VTAPE1L5 to storage pool pool1. GLESL042I(00640: Adding tape VTAPE2L5 to storage pool pool1. Tape VTAPE0L5 successfully formatted. Added tape VTAPE0L5 to pool pool1 successfully. lttape VTAPE1L5 successfully formatted. Added tape VTAPE1L5 to pool pool1 successfully. Tape VTAPE2L5 successfully formatted. Added tape VTAPE2L5 to pool pool1 successfully. [root@spectrumscale ~]# ltfsee info pools Pool Name Total(TiB Free(TiB Unref(TiB Tapes Type Library Node Group pool LTO lib1 G0 [root@spectrumscale ~]# ltfsee info tapes Tape ID Status Type Capacity(GiB Free(GiB Unref(GiB Pool Library Address Drive VTAPE2L5 Valid LTO pool1 lib VTAPE1L5 Valid LTO pool1 lib VTAPE0L5 Valid LTO pool1 lib VTAPE9L5 Unavailable LTO lib VTAPE8L5 Unavailable LTO lib VTAPE7L5 Unavailable LTO lib VTAPE6L5 Unavailable LTO lib VTAPE5L5 Unavailable LTO lib VTAPE4L5 Unavailable LTO lib VTAPE3L5 Unavailable LTO lib Spectrum Archive features In this section, the basic operations for Spectrum Archive are described. Manual migration Manual migration includes the following objectives: Check the file status to determine whether the file data is on disk or tape Understand the migration policy rule file Manually run the migration policy and verify that the file data was migrated 13

18 Complete the following steps: 1. Create test files under the /gpfs/fs1/archive test directory, as shown in Example 6. Example 6 Creating test files in /gpfs/fs1/archive directory [root@spectrumscale ~]# mkdir /gpfs/fs1/archive [root@spectrumscale ~]# cd /gpfs/fs1/archive [root@spectrumscale archive]# dd if=/dev/urandom of=10mfile_1 count=1 bs=10m [root@spectrumscale archive]# for i in $(seq 2 3; do cp 10Mfile_1 10Mfile_$i; done [root@spectrumscale archive]# yes 'This is test' dd of=text bs=2m count=1 [root@spectrumscale archive]# ls -la total rw-r--r--. 1 root root Feb 3 00:43 10Mfile_1 -rw-r--r--. 1 root root Feb 3 00:43 10Mfile_2 -rw-r--r--. 1 root root Feb 3 00:43 10Mfile_3 -rw-r--r--. 1 root root Feb 3 00:47 text 2. To determine whether file data is on disk or tape, run the dsmls command or ltfsee info files command, as shown in Example 7. Example 7 Confirming whether file data is on disk or tape [root@spectrumscale archive]# dsmls IBM Tivoli Storage Manager Command Line Space Management Client Interface Client Version 7, Release 1, Level 4.80 Client date/time: 02/03/ :48:05 (c Copyright by IBM Corporation and other(s 1990, All Rights Reserved. ActS ResS ResB FSt Prvd FName /gpfs/fs1/archive: r - 10Mfile_ r - 10Mfile_ r - 10Mfile_ r - text Example 8 File data is on disk only In the dsmls command output that is shown in Example 7, r indicates that the files are resident (on disk. As shown in Example 8, the file data is on disk only. [root@spectrumscale archive]# ltfsee info files /gpfs/fs1/archive/* Name: /gpfs/fs1/archive/10mfile_1 Name: /gpfs/fs1/archive/10mfile_2 Name: /gpfs/fs1/archive/10mfile_3 Name: /gpfs/fs1/archive/text The following file status also are available: Premigrated (p: Data is on disk and tape Migrated (m: Data is on tape only 14 Ready-to-use Virtual Appliance for Hands-on Spectrum Archive Evaluation

19 3. Run a manual policy to migrate these test files to tapes. The Policy file is found at: /root/policy/policy_migarchivedir. The contents of the policy file are shown in Example 9. Example 9 Policy contents define( exclude_list, PATH_NAME LIKE '/gpfs/fs1/.ltfsee/%' OR PATH_NAME LIKE '/gpfs/fs1/.spaceman/%' OR PATH_NAME LIKE '/gpfs/fs1/.snapshots/%' define( is_migrated, MISC_ATTRIBUTES LIKE '%V%' RULE EXTERNAL POOL 'Archive_files' EXEC '/opt/ibm/ltfsee/bin/ltfsee' OPTS '-p pool1@lib1' RULE 'Mig-test' MIGRATE FROM POOL 'system' TO POOL 'Archive_files' WHERE ( PATH_NAME LIKE '/gpfs/fs1/archive/%' AND NOT (exclude_list AND NOT (is_migrated AND (KB_ALLOCATED > 1024 When this policy is run, it migrates all of the resident and pre-migrated files that are in the /gpfs/fs1/archive directory from disk to tape. Note: Files that are less than 1 MB in size are excluded from migration for best practice. For more information about the policy, see Migration policy rule variations on page Run the mmapplypolicy command to run the policy, as shown in Example 10. Example 10 Running mmapplypolicy command [root@spectrumscale ~]# mmapplypolicy /gpfs/fs1/archive -P /root/policy/policy_migarchivedir [I] GPFS Current Data Pool Utilization in KB and % Pool_Name KB_Occupied KB_Total Percent_Occupied system % [I] 4580 of inodes used: %. [I] Loaded policy rules from /root/policy/policy_migarchivedir. Evaluating policy rules with CURRENT_TIMESTAMP = @05:52:51 UTC Parsed 2 policy rules. RULE EXTERNAL POOL 'Archive_files' EXEC '/opt/ibm/ltfsee/bin/ltfsee' OPTS '-p pool1@lib1' RULE 'Mig-test' MIGRATE FROM POOL 'system' TO POOL 'Archive_files' 15

20 WHERE ( PATH_NAME LIKE '/gpfs/fs1/archive/%' AND NOT (PATH_NAME LIKE '/gpfs/fs1/.ltfsee/%' OR PATH_NAME LIKE '/gpfs/fs1/.spaceman/%' OR PATH_NAME LIKE '/gpfs/fs1/.snapshots/%' AND NOT (MISC_ATTRIBUTES LIKE '%V%' AND (KB_ALLOCATED > 1024 [I] @05:52: Directory entries scanned: 5. [I] Directories scan: 4 files, 1 directories, 0 other objects, 0 'skipped' files and/or errors. [I] @05:52: Sorting 5 file list records. [I] Inodes scan: 4 files, 1 directories, 0 other objects, 0 'skipped' files and/or errors. [I] @05:52: Policy evaluation. 5 files scanned. [I] @05:52: Sorting 4 candidate file list records. [I] @05:52: Choosing candidate files. 4 records scanned. [I] Summary of Rule Applicability and File Choices: Rule# Hit_Cnt KB_Hit Chosen KB_Chosen KB_Ill Rule RULE 'Mig-test' MIGRATE FROM POOL 'system' TO POOL 'Archive_files' WHERE(. [I] Filesystem objects with no applicable rules: 1. [I] GPFS Policy Decisions and File Choice Totals: Chose to migrate 32768KB: 4 of 4 candidates; Predicted Data Pool Utilization in KB and %: Pool_Name KB_Occupied KB_Total Percent_Occupied system % GLESL167I(00509: A list of files to be migrated has been sent to LTFS EE using scan id GLESL038I(00555: Migration result: 4 succeeded, 0 failed, 0 duplicate, 0 duplicate wrong pool, 0 not found, 0 too small to qualify for migration. [I] @05:53: Policy execution. 4 files dispatched. [I] A total of 4 files have been migrated, deleted or processed by an EXTERNAL EXEC/script; 0 'skipped' files and/or errors. Example 11 Process complete When the mmapplypolicy command completes, the files should be migrated to tapes, as shown in Example 11. [root@spectrumscale ~]# dsmls /gpfs/fs1/archive IBM Tivoli Storage Manager Command Line Space Management Client Interface Client Version 7, Release 1, Level 4.80 Client date/time: 02/03/ :54:22 (c Copyright by IBM Corporation and other(s 1990, All Rights Reserved. ActS ResS ResB FSt Prvd FName <dir> archive/ /gpfs/fs1/archive: m ltfs 10Mfile_ m ltfs 10Mfile_ m ltfs 10Mfile_3 16 Ready-to-use Virtual Appliance for Hands-on Spectrum Archive Evaluation

21 m ltfs text [root@spectrumscale ~]# ltfsee info files /gpfs/fs1/archive/* Name: /gpfs/fs1/archive/10mfile_1 Tape id:vtape2l5@lib1 Status: migrated Name: /gpfs/fs1/archive/10mfile_2 Tape id:vtape2l5@lib1 Status: migrated Name: /gpfs/fs1/archive/10mfile_3 Tape id:vtape2l5@lib1 Status: migrated Name: /gpfs/fs1/archive/text Tape id:vtape2l5@lib1 Status: migrated If the files are read, they are recalled from the tapes, as shown in Example 12. Example 12 Files recalled from tapes [root@spectrumscale ~]# tail -5 /gpfs/fs1/archive/text This is test This is test This is test This is test This [root@spectrumscale ~]# dsmls /gpfs/fs1/archive/text IBM Tivoli Storage Manager Command Line Space Management Client Interface Client Version 7, Release 1, Level 4.80 Client date/time: 02/03/ :56:17 (c Copyright by IBM Corporation and other(s 1990, All Rights Reserved. ActS ResS ResB FSt Prvd FName p ltfs text [root@spectrumscale ~]# ltfsee info files /gpfs/fs1/archive/text Name: /gpfs/fs1/archive/text Tape id:vtape2l5@lib1 Status: premigrated Example 13 Resident status The file status is p (premigrated. After a read of the migrated file, the data is recalled on to disks and the data on the tape is still valid. If you modify the migrated (or premigrated file, the status is resident (r, as shown in Example 13. [root@spectrumscale archive]# echo aaa >>10Mfile_2 [root@spectrumscale ~]# dsmls /gpfs/fs1/archive/10mfile_2 IBM Tivoli Storage Manager Command Line Space Management Client Interface Client Version 7, Release 1, Level 4.80 Client date/time: 02/03/ :57:59 (c Copyright by IBM Corporation and other(s 1990, All Rights Reserved. ActS ResS ResB FSt Prvd FName r - 10Mfile_2 17

22 ~]# ltfsee info files /gpfs/fs1/archive/10mfile_2 Name: /gpfs/fs1/archive/10mfile_2 5. Clean up the test environment, as shown in Example 14. Example 14 Cleaning up the test environment ~]# rm -rf /gpfs/fs1/archive ~]# ltfsee info pools sed -e '1d' awk '{print $1}' xargs -I% ltfsee reconcile -p % [root@spectrumscale ~]# ltfsee info pools sed -e '1d' awk '{print $1}' xargs -I% ltfsee reclaim -p % Automatic migration Example 15 Test directory Automatic migration includes the following objectives: Understand the policy rule for automatic migration Configure the automatic migration Complete the following steps: 1. Create test files under a test directory called /gpfs/fs1/archive and run manual migration, as shown in Example 15. Ensure all test files are migrated by running the dsmls command. [root@spectrumscale ~]# mkdir /gpfs/fs1/archive [root@spectrumscale ~]# cd /gpfs/fs1/archive [root@spectrumscale archive]# dd if=/dev/urandom of=10mfile_1 count=1 bs=10m [root@spectrumscale archive]# for i in $(seq 2 3; do cp 10Mfile_1 10Mfile_$i; done [root@spectrumscale archive]# mmapplypolicy /gpfs/fs1/archive -P /root/policy/policy_migarchivedir 2. Add a callback to GPFS, as shown in Example 16. Example 16 Adding a callback to GPFS [root@spectrumscale ~]# mmaddcallback MIGRATION --command /usr/lpp/mmfs/bin/mmstartpolicy --event lowdiskspace --parms "%eventname %fsname --single-instance" [root@spectrumscale ~]# mmlscallback MIGRATION MIGRATION command = /usr/lpp/mmfs/bin/mmstartpolicy event = lowdiskspace parms = %eventname %fsname --single-instance The policy file for auto migration is at: /root/policy/policy_migauto. Unlike the policy_migarchivedir file, the policy_migauto does not pick up files in premigrate state, as shown in Example 17 on page 19. The THRESHOLD statement is described later in this section. 18 Ready-to-use Virtual Appliance for Hands-on Spectrum Archive Evaluation

23 Example 17 Policy file for auto migration define( exclude_list, PATH_NAME LIKE '/gpfs/fs1/.ltfsee/%' OR PATH_NAME LIKE '/gpfs/fs1/.spaceman/%' OR PATH_NAME LIKE '/gpfs/fs1/.snapshots/%' define( is_premigrated, MISC_ATTRIBUTES LIKE '%M%' AND MISC_ATTRIBUTES NOT LIKE '%V%' define( is_migrated, MISC_ATTRIBUTES LIKE '%V%' RULE 'SYSTEM_POOL_PLACEMENT_RULE' SET POOL 'system' RULE EXTERNAL POOL 'Archive_files' EXEC '/opt/ibm/ltfsee/bin/ltfsee' OPTS '-p RULE 'ARCHIVE_FILES' MIGRATE FROM POOL 'system' TO POOL 'Archive_files' THRESHOLD (0, 0 WHERE ( PATH_NAME LIKE '/gpfs/fs1/archive/%' AND NOT (exclude_list AND NOT (is_migrated AND NOT (is_premigrated AND (KB_ALLOCATED > 1024 Example 18 Applying the policy 3. Apply the policy by running the mmchpolicy command, as shown in Example 18. [root@spectrumscale archive]# mmchpolicy fs1 /root/policy/policy_migauto -t "System policy for LTFS" Validated policy `System policy for LTFS': Parsed 3 policy rules. Policy `System policy for LTFS' installed and broadcast to all nodes. Create a new test file under /gpfs/fs1/archive by copying a test file. [root@spectrumscale archive]# cp 10Mfile_1 10Mfile_AutoMigTest [root@spectrumscale archive]# dsmls IBM Tivoli Storage Manager Command Line Space Management Client Interface Client Version 7, Release 1, Level 4.80 Client date/time: 02/03/ :03:29 (c Copyright by IBM Corporation and other(s 1990, All Rights Reserved. ActS ResS ResB FSt Prvd FName 19

24 /gpfs/fs1/archive: p ltfs 10Mfile_ m ltfs 10Mfile_ m ltfs 10Mfile_ r - 10Mfile_AutoMigTest Example 19 File status change The migration is started for the newly created files after approximately 2 minutes. The file status changes from resident (r to migrated (m, as shown in Example 19. [root@spectrumscale archive]# dsmls... ActS ResS ResB FSt Prvd FName m ltfs 10Mfile_AutoMigTest Example 20 Cleaning up Notice that premigrated file is not migrated as defined in the policy. The policy rule that is used for this test had THRESHOLD (0, 0 in the RULE MIGRATE statement so that the lowdiskspace event always occurs, regardless of the disk space. These values can be adjusted, such as THRESHOLD (80, 60, when deploying at a customer s system so that the migration occurs only when 80% or more disk space is being used. The migration continues to run until the used disk space is lower than 60%. Also, WEIGHT can be used to control the order of files to migrate. For example, WEIGHT(CURRENT_TIMESTAMP - ACCESS_TIME makes the migration run from the files with the oldest POSIX. For more information, see the following resources: Timestamp and file size on page 49. IBM Linear Tape File System Enterprise Edition V Installation and Configuration Guide. 4. To clean up, delete the callback, set a default policy, and delete the test files, as shown in Example 20. [root@spectrumscale ~]# mmdelcallback MIGRATION [root@spectrumscale ~]# mmchpolicy fs1 DEFAULT [root@spectrumscale ~]# rm -rf /gpfs/fs1/archive [root@spectrumscale ~]# ltfsee info pools sed -e '1d' awk '{print $1}' xargs -I% ltfsee reconcile -p % [root@spectrumscale ~]# ltfsee info pools sed -e '1d' awk '{print $1}' xargs -I% ltfsee reclaim -p % Reconciliation and reclamation Reconciliation and reclamation includes the following objectives: Determine how GPFS and tape filesystem (LTFS can be out of sync Run the ltfsee reconcile command to synchronize LTFS and GPFS Run the ltfsee reclaim command to defragment a tape 20 Ready-to-use Virtual Appliance for Hands-on Spectrum Archive Evaluation

25 Example 21 Creating test files Complete the following steps: 1. Create test files under a test directory /gpfs/fs1/archive and run manual migration, as shown in Example 21. Ensure that all test files are migrated. [root@spectrumscale ~]# mkdir /gpfs/fs1/archive [root@spectrumscale ~]# cd /gpfs/fs1/archive [root@spectrumscale archive]# dd if=/dev/urandom of=10mfile_1 count=1 bs=10m [root@spectrumscale archive]# for i in $(seq 2 3; do cp 10Mfile_1 10Mfile_$i; done [root@spectrumscale archive]# mmapplypolicy /gpfs/fs1/archive -P /root/policy/policy_migarchivedir Example 22 Checking the tape ID For this test, we want to delete a migrated file. 2. Check the tape ID of a migrated file, as shown in Example 22. [root@spectrumscale archive]# ltfsee info files /gpfs/fs1/archive/10mfile_1 Name: /gpfs/fs1/archive/10mfile_1 Tape id:vtape1l5@lib1 Status: migrated In this case, the 10Mfile_1 file data is in Tape VTAPE1L5. The tape ID can be different on each virtual machine. 3. Delete the file, as shown in Example 23. Example 23 Deleting the file [root@spectrumscale ~]# rm -f /gpfs/fs1/archive/10mfile_1 Example 24 Checking files in tape 4. Check the files in tape by reviewing /ltfs/<tape id>, as shown in Example 24. It is expected that the file 10Mfile_1 is still in the corresponding tape. The actual output can differ in each test environment. [root@spectrumscale archive]# ls -l /ltfs/vtape1l5/gpfs/fs1/archive total 0 lrwxrwxrwx. 1 root root 81 Feb 3 01:13 10Mfile_1 ->../../../.LTFSEE_DATA/ lrwxrwxrwx. 1 root root 81 Feb 3 01:13 10Mfile_2 ->../../../.LTFSEE_DATA/ lrwxrwxrwx. 1 root root 81 Feb 3 01:13 10Mfile_3 ->../../../.LTFSEE_DATA/ Run the reconcile command, as shown Example 25. This command synchronizes the tape and the disk. The deleted files in gpfs are also deleted in /ltfs. Example 25 Running the reconcile command [root@spectrumscale archive]# ltfsee reconcile -p pool1 -l lib1 -t VTAPE1L5 GLESS016I(00178: Reconciliation requested. GLESM401I(00188: Loaded the global configuration. GLESS049I(00821: Tapes to reconcile: VTAPE1L5. GLESS050I(00833: GPFS file systems involved: /gpfs/fs1. GLESS134I(00887: Reserving tapes for reconciliation. GLESS135I(00920: Reserved tapes: VTAPE1L5. GLESS054I(01002: Creating GPFS snapshots: 21

26 GLESS055I(01009: Creating GPFS snapshot for /gpfs/fs1 ( /dev/fs1. GLESS056I(01035: Scanning GPFS snapshots: GLESS057I(01041: Scanning GPFS snapshot of /gpfs/fs1 ( /dev/fs1. GLESS060I(01087: Processing scan results: GLESS061I(01093: Processing scan results for /gpfs/fs1 ( /dev/fs1. GLESS141I(01108: Removing stale DMAPI attributes: GLESS142I(01114: Removing stale DMAPI attributes for /gpfs/fs1 ( /dev/fs1. GLESS063I(01153: Reconciling the tapes: GLESS001I(01214: Reconciling tape VTAPE1L5 has been requested. GLESS002I(01243: Reconciling tape VTAPE1L5 complete. GLESS137I(01365: Removing tape reservations. GLESS058I(02304: Removing GPFS snapshots: GLESS059I(02310: Removing GPFS snapshot of /gpfs/fs1 ( /dev/fs1. GLESL172I(00152: Synchronizing LTFS EE tapes information. [root@spectrumscale archive]# ls -l /ltfs/vtape1l5/gpfs/fs1/archive total 0 lrwxrwxrwx. 1 root root 81 Feb 3 01:16 10Mfile_2 ->../../../.LTFSEE_DATA/ lrwxrwxrwx. 1 root root 81 Feb 3 01:16 10Mfile_3 ->../../../.LTFSEE_DATA/ Run the reclaim command, as shown in Example 26. This command copies the data in the specified tape to a separate tape and the data is non-fragmented. The tape is automatically removed from pool1. Example 26 Running the reclaim command [root@spectrumscale archive]# ltfsee reclaim -p pool1 -t VTAPE1L5 Start reclaiming the following 1 tapes: VTAPE1L5. GLESL372I(00638: Validating tape VTAPE1L5 in pool pool1 Tape VTAPE1L5 is unmounted because it is inserted into the drive. Tape VTAPE1L5 successfully reclaimed, formatted, and removed from storage pool pool1. Reclamation complete. 1 tapes reclaimed, 1 tapes removed from the storage pool. [root@spectrumscale archive]# ltfsee info tapes Tape ID Status Type Capacity(GiB Free(GiB Unref(GiB Pool Library Address Drive VTAPE2L5 Valid LTO pool1 lib1 259 VDRIVE0003 VTAPE0L5 Valid LTO pool1 lib1 257 VDRIVE0001 VTAPE9L5 Unavailable LTO lib VTAPE8L5 Unavailable LTO lib VTAPE7L5 Unavailable LTO lib VTAPE6L5 Unavailable LTO lib VTAPE5L5 Unavailable LTO lib VTAPE4L5 Unavailable LTO lib VTAPE3L5 Unavailable LTO lib VTAPE1L5 Unavailable LTO lib [root@spectrumscale archive]# ls -l /gpfs/fs1/archive total 0 -rw-r--r--. 1 root root Feb 3 01:11 10Mfile_2 -rw-r--r--. 1 root root Feb 3 01:11 10Mfile_3 [root@spectrumscale archive]# ls -l /gpfs/vtape1l5 ls: cannot access /gpfs/vtape1l5: No such file or directory 22 Ready-to-use Virtual Appliance for Hands-on Spectrum Archive Evaluation

27 7. Return the reclaimed tape to the pool, as shown in Example 27. The tape is formatted as a new LTFS tape and added to the pool. Example 27 Returning the reclaimed tape [root@spectrumscale archive]# ltfsee pool add -p pool1 -t VTAPE1L5 -e [root@spectrumscale archive]# ltfsee info tapes Tape ID Status Type Capacity(GiB Free(GiB Unref(GiB Pool Library Address Drive VTAPE1L5 Valid LTO pool1 lib VTAPE2L5 Valid LTO pool1 lib1 259 VDRIVE0003 VTAPE0L5 Valid LTO pool1 lib1 257 VDRIVE0001 VTAPE9L5 Unavailable LTO lib VTAPE8L5 Unavailable LTO lib VTAPE7L5 Unavailable LTO lib VTAPE6L5 Unavailable LTO lib VTAPE5L5 Unavailable LTO lib VTAPE4L5 Unavailable LTO lib VTAPE3L5 Unavailable LTO lib Clean up the test environment, as shown in Example 28. Example 28 Cleaning up the environment [root@spectrumscale ~]# rm -rf /gpfs/fs1/archive [root@spectrumscale ~]# ltfsee info pools sed -e '1d' awk '{print $1}' xargs -I% ltfsee reconcile -p % [root@spectrumscale ~]# ltfsee info pools sed -e '1d' awk '{print $1}' xargs -I% ltfsee reclaim -p % Exporting and importing Exporting and importing includes the following objectives: Run the ltfsee export command to export a LTFS tape. Run the ltfsee import command to import a LTFS tape. Verify how imported files and directories appear in the file system. Complete the following steps: 1. Create test files under a test directory called /gpfs/fs1/archive and run manual migration, as shown in Example 29. Ensure all test files are migrated. Example 29 Creating test file directory (22:05:39 localhost:~ # mkdir /gpfs/fs1/archive (22:05:46 localhost:~ # cd /gpfs/fs1/archive (22:05:46 localhost:/gpfs/fs1/archive # dd if=/dev/urandom of=10mfile_1 count=1 bs=10m (22:05:47 localhost:/gpfs/fs1/archive # for i in $(seq 2 3; do cp 10Mfile_1 10Mfile_$i; done (22:05:49 localhost:/gpfs/fs1/archive # mmapplypolicy /gpfs/fs1/archive -P /root/policy/policy_migarchivedir 2. Check tape ID that contains one or more files by running the ltfsee info files command, as shown in Example 30 on page 24. Here, use the tape VTAPE2L5 that includes three files for export and import test. 23

28 Example 30 Checking the tape ID archive]# find /gpfs/fs1/archive -type f xargs ltfsee info files Name: /gpfs/fs1/archive/10mfile_1 Tape id:vtape2l5@lib1 Status: migrated Name: /gpfs/fs1/archive/10mfile_2 Tape id:vtape2l5@lib1 Status: migrated Name: /gpfs/fs1/archive/10mfile_3 Tape id:vtape2l5@lib1 Status: migrated Example 31 Exporting tape 3. Run the export process for the tape that contains the test files, as shown in Example 31. [root@spectrumscale archive]# ltfsee export -p pool1 -t VTAPE2L5 GLESS016I(00178: Reconciliation requested. GLESM401I(00188: Loaded the global configuration. GLESS049I(00821: Tapes to reconcile: VTAPE2L5. GLESS050I(00833: GPFS file systems involved: /gpfs/fs1. GLESS134I(00887: Reserving tapes for reconciliation. GLESS135I(00920: Reserved tapes: VTAPE2L5. GLESS054I(01002: Creating GPFS snapshots: GLESS055I(01009: Creating GPFS snapshot for /gpfs/fs1 ( /dev/fs1. GLESS056I(01035: Scanning GPFS snapshots: GLESS057I(01041: Scanning GPFS snapshot of /gpfs/fs1 ( /dev/fs1. GLESS060I(01087: Processing scan results: GLESS061I(01093: Processing scan results for /gpfs/fs1 ( /dev/fs1. GLESS141I(01108: Removing stale DMAPI attributes: GLESS142I(01114: Removing stale DMAPI attributes for /gpfs/fs1 ( /dev/fs1. GLESS063I(01153: Reconciling the tapes: GLESS086I(01169: Reconcile is skipped for tape VTAPE2L5 because it is already reconciled. GLESS137I(01365: Removing tape reservations. GLESS058I(02304: Removing GPFS snapshots: GLESS059I(02310: Removing GPFS snapshot of /gpfs/fs1 ( /dev/fs1. Export of tape VTAPE2L5 has been requested... Export of tape VTAPE2L5 complete. GLESL372I(00638: Validating tape VTAPE2L5 in pool pool1 Tape VTAPE2L5 is unmounted because it is inserted into the drive. Removing tape VTAPE2L5 from storage pool pool1. When the export process completes, the test files are deleted from GPFS and the tape is removed from pool, as shown in Example 32. In this case, the /gpfs/fs1/archive does not contain any file after export and the directory is removed from GPFS. Example 32 Completing the export process [root@spectrumscale ~]# ls -l /gpfs/fs1/archive ls: cannot access /gpfs/fs1/archive: No such file or directory [root@spectrumscale ~]# ltfsee info tapes Tape ID Status Type Capacity(GiB Free(GiB Unref(GiB Pool Library Address Drive VTAPE1L5 Valid LTO pool1 lib1 258 VDRIVE0002 VTAPE0L5 Valid LTO pool1 lib1 257 VDRIVE0001 VTAPE9L5 Unavailable LTO lib VTAPE8L5 Unavailable LTO lib VTAPE7L5 Unavailable LTO lib VTAPE6L5 Unavailable LTO lib VTAPE5L5 Unavailable LTO lib VTAPE4L5 Unavailable LTO lib VTAPE3L5 Unavailable LTO lib VTAPE2L5 Exported LTO lib Ready-to-use Virtual Appliance for Hands-on Spectrum Archive Evaluation

29 Example 33 Running the import 4. Run the import process. When the import process completes, the files in the imported tape are in the /gpfs/fs1/import directory and the status is migrated, as shown in Example 33. ~]# ltfsee import -p pool1 -t VTAPE2L5 GLESL042I(00640: Adding tape VTAPE2L5 to storage pool pool1. Tape VTAPE2L5 successfully checked. Added tape VTAPE2L5 to pool pool1 successfully. Import of tape VTAPE2L5 has been requested. Import of tape VTAPE2L5 complete. ~]# find /gpfs/fs1/import /gpfs/fs1/import /gpfs/fs1/import/vtape2l5 /gpfs/fs1/import/vtape2l5/gpfs /gpfs/fs1/import/vtape2l5/gpfs/fs1 /gpfs/fs1/import/vtape2l5/gpfs/fs1/archive /gpfs/fs1/import/vtape2l5/gpfs/fs1/archive/10mfile_1 /gpfs/fs1/import/vtape2l5/gpfs/fs1/archive/10mfile_2 /gpfs/fs1/import/vtape2l5/gpfs/fs1/archive/10mfile_3 ~]# ltfsee info files /gpfs/fs1/import/vtape2l5/gpfs/fs1/archive/* Name: /gpfs/fs1/import/vtape2l5/gpfs/fs1/archive/10mfile_1 Tape Status: migrated Name: /gpfs/fs1/import/vtape2l5/gpfs/fs1/archive/10mfile_2 Tape Status: migrated Name: /gpfs/fs1/import/vtape2l5/gpfs/fs1/archive/10mfile_3 Tape Status: migrated If you read the file, a recall occurs and the status changes to premigrated, as shown in Example 34. Example 34 Recall occurs when file is read [root@spectrumscale ~]# cat /gpfs/fs1/import/vtape2l5/gpfs/fs1/archive/10mfile_1 >/dev/null [root@spectrumscale ~]# ltfsee info files /gpfs/fs1/import/vtape2l5/gpfs/fs1/archive/10mfile_1 Name: /gpfs/fs1/import/vtape2l5/gpfs/fs1/archive/10mfile_1 Tape id:vtape2l5@lib1 Status: premigrated 5. Clean up the test environment, as shown in Example 35. Example 35 Cleaning up the environment [root@spectrumscale ~]# rm -rf /gpfs/fs1/archive /gpfs/fs1/import [root@spectrumscale ~]# ltfsee info pools sed -e '1d' awk '{print $1}' xargs -I% ltfsee reconcile -p % [root@spectrumscale ~]# ltfsee info pools sed -e '1d' awk '{print $1}' xargs -I% ltfsee reclaim -p % 25

30 Bulk recall Bulk recall process includes running the ltfsee recall command to start selective recalls. Migration test cases demonstrated a transparent recall of a file by running the cat <migrated file> command. In addition to the transparent recall, an explicit recall of a specified group of files can be run in a performance optimized manner (referred to as selective bulk recall. Complete the following steps: 1. Create test files, as shown in Example 36. Example 36 Creating test files [root@spectrumscale ~]# mkdir /gpfs/fs1/archive [root@spectrumscale ~]# cd /gpfs/fs1/archive [root@spectrumscale archive]# dd if=/dev/urandom of=10mfile_1 count=1 bs=10m [root@spectrumscale archive]# for i in $(seq 2 10; do cp 10Mfile_1 10Mfile_$i; done [root@spectrumscale archive]# mmapplypolicy /gpfs/fs1/archive -P /root/policy/policy_migarchivedir As shown in Example 37, 10 test files were migrated. Example 37 Migrated test files [root@spectrumscale archive]# find /gpfs/fs1/archive -type f sort xargs ltfsee info files Name: /gpfs/fs1/archive/10mfile_1 Tape id:vtape0l5@lib1 Status: migrated Name: /gpfs/fs1/archive/10mfile_10 Tape id:vtape0l5@lib1 Status: migrated Name: /gpfs/fs1/archive/10mfile_2 Tape id:vtape0l5@lib1 Status: migrated Name: /gpfs/fs1/archive/10mfile_3 Tape id:vtape0l5@lib1 Status: migrated Name: /gpfs/fs1/archive/10mfile_4 Tape id:vtape0l5@lib1 Status: migrated Name: /gpfs/fs1/archive/10mfile_5 Tape id:vtape0l5@lib1 Status: migrated Name: /gpfs/fs1/archive/10mfile_6 Tape id:vtape0l5@lib1 Status: migrated Name: /gpfs/fs1/archive/10mfile_7 Tape id:vtape0l5@lib1 Status: migrated Name: /gpfs/fs1/archive/10mfile_8 Tape id:vtape0l5@lib1 Status: migrated Name: /gpfs/fs1/archive/10mfile_9 Tape id:vtape0l5@lib1 Status: migrated 26 Ready-to-use Virtual Appliance for Hands-on Spectrum Archive Evaluation

31 2. Run the ltfsee recall command by passing the file names, as shown in Example 38. Example 38 Running the ltfsee recall command archive]# find /gpfs/fs1/archive -type f ltfsee recall GLESL277I(00303: The ltfsee recall command is called without specifying an input file waiting for standard input. If necessary press ^D to exit. GLESL268I(00142: 10 file name(s have been provided to recall. GLESL263I(00191: Recall result: 10 succeeded, 0 failed, 0 duplicate, 0 not migrated, 0 not found. [root@spectrumscale archive]# find /gpfs/fs1/archive -type f sort xargs ltfsee info files Name: /gpfs/fs1/archive/10mfile_1 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_10 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_2 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_3 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_4 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_5 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_6 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_7 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_8 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_9 Tape id:vtape0l5@lib1 Status: premigrated Also, a file list can be passed. 3. For testing purposes, run the migration policy again and have 10 files in migrated state, as shown in Example 39. Example 39 Total of 10 files in migrated state [root@spectrumscale archive]# mmapplypolicy /gpfs/fs1/archive -P /root/policy/policy_migarchivedir Example 40 Creating a list of files 4. Create a file list of these files and pass it to the ltfsee recall command, as shown in Example 40. [root@spectrumscale archive]# find /gpfs/fs1/archive -type f -name "10Mfile*" >/tmp/file_list [root@spectrumscale archive]# cat /tmp/file_list /gpfs/fs1/archive/10mfile_6 /gpfs/fs1/archive/10mfile_7 /gpfs/fs1/archive/10mfile_8 /gpfs/fs1/archive/10mfile_10 /gpfs/fs1/archive/10mfile_1 /gpfs/fs1/archive/10mfile_4 /gpfs/fs1/archive/10mfile_2 27

32 /gpfs/fs1/archive/10mfile_3 /gpfs/fs1/archive/10mfile_9 /gpfs/fs1/archive/10mfile_5 archive]# ltfsee recall -f /tmp/file_list GLESL268I(00142: 10 file name(s have been provided to recall. GLESL263I(00191: Recall result: 10 succeeded, 0 failed, 0 duplicate, 0 not migrated, 0 not found. [root@spectrumscale archive]# find /gpfs/fs1/archive -type f sort xargs ltfsee info files Name: /gpfs/fs1/archive/10mfile_1 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_10 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_2 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_3 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_4 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_5 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_6 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_7 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_8 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_9 Tape id:vtape0l5@lib1 Status: premigrated If a customer has millions of files to migrate, the find command can be slow. The mmapplypolicy command can be used to generate the file list faster. As shown in Example 41, the following policy can be used to enable the mmapplypolicy command to find the files and run the ltfsee recall command: /root/policy/policy_recallarchivedir Example 41 Enabling mmapplypolicy command RULE EXTERNAL POOL 'pool1' EXEC '/opt/ibm/ltfsee/bin/ltfsee' RULE 'ee_recall' MIGRATE FROM POOL 'pool1' TO POOL 'system' WHERE ( PATH_NAME LIKE '/gpfs/fs1/archive/%' Note: The rule ee_recall indicates data movement from pool1 (LTFS/tape to system (GPFS/disk. 28 Ready-to-use Virtual Appliance for Hands-on Spectrum Archive Evaluation

33 5. Run a manual migration to migrate all test files, as shown in Example 42. Example 42 Running a manual migration (08:44:14 localhost:~ # mmapplypolicy /gpfs/fs1/archive -P /root/policy/policy_migarchivedir 6. Run the policy and verify the status, as shown in Example 43. Example 43 Verifying the migration [root@spectrumscale archive]# mmapplypolicy /gpfs/fs1/archive -L 6 -P /root/policy/policy_recallarchivedir [root@spectrumscale archive]# find /gpfs/fs1/archive -type f sort xargs ltfsee info files Name: /gpfs/fs1/archive/10mfile_1 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_10 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_2 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_3 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_4 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_5 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_6 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_7 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_8 Tape id:vtape0l5@lib1 Status: premigrated Name: /gpfs/fs1/archive/10mfile_9 Tape id:vtape0l5@lib1 Status: premigrated 7. Clean up the test environment, as shown in Example 44. Example 44 Cleaning up the test environment [root@spectrumscale ~]# rm -rf /gpfs/fs1/archive [root@spectrumscale ~]# ltfsee info pools sed -e '1d' awk '{print $1}' xargs -I% ltfsee reconcile -p % [root@spectrumscale ~]# ltfsee info pools sed -e '1d' awk '{print $1}' xargs -I% ltfsee reclaim -p % Preserving file system objects on tape Preserving file system objects on tape includes the following objectives: Run a policy to save symbolic links and empty directories on a tape. Run the ltfsee rebuild or ltfsee import command to restore the file system objects from the tape to the file system. 29

34 Example 45 Creating test files Symbolic links, empty regular files, and empty directories are some file system objects that do not contain data or content. These objects can not be migrated to tapes by using the ltfsee migrate command. The policy rules that were used in this guide exclude directories and symbolic links by default. Also, (KB_ALLOCATED > 1024 statement excludes the empty file. (Non-empty directories are saved in tapes. Although this configuration is available for pure space management purposes, a customer might want to use tapes in the following ways: Export or import to replicate the entire file system from one system to another. As a complete backup of the file system so that if the file system is corrupted, it can be recovered from the tape. For these needs, the ltfsee save command can be used. Complete the following steps to create a simple file list that is used for saving the data: 1. Create the test files, as shown in Example 45. [root@spectrumscale archive]# mkdir dir emptydir [root@spectrumscale archive]# cd dir [root@spectrumscale dir]# cp../10mfile_1. [root@spectrumscale dir]# touch emptyfile [root@spectrumscale dir]# ln -s 10Mfile_1 sln_10mfile_1 [root@spectrumscale ~]# ls -lr /gpfs/fs1/archive /gpfs/fs1/archive: total rw-r--r--. 1 root root Feb 3 02:17 10Mfile_1 drwxr-xr-x. 2 root root 4096 Feb 3 02:17 dir drwxr-xr-x. 2 root root 4096 Feb 3 02:17 emptydir /gpfs/fs1/archive/dir: total rw-r--r--. 1 root root Feb 3 02:17 10Mfile_1 -rw-r--r--. 1 root root 0 Feb 3 02:17 emptyfile lrwxrwxrwx. 1 root root 9 Feb 3 02:17 sln_10mfile_1 -> 10Mfile_1 /gpfs/fs1/archive/emptydir: total 0 [root@spectrumscale ~]# ltfsee info files /gpfs/fs1/archive/emptydir/ Name: /gpfs/fs1/archive/emptydir/ [root@spectrumscale ~]# ltfsee info files /gpfs/fs1/archive/dir/sln_10mfile_1 Name: /gpfs/fs1/archive/dir/sln_10mfile_1 30 Ready-to-use Virtual Appliance for Hands-on Spectrum Archive Evaluation