Wearing Both Belt and Suspenders: Not the latest in fashion, but it keeps your butt covered

Transcription

1 Wearing Both Belt and Suspenders: Not the latest in fashion, but it keeps your butt covered EMC Proven Professional Knowledge Sharing October, 2007 Tore Eikanger Senior Systems Engineer Ementor AS

2 Table of Contents Summary... 3 Introduction... 3 Setting it all up... 4 Pools... 4 Schedules... 4 Groups... 5 Notification... 6 Clients... 7 About the author... 9 Disclaimer: The views, processes or methodologies published in this compilation are those of the author. They do not necessarily reflect EMC Corporation s views, processes or methodologies. 2

3 SUMMARY When we ask customers whether they want security or long-term nearline storage, they tend to answer, Yes, I ll take both please. This article describes a method for using groups, schedules, pools and scripts to maintain two copies of your backup data, and to utilize your tape library to keep your backups nearline as long as possible. The scenario we are building consists of a NetWorker server, a number of clients, one disk device and a tape library. INTRODUCTION When we plan an EMC NetWorker installation, we place a disk device in a fire cell separate from the backup and production servers. Among the reasons for this: The disk device will contain the latest backups It will be the easiest device to set up in a new environment in case of disaster recovery Disk devices support concurrent restores The disk device can be a single USB attached disk to a NAS server or a SAN disk array. EMC NetWorker uses the built-in cloning function to maintain two copies of your backup data, and stage data to move from one media to another. Combining the two is not easy, and deadlock situations often occur when the automatic staging and cloning fight for devices to perform its function. To avoid this, we are going to use cloning only, and a script to remove the original that is on the disk device. When I first started working with backups, we used the grandfather-father-son media rotation scheme to maintain backup history. To keep backup data nearline for as long as possible, we are going to use this scheme to keep daily backups for a month, weekly backups for the next two months, and monthly backups for a year. We are using several groups and pools for backup and cloning. To keep the amount of data to backup as small as possible, we are going to use a month-based full, differential and incremental schedule for file system backups. Customers seldom need to restore databases older than a couple of weeks. For this reason, we are running database backups in a weekly full and incremental schedule that we keep for a month. We treat mail servers the same way as database servers. If the customer needs longer history for conversations, or to comply with regulations, we recommend an archiving solution. 3

4 SETTING IT ALL UP POOLS We assign the disk device to the Default pool which is the destination for all backups. For cloning in a grandfather-father-son scheme, we set up three pools: Keep a month, Keep a quarter and Keep a year. We need a backup clone that is able to recycle to and from other pools, will not store index entries, and has a retention policy based on backup policies. We enable recycle to/from other pools to let NetWorker dynamically allocate tapes between these pools. This removes the guesswork in assigning the right amount of tape to each pool. We do the initial assignment; NetWorker will then redistribute tapes when needed. SCHEDULES For file system backup, we use monthly schedules to obtain a full backup the first weekend of the month, incremental during weekdays and differential backups the rest of the weekends. Since we clone these backups automatically after the groups are finished, we must separate the backup levels in different schedules. The first schedule handles full backups. We call it FS_Full. To set it up in the GUI, select Schedules, click configuration and turn off Show Schedule as Calendar. Create a new schedule with the following properties: Name : FS_Full Period : Week Action : skip skip skip skip skip skip skip Override : full first friday every month The FS_Diff schedule handles differential backups. The properties are as follows: Name : FS_Diff Period : Week Action : skip skip skip skip skip 5 skip Override : skip first friday every month Our customer works on a 5-day work schedule. We create a schedule, FS_Incr, for handling the incremental backups Monday to Thursday: Name : FS_Incr Period : Week Action : skip incr incr incr incr skip skip Override : 4

5 For handling database backups, we create a weekly full and incremental schedule with these properties: Name : DB_Full_Incr Period : Week Action : skip incr incr incr incr full skip Override : If we turn on Show Schedule as Calendar, and view the newly created schedules, we will easily see the save levels of the backups during the month. GROUPS We use groups to organize which clients are backed up at what time, at which backup level and to which pool we cloned the save sets. For file system backups, we create three groups with the following properties: Name : Grp_Full Start time : 23:50 Autostart : Enabled Clones : Enabled Clone pool : Keep a year Schedule : FS_Full Name : Grp_Diff Start time : 23:50 Autostart : Enabled Clones : Enabled Clone pool : Keep a quarter Schedule : FS_Diff Name : Grp_Incr Start time : 23:50 Autostart : Enabled Clones : Enabled Clone pool : Keep a month Schedule : FS_Incr For database backups, the group looks like this: Name : Grp_DB Start time : 22:00 Autostart : Enabled Clones : Enabled Clone pool : Keep a month Schedule : DB_Full_Incr 5

6 NOTIFICATION We create a custom notification to trigger the script to purge old save sets from disk, keeping the clones on tape. Make a copy of the existing notification Filesystem full - recover adv_file space. Call it Filesystem full remove savesets on disk. In the action field, put: C:\Program Files\Legato\nsr\bin\purge-ss.cmd. The purge-ss.cmd script looks like this: REM ********************************************************************** REM ** This batch-file creates a list of all savesets on the disk-device ** REM ** with more than two copies, i.e. also cloned to tape. The list is ** REM ** sorted with the oldest savesets first and output to a file. ** REM ** This file is then parsed, and the size of the savesets are added ** REM ** together until it reaches the amount of MB we want to free up on ** REM ** the advanced file-device. The save sets that where added up, are ** REM ** removed from disk using ssid/cloneid, thus keeping the clone ** REM ** located on tape. ** REM ** ** REM ** This script is provided AS IS, with no support or warranties of ** REM ** any kind, including but not limited to warranties of ** REM ** merchantability or fitness of any kind, expressed or implied ** REM ** ** REM ** 2007 Tore Eikanger, Ementor Norge AS ** REM *********************************************************************** REM Remove temp file used. del %temp%\ssdelete.txt del %temp%\mminfo.txt del %temp%\nsrstage.txt REM Counter for saveset sizes set totalsize=0 REM Amount of data in MB to be purged from disk. set freeup= REM Volume name of the advanced-file device (case-sensitive) set volname=diskdev1 REM Create list of ssid/cloneid ordered by time and with total size less that freeup mminfo -xc, -q "volume=%volname%.ro,copies>2" -r "ssid, cloneid, sumsize, client, savetime, level, name" -ot>%temp%\mminfo.txt for /f "skip=1 tokens=1,2,3,4* delims=, " %%f in (%temp%\mminfo.txt) do call :PARSE %%f %%g %%h %%i REM Delete savesets on diskdevice for /f %%f in (%temp%\ssdelete.txt) do nsrmm -d -v -y -S %%f REM Clean up the adv_file device nsrstage -vvvv -C -V %volname%.ro >%temp%\nsrstage.txt goto END REM Here we add together sizes in KB, Mb and GB ignoring sizes in bytes 6

7 REM Sizes are accumulated in MB :PARSE if %totalsize% LEQ %freeup% ( echo %1/%2 >> "%4"=="MB" ( set /a totalsize="totalsize + %3" ) else ( if "%4"=="GB" ( set /a totalsize="totalsize + (%3 * 1024)" ) else ( if "%4"=="KB" ( set /a totalsize="totalsize + (%3 / 1024)" )))) goto :EOF :END CLIENTS Finally, we define clients into this environment. Clients with file-system backups must be members of the following groups: Grp_Full, Grp_Diff and Grp_Incr. Database backups must be member of the Grp_DB group. All clients should have a browse and retention policy equal to the son in our Grandfather-father-son setup, i.e. one month. The retention policy defined in the destination pool for the clones will extend the retention time for the save set once it has been cloned. To fully utilize the capacity of the disk device, we enable compression directives according to the clients OS, for instance NT with compression directive. Typically we will be able to store somewhere between 1.5 and twice the amount of data on the disk device when enabling compression. HOUSEKEEPING Savegroups will occasionally fail or be interrupted before cloning is complete. This will leave save sets on disk with no clones on tape. To handle this, we schedule a script to run once a week, staging orphaned save sets to tape. Take care to schedule this script to a day and time when the devices will not be busy with backup, cloning or restores. In our scenario, we run this script every night between Sunday and Monday. The script looks like this: REM ********************************************************************** REM ** This batch-file creates a list of all savesets on the disk-device ** REM ** with only two copies, i.e. no clones on tape. The list is parsed ** REM ** sorting savesets according to which group it belongs. These save ** REM ** sets are then staged to the pool in which it should have been ** REM ** cloned to initially. ** REM ** ** REM ** This script is provided AS IS, with no support or warranties of ** REM ** any kind, including but not limited to warranties of ** REM ** merchantability or fitness of any kind, expressed or implied ** REM ** ** 7

8 REM ** 2007 Tore Eikanger, Ementor Norge AS ** REM *********************************************************************** REM Volume name of the advanced-file device (case-sensitive) set volname=diskdev1 REM Set policy for which savesets are staged set stagetime=two weeks del %temp%\*.txt REM We find savesets that are not cloned, and output the ssid's to separate REM files named with group name. We are also adding the group name to groups.txt REM if it isn't already present there. for /f "skip=1 tokens=1,2*" %%f in ('mminfo -a -q "volume=%volname%, copies=2, savetime<%stagetime%" -r "ssid,group"') do echo %%f>>"%temp%\%%g.txt" & findstr %%g %temp%\groups.txt echo %%g >>%temp%\groups.txt REM We find if there are defined active clone pools. If so, start cloning for /f %%f in (%temp%\groups.txt) do echo print type: NSR group; Name: %%f; clones: Yes nsradmin find "clone pool:">%temp%\cpool.txt && call :CLONE %%f goto END :CLONE REM Extract clone pool to variable cpool for /f "tokens=1,2* delims=;:" %%f in (%temp%\cpool.txt) do set cpool=%%g REM Remove space from start of variable set cpool=%cpool:~1% REM Start staging of savesets nsrstage -d -b "%cpool%" -m -S -f "%temp%\%1" goto :EOF :END Name this script, for instance, housekeeping.cmd, and place it in the bin directory of NetWorker. We can schedule this script with anat command from the command prompt: at 03:00 /every: mo C:\Program Files\Legato\nsr\bin\housekeeping.cmd This completes our setup of the backup environment. As with all backup software installations, it must be monitored for a period of time to ensure that all is running smoothly. Use this setup as a framework to build customized installations. Some customers may want WORM media incorporated with retention policy forever. Others may want the script modified to set the browse time of the cloned save sets equal to the new retention time before deleting the original save set on disk. Feel free, and enjoy. 8

9 ABOUT THE AUTHOR Tore is a Senior Systems engineer working for Ementor AS, Norway s largest IT infrastructure solution provider. He has been working with NetWorker products since He has been working in the IT industry for nearly 20 years, gaining broad experience in different areas including DOS scripting, IBM AS/400, Novell NetWare, Linux, Solaris, Microsoft Windows, Cisco networking and training from different hardware vendors. This broad experience helps him to overcome backup related challenges related to hardware, OS and applications. 9