ImprovingBackup andrecovery Strategies

Transcription

1 ImprovingBackup andrecovery Strategies The Sun StorEdge SAM-FSStorage andarchive Manager File System from Sun Microsystems, Inc. Technical White Paper

2 2002 Sun Microsystems, Inc. All rights reserved. Printed in the United States of America. 901 San Antonio Road, Palo Alto, California U.S.A RESTRICTED RIGHTS LEGEND: Use, duplication, or disclosure by the government is subject to restrictions as set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software clause at DFARS and FAR The product described in this manual may be protected by one or more U.S. patents, foreign patents, or pending applications. TRADEMARKS Sun, Sun Microsystems, the Sun logo, Sun StorEdge SAM-FS, and Solaris are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries.unix is a registered trademark in the United States and other countries, exclusively licensed through X/Open Company, Ltd. THIS PUBLICATION IS PROVIDED AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. THIS PUBLICATION COULD INCLUDE TECHNICAL INACCURACIES OR TYPOGRAPHICAL ERRORS. CHANGES ARE PERIODICALLY ADDED TO THE INFORMATION HEREIN; THESE CHANGES WILL BE INCORPORATED IN NEW EDITIONS OF THE PUBLICATION. SUN MICROSYSTEMS, INC. MAY MAKE IMPROVEMENTS AND/OR CHANGES IN THE PRODUCT(S) AND/OR THE PROGRAM(S) DESCRIBED IN THIS PUBLICATION AT ANY TIME. Please Recycle

3 TableofContents 1. Backup TechnologyofthePast... 1 ThePeaceofMind... 1 TheBitterTruth WhatDataNeedstoBeProtected?... 3 TheFileServerIssue... 3 TheEnterpriseIssue OutdatedBackupStrategies... 7 FullBackupandRecovery... 7 IncrementalBackup... 8 Server-lessbackup WhyisBackupOutdatedandaTechnologyofthePast? DataGrowthHasOutgrownCurrentBackupStrategies DataRecoveryisTooTimeConsuming Scaling Problems: Backup Cannot Drive All the Hardware Needed.. 12 Enterprise-wideBackupIsLimitedbytheNetwork Excessive Media Usage and Cost Enhancing Backup Strategies with the SunStorEdgeSAM-FSFileSystem The Sun StorEdge SAM-FS File System Easily KeepsUpwithDataGrowth ProtectingDataforEasyRecoverywithoutaFullBackup Protecting and Restoring Multiple File Versions or Deleted Files RecoverLargestFileSystemsinaFractionofTime i

4 PhaseOne PhaseTwo PhaseThree Sun StorEdge SAM-FS File System Scales Linearly withavailablehardware Optimizing Enterprise-wide Backup Strategies with the Sun StorEdge SAM-FS File System RestoringBackupData EfficientMediaUsageandLimitedCost Advanced Options in the Sun StorEdge SAM-FS File System for OptimizedFileRestoration The Sun StorEdge SAM-FS File System stage never Feature Using mentation Technology of Sun StorEdge SAM-FS File System Conclusion-TheDifference ii Improving Backup and Recovery Strategies

5 List of Figures and Tables Figure1:ServerBackup... 3 Figure2:EnterpriseBackup... 4 Figure3:FullBackupandRecovery... 8 Figure4:IncrementalBackupandRecovery... 8 Figure5:Server-lessBackupandRecovery... 9 Figure6:MarketSize Table 1: Backup Resources Needed for 1, 10, and 30 TB Backups Figure7:CurrentEnterpriseBackup Figure8:MediaUsedforFullBackup Figure9:ExcessiveMediaUsageforFullBackup Figure 10: Sun StorEdge SAM-FS File System Continuous Backup Figure 11: Multiple Copies with the Sun StorEdge SAM-FS File System 23 Figure 12: Sun StorEdge SAM-FS File System Concurrent Online..... andnear-linefiles Figure 13: Sun StorEdge SAM-FS File System Multi-threaded Architecture 26 Figure 14: Sun StorEdge SAM-FS File System Enhances Enterprise Backup 27 Figure 15: Sun StorEdge SAM-FS File System Enhances Tape Utilization 28 Figure 16: Sun StorEdge SAM-FS File System Enhances Restore Figure 17: Sun StorEdge SAM-FS File System Optimizes Media Usage. 30 Figure 18: Sun StorEdge SAM-FS File System Optimized File Access.. 33 Figure19:SunStorEdgeSAM-FSFileSystemmentation Figure 20: Sun StorEdge SAM-FS File System mented File Management 35 Figure 21: Sun StorEdge SAM-FS File System Optimizes Restore Improving Backup and Recovery Strategies iii

6 iv Improving Backup and Recovery Strategies

7 Backup Technologyofthe Past 1 The Peace of Mind Historically, backup has been used to protect data from data loss. Data is copied using different backup techniques at specific times to secondary media, usually tape. The main reason for backup is to have the ability to restore data in case of a disaster, user error, hardware failure, or to recover old versions of a file. Using backup to protect data for data recovery when needed gave IT departments peace of mind that their data was safe. However, IT departments learned quickly that protecting data with backup is not really the problem. The real problem is restoring data when needed. Backing up or restoring enormous amounts of data in a reasonable time challenges IT departments. Often, backing up or restoring entire systems, or the enterprise, after a disaster takes many hours or even days. During the backup and restore process, users and applications do not have access to the data and need to wait until the entire backup or restore process is complete. Due to the global economy, many companies today can no longer afford to take data offline for a long period of time to back up data. This means that the time window for backup is constantly shrinking while the amount of data to be backed up is growing rapidly. In this environment, backup becomes more and more difficult or even impossible. Hardware or media failures, catastrophes, and scalability issues are constantly adding to the backup problem. Data can no longer be protected properly and companies are vulnerable to data loss. Improving Backup and Recovery Strategies 1

8 1 The Bitter Truth Current backup strategies worked well during the past decade. These strategies are no longer feasible as the amount of data continues to grow exponentially and a 24x7 global economy shrinks what is left of the backup and recovery window. The major bottlenecks expose themselves in the time it takes to back up extremely large file systems on local servers or to move large amounts of data directly to a tape drive via the network to a centralized backup server in an enterprise backup scenario. Moving ahead, the backup application must be able to back up more data in less time. The problem will persist as long as the amount of enterprise data continues to grow rapidly and the time window for backup becomes non-existent. 2 Improving Backup and Recovery Strategies

9 What DataNeeds to BeProtected? 2 The File Server Issue One way to implement a backup strategy is to back up data on a local or centralized server in file serving environments. In this scenario, the local file server stores most or all of the data of the enterprise. This data is made available to clients via the LAN, using common IP network protocols like NFS, ftp, or CIFS. Backup applications protect data on this local server by making full and incremental copies of the data directly to the local tape drive. With this configuration, no network is used for the local data transfer. The data is transferred directly to the tape devices connected to the local server. Usually, a full and incremental backup strategy is used to back up the local file systems. Backup Server Local File System Figure 1 Server Backup Improving Backup and Recovery Strategies 3

10 2 Server backup and recovery inherits the same limitations discussed before. The vast amount of data to be managed, backed up, or recovered keeps growing rapidly and a full server backup or recovery is no longer feasible in the time window available. The Enterprise Issue The current enterprise-wide backup strategy allows disparate enterprise-wide network clients to automatically move the backup data, via a network, to a tape drive connected to a backup server. Automated tape libraries with multiple tape drives allow multiple backup streams to be received from multiple backup clients in parallel. In a server backup implementation, data is backed up from the local file systems directly to local tape devices. Acommonbackupstrategyfortheenterpriseistodeployoneorseveralbackup servers throughout the enterprise. Backup clients are deployed on every system or workstation in the enterprise that needs backup. It is not uncommon for customers to have two to three backup servers and hundreds of backup clients. On a regular time schedule, the backup clients send data, utilizing an incremental or full backup strategy, via a TCP/IP network directly to a tape drive connected to one of the backup servers. Backup Clients TCP/IP Backup Server Figure 2 Enterprise Backup 4 Improving Backup and Recovery Strategies

11 2 Enterprise backup and recovery solutions inherit the same limitations discussed before. The vast amount of data to be managed, backed up, or recovered keeps growing rapidly and a full enterprise backup or recovery is no longer feasible in the time window available. To back up or recover data in the enterprise, data must be moved from tape via a network to the clients. The networks become the bottleneck, limit the backup and restore performance, and add unnecessary time. Improving Backup and Recovery Strategies 5

12 2 6 Improving Backup and Recovery Strategies

13 OutdatedBackupStrategies 3 Full Backup and Recovery A common backup strategy is to fully back up every file on clients in the enterprise or on local file systems on local servers, at least once a week. Full backup strategies can include keeping many backup sets, that is, file or system backup versions for backups collected over a specific time period. Backup applications are also capable of making copies of backup media, also called backup clones, to protect the backed up data when media errors occur. These clones can only be made automatically, but they must be done from the original backup tape. Full enterprise or server backups can be very time consuming. They require backup of every file, whether changed or not, during every full backup cycle. Because of the time required to complete the full backup cycle, a full backup is usually performed once a week during weekends. This implies that files created or changed during the week are not protected by the full backup strategy. Users do not have access to their data during the full backup or recovery cycle. Improving Backup and Recovery Strategies 7

14 3 New and Changed Data File System All Data Tape Library Unchanged Data Figure 3 Full Backup and Recovery Incremental Backup To enable customers to protect and restore files and file versions between the full backup cycles, customers have deployed incremental backup strategies. With incremental backup, only new or changed files are copied to secondary storage devices. This strategy allows customers to back up and protect newly created or changed files more efficiently without having to complete a full backup of all files every day. The incremental backup is usually performed on a daily basis, at times of no user activity. Each incremental backup creates its own daily restoration point for new and changed files. For an enterprise-wide backup, incremental backup also minimizes network traffic during the normal work week, because only the changed and new files need to be moved through the backup network. After taking daily incremental backups during the normal work week, a new full backup is performed on the weekend. Unchanged Data File System New or Changed Data Only Tape Library Figure 4 Incremental Backup and Recovery 8 Improving Backup and Recovery Strategies

15 3 Server-less Backup With the introductions of SANs, backup applications have implemented data mover strategies in which data is moved via a separate backup network or Fibre Channel SAN directly from disk to tape. However, this solution does not resolve all the bottlenecks, because typically only the main servers are connected to these fast and expensive networks. In an enterprise environment, most workstations are connected via the TCP/IP network and data is written from the client directly to a tape drive via the relatively slow, but much cheaper, LAN. In addition, a server must be involved in initiating and controlling the data moving over the SAN. Since a server must be involved for initiation and record keeping, the common usage in the industry as server-less for this type of backup is misleading. Appliance Backup/ Restore Path Backup Initiator Fibre Channel Clients TCP/IP Backup Server Intelligent Switch Local File System Tape Library Figure 5 Server-less Backup and Recovery Improving Backup and Recovery Strategies 9

16 3 10 Improving Backup and Recovery Strategies

17 WhyisBackupOutdatedanda Technologyofthe Past? 4 Data Growth Has Outgrown Current Backup Strategies The growing amount of data in today s enterprise environment is attributed to the Internet, faster computers, more powerful applications, and the conversion from analog to digital data. Data research companies suggest that the amount of data in an average enterprise will at least triple in the next two to three years. It is not uncommon today to have 10 TB of data on local file systems or distributed throughout an enterprise, that must be backed up or restored quickly by backup applications. $3,000 $2,500 $2,000 $1,500 $1,000 Advanced Storage Market Backup Total Market $500 $ Note: Market size in Millions. Excludes mainframe-only software. Source: Strategic Resource Corp. Figure 6 Market Size Improving Backup and Recovery Strategies 11

18 4 Data Recovery is Too Time Consuming In addition, a global economy does not allow data to be offline during nights or weekends to create full backups. Corporations cannot wait many hours or days to be back online and to continue business as usual in case of a disaster or hardware failure. This limits the time window for backup and recovery and creates an even bigger problem for current backup and recovery strategies. The only reason to back up files on servers or through the enterprise is to be able to recover files when needed. Due to the ever-increasing amount of data, disaster recovery cannot be achieved easily with current full and incremental backup strategies. First, the initial full backup must be restored. Then, every incremental backup must be restored sequentially in order to restore the file system or client system to its accurate state. Restoring an entire server or enterprise is extremely time consuming and difficult. It can take many hours or days. Users do not have access to their data or system until all files are restored andtherestoreoperationiscomplete.valuabletimeandmoneyislostduring the time consuming recovery process. This strategy is unacceptable to any conscientious business. Because backup of large amounts of data is very time consuming, many companies can afford to make only one copy of the data to tape. This makes protected data vulnerable to media errors or tape failures, and data can be lost in case of a disaster recovery. To solve this issue, companies are making multiple copies of backup tapes. These copies are called backup clones. A backup clone can only be created from the original backup copy. These backup clone tapes are usually moved off-site into a vault. In case of a catastrophe and loss of the original backup media, customers can use the backup clones to recover their data. If no backup clones are created or backup tapes are not moved off-site, companies are not protected from data loss due to a catastrophe or total loss of backup tapes. Cloning of backup tapes is very time consuming. Making clones multiplies the time it takes to create the additional backup sets and adds tremendous media cost and administration overhead. Scaling Problems: Backup Cannot Drive All the Hardware Needed While the amount of data on local file servers increases steadily, a full backup or restore can no longer be accomplished in the time allowed. For example, to complete a full backup or restoration of a 10 TB file system in 24 hours, the backup application must be able to stream data at a rate of approximately Improving Backup and Recovery Strategies

19 4 MB/sec. for 24 hours, to or from local tape drives. Tape also incurs overhead for loading and positioning. With current tape technology, streaming data at 113 MB/sec. requires about 15 tape devices with an average performance of 10 MB/sec. This number includes time for loading, unloading, and positioning, and it assumes that the backup application is capable of fully streaming data to all 15 local devices. If the backup or restore window for 10 TB is 6 hours, the customer would have to deploy four times as many tape drives, that is, 60 tape drives on the local servers, to complete a full backup. Table 1 shows time, hardware, and streaming requirements needed to back up 1, 10, and 30 TB of data. Given the data growth rate, a customer would need approximately 180 tape drives streaming at 10 MB/sec. to back up or restore 30 TB of data in 6 hours. #TB Continuous Data Stream of... # of 10 MB Drives Needed in 24 hours # of 10 MB Drives Needed in 6 hours 1 ~11.3 MB/sec ~11.3 MB/sec ~339 MB/sec Table 1 Backup Resources Needed for 1, 10, and 30 TB Backups This is not feasible because of the: Limited number of tape drives in a tape library Limited number of I/O channels in a server Limited scalability of the backup application Inability to keep all tape drives streaming at one time Restoringorbackingupdatain6hoursisstilltoolongformostcompanies.Data cannot be accessed during this time, so these companies must deploy even more hardware in order to decrease the backup and restore time. Improving Backup and Recovery Strategies 13

20 4 Enterprise-wide Backup is Limited by the Network Enterprise-wide backup shares all the limitations of the local backup as discussed previously. Enterprise-wide backup must utilize the LAN to back up data from hundreds of workstations to one or multiple backup servers, which creates additional problems. Most customers today have a 100 Mbit Ethernet LAN installed throughout their enterprise. The maximum data transfer rate through a 100-Mbit Ethernet LAN, at best case, is 10 MB/sec. If the total enterprise data on these hundreds of workstations is 10 TB, the site needs a minimum of ten 100-Mbit Ethernet networks in order to receive the 10 TB in 24 hours. The reality is that no one achieves 10 MB/sec. on a 100-Mbit Ethernet, nor can sites operate within a backup cycle of 24 hours. This solution has become obsolete! Further compounding with enterprise backup through the LAN is the inability of backup clients to write directly to the tape drive on the server with or without minimal caching. In an environment with four clients writing backups to four tape drives installed on a server, there is no way to keep the drives streaming. If the tape technology in use is capable of writing at 10 MB per second continuously to tape and the 100 Mbit Ethernet network offers a maximum of 10 megabytes per second, we cannot keep the four tape drives streaming (10 / 4 = 2.5). Each tape drive gets about 2.5 MB/sec. of data. This means that the tape drive actually records in a start/stop motion (shoe-shining). Remember that every time a tape drive is stopped, the performance decreases dramatically due to the overhead of stopping and re-starting the tape drive. Aggregate performance can fall well below 1 MB/sec. if a tape drive is ever stopped during data transfer because it is not fed enough data to keep streaming. Deploying even faster and newer tape technologies only worsens the problem. 14 Improving Backup and Recovery Strategies

21 4 Backup Clients Current Enterprise Backup TCP/IP Library contains four tape drives NOTE: Number of concurrent backup streams depends on available tape drives in the library!!! Backup Server Figure 7 Current Enterprise Backup Given all these facts, backup of 10 TB of enterprise data is no longer feasible using this method. How can we solve these inherited backup problems going forward? Excessive Media Usage and Cost Current backup strategies require much more media space than actual data space. Each full backup creates a new copy of a file on tape, whether the file has changed or not. If a file exists at the beginning of a year, has not changed, and the customer selects to keep one full backup every week, this file will have 52 copies on 52 different pieces of media at the end of the year. This wastes an enormous amount of media and adds drastically to media cost and administration overhead. The following simple example for full backup shows the amount of media needed per year. At the beginning of the year, the enterprise has a total of 10 TB and grows at 50 GB per week. The customer needs to keep a full backup every quarter and one at the end of the year. We assume that the average tape capacity is 50 GB per tape. Improving Backup and Recovery Strategies 15

22 4 Media Needed per year to fully backup 10 TB of data, growing at 200 GB/month Media Media Needed Changed or New Data Old/Unchanged Data 0 Year Mar. 31 Jun. 30 Sep. 30 Dec. 31 End Keep One Full Backup / Quarter Figure 8 Media Used for Full Backup Doing the math. Tapes we need to keep for long-term recovery per year: We need to keep one full backup per year = 200 tapes We need to keep one full backup per quarter = 800 tapes We need to add media for data growth at 50 GB/week = 120 tapes Number of 50 GB tapes needed per year: = 1120 tapes At the end of the year, the customer has a total 12.4 TB of data. Storing 12.4 TB of data natively on 50 GB tapes would only occupy 248 tapes. The amount of media needed in this simple example is more than four times what would be needed to protect the data. Most companies cannot implement such a simple backup scenario. Most need a strategy with much more data protection and many more restore points. Adding incremental backup and more restore points to a full backup strategy drastically increases the amount of media needed. In the addition to the simple backup example above, the customer s disaster recovery plan requires keeping one incremental backup each day for one week until the full backup is performed. 16 Improving Backup and Recovery Strategies

23 4 In addition, the customer needs to keep three weeks of full backup data, and one full backup for each month for the last two months. Starting with 10 TB of data on January 1, and growing at 50 GB a week, at the end of the year, the customer needs to following tapes: Annual (248) Actual Data (248) Year End (200) Incremental (10) Year End Incremental 3 Weeks 2Month 3Quarters Annual 3 Quarters (672) 2Months (484) 3 Weeks (726) Data () = Tapes Figure 9 Excessive Media Usage for Full Backup A total of 2295 tapes needed at end of year to store 12.4 TB of backup data Total amount of data on 2295 tapes is actually TB Each file is stored 9.25 times throughout the tapes Last annual full backup of 10 TB= 200 tapes Incremental last week = 10 tapes Weekly for last three weeks = 726 tapes Monthly for last two months = 484 tapes Quarterly for last three quarters = 627 tapes Newannualfullbackupof12.4TB = 248 tapes Tapes needed at year end = 2295 tapes For the first year, the site needs to have roughly 2,300 tapes at 50 GB capacity in order to guarantee data recovery at weekly, monthly, quarterly, and annual intervals. Mid-weekly recovery can be done with incremental backup. The additional 4 media required can be re-used each week after the full backup has been completed. Improving Backup and Recovery Strategies 17

24 4 In addition, every file that exists at the beginning of the year, even if it has not changed, is backed up each time a full backup is initiated. In the example, a file that exists at the beginning of the year would have ten copies on ten different backupsets(3perweek,3permonth,3perquarter,oneperyear).thiswastes valuable media space and creates unnecessary data transfers from the file system to the media. Files, that have been created recently would only have one copy and are vulnerable to data loss should the backup media break or if it is corrupted. So, why is it necessary to have a complete backup performed on the weekend, if there is already an incremental backup in place along with a record of all subsequent changes? This is because restoring a file system from a full backup and the incremental backups is time consuming and difficult. First, all tapes from the last full backup need to be restored. Now, every incremental backup needs to be restored in sequence before users are allowed access to their data. The restore time depends on the number of incremental backups to be restored. If the time window between full backups is increased, that is, more incremental backups are collected, the problem gets worse. The traditional cycle of daily incremental and weekly full backups was developed as a way of balancing backup windows versus convenience of restores. 18 Improving Backup and Recovery Strategies

25 EnhancingBackupStrategieswith thesun SAM-FSFile System 5 The Sun StorEdge SAM-FS File System Easily Keeps Up with Data Growth The Sun SAM-FS File System continuously, automatically, and transparently makes copies of new or changed files to tape or other secondary media. There is no need to stop access to the file system, or files, while the Sun StorEdge SAM-FS File System copies files, and no backup window is required. The Sun StorEdge SAM-FS File System can make multiple copies of the files simultaneously to multiple locations, including remote sites. Unlike a backup solution, which copies all files again and again, whether they are changed or not, the Sun StorEdge SAM-FS File System copies only changed or new files. This dramatically decreases the amount of data copied by the Sun StorEdge SAM-FS File System and uses less media much more efficiently. Utilizing the segmentation feature of the Sun StorEdge SAM-FS File System for large files can further decrease the amount of data to be copied. ments or partsoffilescanbeprotectedindividuallybyonlycopyingthechanged segments of a large file and not the entire file. After the Sun StorEdge SAM-FS File System has made a copy (or copies) of a file or segment, information about media type, tape name (VSN), and physical location of the file or segment on removable media is stored in the inode (metadata) of each file and segment. The Sun StorEdge SAM-FS File System can optionally free up disk space for dormant files or segments, which have not been used or accessed in a long time, managing file systems and expensive disk space much more efficiently. Files or file segments, which have been freed up from the Sun StorEdge SAM-FS File System and are near-line on tape, are automatically restored quickly from tape to the file system when accessed without having to search through large databases. This provides for a much faster and easier file protection and restoration. Improving Backup and Recovery Strategies 19

26 5 Document.doc Foo Picture.jpg = Online File = Near-Line File =RestoringFile Copy 1 A00001 Sun SAM-FS File System copies file to tapes Application Writes File Document.doc Copy 2 B00001 Copy 3 C00001 Report.txt Foo Joe Application Reads File Picture.jpg Document.doc Picture.jpg Bob Sun SAM-FS File System stages file from tape A00001 Copy 1 Figure 10 Sun StorEdge SAM-FS File System Continuous Backup Protecting Data for Easy Recovery without a Full Backup As discussed previously, the Sun StorEdge SAM-FS File System continuously protects files by making copies of only new and changed files to media on local or remote tape and magneto optical libraries. The Sun StorEdge SAM-FS File System can automatically make up to four copies of each file to different types of media. The number of copies of files or file groups (archive sets) and media to be used can be specified by powerful policies. After a file has been successfully copied to media, all information of the file copy is entered into the inode of the file. The file copy information includes the tape volume serial number (VSN) of the media used, block location of the file on media, and the media type used. The copy information on the media can be displayed with the sls command provided by the Sun StorEdge SAM-FS File System. An example of the sls command follows: # sls -D alvin/mandril alvin/mandril mode: -rw-r----- links: 1 owner: emu group: sam length: inode: 1678 archdone; copy1: ---- Dec 10 14:25 d2e.b mo OPT000 copy2: ---- Dec 10 14:36 1ff74.159fb lt DLT003 copy3: ---- Dec 10 14:36 378fe7.334e lt DLT289 copy4: ---- Dec 10 14: ab.f45 lt DLT Improving Backup and Recovery Strategies

27 5 As shown by the output of the sls command, the file alvin/mandril has four copies. Copy 1 is located on a magneto optical platter (mo) with the volume serial number OPT000 and is located at physical location d2e.b. Copy2islocated on a DLT tape (lt) with the volume serial number DLT003 at location 1ff74.159fb. Copies 3 and 4 are located on additional DLT (lt) media as indicated in the above inode information. With current backup technologies, the file alvin/mandril would be copied again and again at each full backup cycle even if it has not changed. With the SAM-FS File System, the file never needs to be copied again, because it is protected by the copies made to different media. Should the file alvin/mandril change after it has been copied, the Sun StorEdge SAM-FS File System will automatically make new copies to new media or new locations, protecting the new version of this file again. After the new copies are made successfully, the inode of the file is updated to reflect the new location and media of these new copies. The inode information of all files in the Sun StorEdge SAM-FS File System can be saved using the samfsdump utility, which is provided by the Sun StorEdge SAM-FS File System software. The samfsdump utility stores only the inode information of files, not the data portion, into a single file image and represents a snapshot of the file system. Unlike backup, there is no time window required to run the samfsdump utility. It can be run at any time, even while data is accessed in the file system. The inode information stored in the samfsdump snapshot file together with the media used for the copies represents the full backup of the Sun StorEdge SAM-FS File System. This samfsdump snapshot file together with the media copies can now easily be used to reconstruct the directory structure and file names of a Sun StorEdge SAM-FS File System in case of a disaster and provides transparent access to the actual file data from near-line media. Protecting and Restoring Multiple File Versions or Deleted Files A common strategy used with the Sun StorEdge SAM-FS File System is to take a samfsdump once a day, usually at midnight. The created samfsdump image file contains only meta-data (inodes) and represents a snapshot of the file system. Each inode in the Sun StorEdge SAM-FS File System is 512 bytes in size. Because there is no user data to be dumped, the samfsdump snapshot file is very small. For a 10 TB file system with one million files, the samfsdump snapshot is approximately 500 MB in size. Each night, a new file system snapshot is created and stored. Improving Backup and Recovery Strategies 21

28 5 If files are deleted or have changed during the day, the new nightly samfsdump snapshot will indicate the changes in the file system. The old samfsdump snapshot, for example from yesterday, contains the old location of the files on media. Information stored in the samfsdump snapshot from Monday night would look like this: alvin/mandril mode: -rw-r----- links: 1 owner: emu group: sam length: inode: 1678archdone; copy1: ---- Dec 11 14:36 1ff74.159fb lt DLT003 If a file was deleted on Tuesday, the samfsdump on Tuesday night will no longer contain the inode information for that file. This deleted file can be restored by using the snapshot from Monday night. If the file is modified on Tuesday, the samfsdump snapshot from Tuesday night will contain the new copy information. alvin/mandril mode: -rw-r----- links: 1 owner: emu group: sam length: inode: 1678archdone; copy1: ---- Dec 12 17:29 3bffe4.234f lt DLT004 By selecting a snapshot file from a specific date, the inode of a file from the selected data can be restored. When this file is now accessed, the data portion of this file will be restored from tape based on the inode information. If one daily snapshot is not sufficient but more restore points or versions are needed, customers can run the samfsdump utilitymorefrequentlyduringtheday. In addition, the copy information is stored in a log file, which can also be used to recover older file versions or deleted file. Recover Largest File Systems in a Fraction of Time Large servers in an enterprise, which can also act as large file servers, usually have multiple terabyte file systems attached. The Sun StorEdge SAM-FS File System drastically simplifies the backup process by hosting all data in single, or multiple, Sun StorEdge SAM-FS File Systems. The inherent functionality of the Sun StorEdge SAM-FS File System, automatically and transparently making multiple copies of new and changed files, eliminates the need for further backup of these files and solves the problem of media malfunctioning. In addition, the Sun StorEdge SAM-FS File System can write copies of files to remote locations via a TCP/IP network, solving the vaulting and potential data loss problem in case of a local fire or catastrophe. 22 Improving Backup and Recovery Strategies

29 5 Sun SAM-FS foo File System foo Copy 4 Copy 3 TCP/IP Sun SAM-FS File System Copy 2 foo foo Copy 1 foo Figure 11 Multiple Copies with the Sun StorEdge SAM-FS File System In case of a disaster or total data loss, the Sun StorEdge SAM-FS File System can restore the structure of entire file systems in minutes, regardless of the amount of data in the file system. The Sun StorEdge SAM-FS File System restores file systems using a multi-phase approach. Phase One From the samfsdump image file, the Sun StorEdge SAM-FS File System restores the entire directory structure, including all directories and all file names in the file system to the state of the file system when the samfsdump was taken. This process is extremely fast and restores the directory and file structure at approximately 100,000 inodes per minute. After the file system structure is restored, the data portion of each file remains on tape and all files in the file system are marked near-line. As soon as the directory and file structure is restored, users and applications can immediately access files in the Sun StorEdge SAM-FS File System. Improving Backup and Recovery Strategies 23

30 5 Phase Two When a near-line file is accessed by a user or application, the Sun StorEdge SAM-FS File System reads the copy information of the file, which is stored in the inode of the file, and uses this information to instruct the robotic library to mount the media required to restore the data portion of the file from the primary near-line copy (Copy 1) on the media. Once the media has been mounted into a drive and the drive has become ready, the Sun StorEdge SAM-FS File System automatically positions to the data portion of the file on the media and restores the data from the media back into the file system. As soon as the first data block is read off the media, the user or application can immediately start working with the data while the remaining part of the file is restored to disk in the background. This provides for complete transparent on-demand restore capabilities of near-line files in the Sun StorEdge SAM-FS File System. If the primary near-line tape copy is not available, the Sun StorEdge SAM-FS File System automatically and transparently restores the data portion from a secondary copy. This secondary copy can be located locally in another tape library or remotely via TCP/IP on a server hundreds of miles away. Phase Three While the Sun StorEdge SAM-FS File System restores files on demand, that is, users and applications are accessing data in the file system, the Sun StorEdge SAM-FS File System can transparently restore the data portion of all other files, not yet restored, in the background to exactly the state of the file system when the samfsdump was taken. Administrators also have the ability to restore specific data portions, for example, files in a specific directory or files from a specific user, automatically. The Sun StorEdge SAM-FS File System can use and stream all available hardware for the restore process. This assures that recovery time for even the largest file systems is minimized and optimized. During the restore process, some files may already be online and others may still be near-line. This is totally transparent to users and applications, and they can work normally during the restore process. The Sun StorEdge SAM-FS File System protects only the files in its own file system. Additional backup strategies for operating file systems, like /root or /usr, can be implemented by using common backup strategies. However, a site s backup strategy can focus on these operating system files and does not require backup for the much larger user data file systems. 24 Improving Backup and Recovery Strategies

31 5 /sam = Online Directories =OnlineFiles = Near-Line Files Figure 12 Sun StorEdge SAM-FS File System Concurrent Online and Near-line Files Sun StorEdge SAM-FS File System Scales Linearly with Available Hardware Customers can manage many Petabyte to Exabyte of data through a Sun StorEdge SAM-FS File System. As discussed previously, the Sun StorEdge SAM-FS File System needs to make copies of only the changed and new files in thefilesystem.onceafilehasbeencopiedtotape,itneverneedstobecopied again until it is modified. This allows the Sun StorEdge SAM-FS File System to use much less hardware than historical backup systems, where all data needs to be copied again and again. In addition, the Sun StorEdge SAM-FS File System was developed utilizing the multi-threading technology in the Solaris Operating Environment. Each copy or restore process is fully independent, allowing each thread to utilize system resources efficiently. This technology enables the Sun StorEdge SAM-FS File System to stream any tape technology at its rated device speeds. Improving Backup and Recovery Strategies 25

32 5 File Systems (multiple) arfind stage arcopy TCP-IP TCP-IP Remote Tape Library /sam1 /sam2 arfind stageall arcopy stage Local Tape Library /sam3 arfind arcopy Parallel, High speed searches through all mounted file systems. Thousands of files & directories scanned in minutes, not hours... SAM-Init archiver/ stageall Parallel, high performance data transfers to all selected media drives. Small files batched together for maximum tape speed.. Stage requests are sorted based on position for optimized restores. arcopy Local Magneto- stage Optical Library arcopy arcopy Figure 13 Sun StorEdge SAM-FS File System Multi-threaded Architecture Optimizing Enterprise-wide Backup Strategies with the SunStorEdgeSAM-FSFileSystem Enterprise backup is essentially different from server backup. In an enterprise, multiple client systems, often with many hundreds of clients, need to be backed up via the common LAN or TCP/IP network (Ethernet). The Sun StorEdge SAM-FS File System utilizes the current backup infrastructure already deployed throughout the enterprise. The clients and servers remain in the configuration as they were before. The only change is that the backup application is reconfigured so that it directs all backup files into a Sun StorEdge SAM-FS File System, instead of going directly to tape. 26 Improving Backup and Recovery Strategies

33 5 Backup Clients Sun SAM-FS File System Enhanced Enterprise Backup TCP/IP Many concurrent backup streams no longer depend on tape drives available Sun SAM-FS File System Sun SAM-FS and Backup Server Figure 14 Sun StorEdge SAM-FS File System Enhances Enterprise Backup The tape drives on the servers are no longer connected directly to the LAN and are now managed by a Sun StorEdge SAM-FS File System. Hundreds of client backup streams can be dumped simultaneously into the Sun StorEdge SAM-FS File System. Because these backup streams no longer are required to write directly to the tape drive, the drives are free during the collection of the backups. After a backup stream from a client is finished, the Sun StorEdge SAM-FS File System automatically writes copies of the backup file to tape. Since the files reside in the file system, the Sun StorEdge SAM-FS File System can make these copies at maximum rated device speeds. This differs from regular backup systems in which the data is written to tape via the network at much lower speeds. the Sun StorEdge SAM-FS File System maximizes tape drive utilization. In addition, the Sun StorEdge SAM-FS File System can make multiple copies of the data, providing automatic cloning capabilities. Improving Backup and Recovery Strategies 27

34 5 Multiple Backup Streams 1999 /backup 2000 Example Directory Location for Backup Files Monthly Daily Weekly Inc_Mon Inc_Tue Inc_Wed Inc_Thu Inc_Friday = Directories = Online Files = Near-Line Files Earth Mars Venus Pluto Figure 15 Sun StorEdge SAM-FS File System Enhances Tape Utilization Restoring Backup Data An additional Sun StorEdge SAM-FS File System benefit is realized after the backup files have been copied to tape. After the successful copy of these backup files to tape, the file system space of these files can be freed up to make room for other backup streams. This means that a Sun StorEdge SAM-FS File System can be smaller than the total amount of backed up data. After the Sun StorEdge SAM-FS File System makes copies of the backup files, the data portion of the file is freed from the file system. All file restore operations from the backup application assume that the backup files needed for the restore are online. The reality that many of these files may not be on disk, but on tape, is completely transparent to the backup application, that is, the backup application has no knowledge of where the files physically reside. The backup application s internal database maintains the information that maps clients to backup files in the Sun StorEdge SAM-FS File System. When a client needs to restore a deleted user file or wants to restore a previous version of a user file, the restore process simply looks at its internal database and finds the 28 Improving Backup and Recovery Strategies

35 5 backup file, which contains the requested user file. The restore operation then opens this backup file, seeks to the location of the requested user file on tape, and transfers the data of the user file directly to the requesting client or user. If the backup-file opened for restore is off-line, that is, the data portion of this file is not on the disk cache at that time, the Sun StorEdge SAM-FS File System automatically and transparently mounts the tape, positions to the physical location of the file, and restores the data portion of the backup-file back into the file system. As soon as the first block of the backup-file is read off the tape and written into the Sun StorEdge SAM-FS File System, the restore process can immediately read the requested data and deliver it back to the client. Restore Request from Earth 1999 /backup 2000 Example Directory Location for Backup Files Monthly Daily Weekly Inc_Mon Inc_Tue Inc_Wed Inc_Thu Inc_Friday = Directories = Restoring File = Near-Line Files Earth Mars Venus Figure 16 Sun StorEdge SAM-FS File System Enhances Restore Efficient Media Usage and Limited Cost Because the Sun StorEdge SAM-FS File System needs to copy only new and changed files, only the tape space for these files is needed. The Sun StorEdge SAM-FS File System can easily keep up with explosion of data in today s economy by never having to do a full backup of all old and new data. Not only does the Sun StorEdge SAM-FS File System save cost by not having to add Improving Backup and Recovery Strategies 29

36 5 media for old and stale data, it also saves valuable time and money for limiting administration overhead by automating the data protection and recovery process. Efficient Media Usage with Sun StorEdge SAM-FS File System: Year End Figure 17 Media needed to fully protect 10 TB of data, growing at 200 GB/month Mar. 31 Media Count Example: Jun. 30 Sep. 30 Dec. 31 Continuous Data Protection Total year Sun StorEdge SAM-FS File System Optimizes Media Usage Media Change or New Old Data With the Sun StorEdge SAM-FS File System, the following media is used to protect 10 TB of data. The media used in this example has 50 GB capacity per tape. For 10 TB at the beginning of the year we will need = 200 tapes For 200 GB data growth per month we need (4 tapes/month x 12) = 48 tapes Tapes needed: = 248 tapes A total of 248 tapes needed at end of year to store 12.4 TB of backup data Total amount of data on 248 tapes is actually 12.4 TB Each file is stored 1 time throughout the tapes Additional copies can be made in parallel transparently to local or remote locations for disaster recovery or to account for media errors 30 Improving Backup and Recovery Strategies

37 Advanced Options in the Sun SAM-FSFileSystem for Optimized File Restoration 6 The Sun StorEdge SAM-FS File System incorporates many additional features and options, which can be used to better optimize data protection and recovery. In this paper, we want to focus and explain three additional technologies, which will increase data protection and recovery speeds, and help utilize hardware even better. The Sun StorEdge SAM-FS File System stage never Feature The Sun StorEdge SAM-FS File System stage never attribute can facilitate file restoration. If a requested file is off-line (that is, the data portion of the requested file is currently not on the disk) and the stage never attribute is set on this file, the Sun StorEdge SAM-FS File System never reads the entire data portion of this file. Instead, the Sun StorEdge SAM-FS File System calculates the seek address to a physical tape address and can quickly position directly to the requested address within this file on tape without having to read the entire file back to the disk cache. For example, using backup s restore tool, a client or user on the host earth requests to restore a single file called Picture.jpg. The restore operation determines from its database that Picture.jpg is contained inside the backup file /backup/2000/daily/inc_friday/earth_backup. The restore operation now opens.../earth_backup by submitting the following system call to the file system: open earth_backup The Sun StorEdge SAM-FS File System receives this file open and knows that the data portion of earth_backup is not on the disk. From the inode (metadata) information, the Sun StorEdge SAM-FS File System knows that the stage never Improving Backup and Recovery Strategies 31

38 6 attribute is set for earth_backup and knows the physical location of the file s copy on media. The Sun StorEdge SAM-FS File System now instructs the library to load the media containing earth_backup into a specific tape drive. The restore operation has also determined from its database that the requested user file Picture.jpg is at the physical location, for example 0x335fe68 within earth_backup. It now submits a system call to the file system to seek to location 0x335fe68 within earth_backup as follows: lseek 0x335fe68 In addition, the restore operation knows the file size, for example 10 hexadecimal blocks (0x10), of file Picture.jpg (the file to be restored) and submits a read system call to the file system as follows: read 0x10 The Sun StorEdge SAM-FS File System now waits for the tape drive to become ready and stalls the restore operation. Once the drive has become ready and because of the stage never attribute, Sun StorEdge SAM-FS File System can quickly locate directly to requested address 0x335fe68 within file earth_backup and start reading exactly 0x10 data blocks, the requested size from the restore operation. After the Sun StorEdge SAM-FS File System has read the 0x10 blocks, it can stop the tape and wait for further seeks. If no other requests are pending for this media, the Sun StorEdge SAM-FS File System can now unload the media and make room for other media load requests. 32 Improving Backup and Recovery Strategies

39 6 Example: Actual data is nearline on tape never stage attribute set Requested data is transferred directly from tape to application 3. read 0x10 blocks 1. open earth_backup Fast Locate Stop Read Stop Beginning of file, Address is 0x0 Tape Motion Requested Address is: 2. lseek 0x335fe68 Figure 18 Sun StorEdge SAM-FS File System Optimized File Access The benefits of using the stage never feature of the Sun StorEdge SAM-FS File System are that restore operations of single files are much improved and that no additional file system space is required. The Sun StorEdge SAM-FS File System does not need to read the entire backup file earh_backup back into the file system. The Sun StorEdge SAM-FS File System can fast locate to the location of the requested restore file Picture.jpg, which is inside earh_backup, and start reading only Picture.jpg. This saves valuable time, because we only need to read a small portion of the rather large backup file earh_backup. In addition, no file system spaceisusedfortherestore,becausethedataforthefilepicture.jpg is transferred directly to the requesting restore process. UsingmentationTechnologyoftheSunStorEdgeSAM-FSFileSystem Because backup files can be extremely large (multiple Gigabytes to Terabytes), it can take a long time to read or write a single large file to a single tape drive. The Sun StorEdge SAM-FS File System segmentation can break up large files into smaller file segments transparently. The segment size to be used to break up a large file can be determined by the user. Individual segments of a single file can Improving Backup and Recovery Strategies 33