Global Headquarters: 5 Speen Street Framingham, MA USA P F

Transcription

1 Global Headquarters: 5 Speen Street Framingham, MA USA P F WHITE PAPER VERITAS Bare Metal Restore: Automating Recovery from Server Failure Sponsored by: VERITAS Bill North June 2003 Eric Sheppard EXECUTIVE SUMMARY VERITAS Bare Metal Restore (BMR) extends the functionality of the company's NetBackup product by providing users with the ability to capture and restore system operating environments simply and automatically. Restoring system operating environments can be complex and can add to the time spent bringing an application back online. Bare Metal Restore automates the recovery process for both Windows and Unix environments. VERITAS Software commissioned IDC to survey user experiences with system failure. One hundred IT professionals directly responsible for recovery planning were interviewed. Findings from the VERITAS Server Restore Survey confirm the challenges that users face when restoring operating systems manually. All respondents had responded to server failures. On average, 17 person-hours were spent bringing systems back into operation. In most cases, more than one IT specialist was required. VERITAS Bare Metal Restore is able to re-create a server operating environment in minutes using a highly automated, two-step process. In the first step, the user initiates the recovery process for the appropriate BMR client via the BMR administrative GUI or command line. In the second step, the user boots the client either from local media or over the network. BMR will then re-create the server's operating environment and, if desired, also restore user data. IDC believes that Bare Metal Restore represents a logical extension to VERITAS NetBackup by addressing an increasingly important need to reliably restore not just application data, but also the complex operating environments upon which applications depend. SYSTEM RECOVERY RECONSIDERED Traditional backup and recovery methods often focus exclusively on the protection of valuable corporate data assets. These assets typically are stored redundantly on different media and at different geographical locations to ensure that replicas of critical information are available and accessible in case primary systems fail. In the event of a disaster that disables one or more primary systems, copies of key datasets can be reloaded for processing on alternative systems -- or can they? VERITAS SERVER RESTORE SURVEY In late 2002, VERITAS Software commissioned IDC to investigate the user experiences restoring servers following system failures. One hundred IT professionals were interviewed, all of whom were responsible for recovery planning and had direct experience bringing failed systems back online.

2 The sample was stratified by company size, with 23 representatives from small companies (less than 100 employees), 41 from midsize companies (100 to 999 employees), and 36 from large companies (greater than 1,000 employees). Microsoft Windows and versions of Unix (including Linux) were the most common server operating environments. Of the respondents, 85% reported use of Windows and 20% reported use of a version of Unix. Survey participants spent approximately a half-hour discussing server-recovery issues with IDC researchers. Results of the VERITAS Server Restore Survey provide an empirical foundation for IDC's discussion of system recovery reconsidered. OPERATING ENVIRONMENTS Before information access can be restored following an application or infrastructure server failure, the operating environment that the server depends upon must be fully recovered. The operating environment consists of the operating system, all necessary utilities and application software, as well as system configuration files, user directories, security systems, and network connections. Software versions, patches, and upgrades must also be restored and add even more variables to the operating environment. Recreating a working server platform for application processing has become an increasingly demanding task. Operating environments can be complex and volatile. They contain an intricate mix of both relatively permanent and variable information. Reloading the operating system from source media is an example of recovering information that makes up part of the relatively permanent portion of the operating environment. Recreating user directories and network connections are examples of recovering more volatile, user configurable information. The complexity grows as different versions, upgrades, service packages, and customizations are considered. VERITAS Server Restore Survey: Person-Hours to Recover IDC's survey asked respondents how many person-hours were spent recovering from the last server failure. The average was 17 person-hours, with 15% of respondents indicating that over 24 person-hours were expended. Larger companies reported more time spent (average 23 hours), while smaller companies reported an average of 9 person-hours needed to recover. 2

3 Recreating an operating environment "by hand" takes time and a broad collection of skills. Network, database, operating system, and application software expertise will be needed, as will accurate, up-to-date records of system configuration. Source media for components to be recovered must be located, loaded, and tested. The correct combination of patches and updates needs to be applied in the right sequence. Even when copies of enterprise data are immediately available, bringing a server to the correct operating state to restore data access can require a great deal of human intervention and may take days, not hours, to accomplish. AUTOMATED SYSTEM RECOVERY Automated system recovery depends on two complementary tasks: the ability to capture and store all necessary operating environment components and variables and the ability to restore or recreate those components and variables quickly and accurately on an available server. Each server has a potentially unique configuration, and records of the configuration must be up to date. While system environment variables do not form large datasets, they can and do exceed the capabilities of most ad-hoc management techniques. VERITAS Server Restore Survey: Use of Tools IDC survey results found that 84% of the respondents use manual recovery methods. As shown in figure, this percentage was approximately the same for small and midsize companies and slightly lower for larger companies. Manual methods are the dominant ways that system recovery is currently handled. Manual capture of system environments has been accomplished with tools such as spreadsheets, notebooks, and installation guides. System administrators often collect bookshelves of source media for operating systems, applications, and their upgrades. Perhaps a snapshot of user directories and the authorization database have been stored on the operator's workstation. Experience shows, however, that these ad-hoc methods do not reliably collect and update all necessary configuration information on a reliable, routine basis. Capturing the server operating environment can be a regularly scheduled process that, like dataset backup, stores a snapshot of the system environment often enough to assure that the system can be restored to its most recent operational state. Alternately, the capture of the operating environment can be undertaken before 3

4 VERITAS Server Restore Survey: Time to Recover IDC survey results found that median expected downtimes for five scenarios range from as little as an hour following a power outage to as much as two days following the loss of 20 servers. The graph shows the range of estimates in days for each of five scenarios. Legend 5 Days 90% 75% 4 Days 3 Days Median 25% 10% 2 Days 1 Day Power Outage Network Failure Storage System Failure Complete Server Failure Failure of 20 Servers episodes of potential risk. For example, storing the operating environment immediately prior to installing software upgrades will provide operators with the option of rolling the operating environment back should the upgrade process go astray. The capture process must accommodate today's typical datacenter, with many servers potentially running on different operating systems, each uniquely configured to support different application and user requirements. In some cases, recovery may simply mean rolling back to an earlier operating environment on a single server. In more extreme cases, however, datacenter staff may face the task of recreating tens of servers, each with its own unique operating environment. The requirements for automated system recovery mirror the capture functions. System administrators need to be able to locate a particular server's operating environment and reinstall that environment on a replacement server. Once this is completed, normal restore procedures can be used to recover all remaining files from the standard backup server. Moreover, when multiple server failures occur, administrators need the tools that will allow a new collection of servers to be reconfigured quickly and accurately. VERITAS BARE METAL RESTORE VERITAS Bare Metal Restore extends the functionality of VERITAS NetBackup and provides users with the ability to capture and restore system operating environments simply and automatically. VERITAS NetBackup provides backup and recovery services for both application and system data, while VERITAS Bare Metal Restore accelerates server recovery by automating the entire system recovery process. By accelerating the system recovery process, Bare Metal Restore improves system availability for NetBackup users. 4

5 OVERVIEW VERITAS Bare Metal Restore works together with VERITAS NetBackup DataCenterTM or VERITAS NetBackup BusinesServerTM. After BMR is installed, clients are backed up to the NetBackup server as before, but an additional procedure called "bmrsaveconfig" is automatically executed before every scheduled backup to record the current state of the system's configuration, including the disk layouts and TCP/IP configuration. If changes are made to a system's configuration, these changes are automatically captured and recorded at the next scheduled backup without any user intervention. In addition to the existing NetBackup server, BMR components include:! BMR client code.! BMR server components: the BMR main, file, and boot servers. The BMR main server manages the clients supported by the Bare Metal Restore system, the process of preparing for the client restoration, and the post-processing after the client has been restored. It makes the appropriate recovery resources available to the client and creates a customized recovery procedure. The BMR boot server uses standard protocols to network-boot the BMR client. The BMR file server provides the client with the necessary operating system programs and libraries, the Bare Metal Restore client package, the NetBackup client package and any other software necessary to recover the system (e.g., VERITAS Volume Manager ). Collectively, these BMR file server resources are called the Shared Resource Tree (SRT). VERITAS Server Restore Survey: Skill Sets Required IDC's survey asked respondents how many IT professionals were needed to restore the last server failure at their organization. IDC found that as many as six different individuals representing different technology skill sets were required. While 35% of respondents reported that a single individual restored the server, most respondents reported that two or more technicians were required. The trend for more individuals to be involved was more pronounced for larger companies. 5

6 FIGURE 1 (a) BARE METAL RESTORE PREPARE TO RESTORE (b) Source: VERITAS,

7 DAILY OPERATIONS NetBackup's policy engine allows system managers to schedule regular backups. Bare Metal Restore automatically captures operating environment metadata whenever a NetBackup full or incremental backup initiates. In addition to regularly scheduled backups, system operators can request that Bare Metal Restore capture server metadata at any time. System operators may perform this operation, a process taking just a few minutes, before undertaking an operating system upgrade, for example, or before testing a new version of application software. RESTORE PROCESS Figure 1 depicts the browser-based Prepare to Restore interface. Invoking this interface and specifying parameters, such as which client is to be restored or IP networking addresses, typically is the first step towards system and data recovery with Bare Metal Restore. The entire restore process for Bare Metal Restore is highly automated, and allows NetBackup customers to completely recover their machines from normal backups, without requiring separate system backups or reinstalls. Following a severe system failure, Bare Metal Restore and NetBackup can restore the machine to the state at which it was last backed up with minimum intervention and opportunity for human error. In a few minutes after configuring the restore, Bare Metal Restore indicates that the client is ready to be restored, and an "OK" prompt is displayed (Figure 2). The operator then reboots the client, and the remainder of the recovery process is automated. When the recovery is completed the client automatically reboots, and the operating and application environments are ready to go. FIGURE 2 THE BARE METAL RESTORE CLIENT IS READY TO BE RESTORED Source: VERITAS,

8 SCOPE OF OPERATING ENVIRONMENTS VERITAS Software supports the Bare Metal Restore product in Windows and Unix environments. Both versions will be needed for most IT shops. Storage administrators who straddle Unix and Windows environments will be pleased to note that Bare Metal Restore's graphical user interface is identical for the two environments, as Figure 3 shows. By using a common interface across environments, Bare Metal Restore provides centralized system recovery management with a single product. Operators work from a single skill set, rather than needing to understand each operating environment separately. FIGURE 3 THE BARE METAL RESTORE INTERFACE IS THE SAME FOR UNIX AND WINDOWS Source: VERITAS, 2003 IDC ANALYSIS: OPPORTUNITIES AND CHALLENGES IDC believes that Bare Metal Restore represents a logical extension to NetBackup that addresses the increasingly important need to reliably restore not just an application and its data, but also the complex operating environment on which the application depends. Moreover, the ability to map and record operating environments for a heterogeneous collection of servers with a single tool makes Bare Metal Restore an important part of an enterprise business continuity plan. The high degree of automation designed into the Bare Metal Restore product is essential to its success. Backup cannot depend on the best intentions of a disciplined system operator, and recovery should be accomplished as quickly and reliably as 8

9 possible. VERITAS Bare Metal Restore has assimilated datacenter requirements well. Using NetBackup to store Bare Metal Restore's operating environment metadata along with traditional databases and files extends the functionality of the VERITAS product line. Bare Metal Restore's dependency on NetBackup does make BMR an accessory rather than a standalone solution. Customers shopping more narrowly for a third-party server restore solution may not value this dependency. On the other hand, BMR's ability to leverage NetBackup enables automation and eliminates redundant usage of storage media and system backup procedures, saving time and money. The VERITAS focus on Unix and Windows precludes the Bare Metal Restore option for companies that need a restore product for mainframe and other non-unix/windows operating systems. To be fair, however, competing products are offered by system suppliers and support rapid recovery for a particular operating environment. By supporting Unix and Windows with a single product, VERITAS Bare Metal Restore will appeal to IT departments with multivendor servers in the datacenter. CONCLUSION Findings from the VERITAS Server Restore Survey demonstrate empirically the need for IT departments to automate the ability to backup and restore server operating environments much in the manner that file systems and databases are protected. VERITAS Bare Metal Restore is aimed squarely at reducing the time to recovery, labor required, and skills required. IDC encourages IT professionals responsible for business continuity planning to evaluate the ability of NetBackup and Bare Metal Restore to accelerate the process of bringing an application back into production quickly and reliably. COPYRIGHT NOTICE External Publication of IDC Information and Data - Any IDC information that is to be used in advertising, press releases, or promotional materials requires prior written approval from the appropriate IDC Vice President or Country Manager. A draft of the proposed document should accompany any such request. IDC reserves the right to deny approval of external usage for any reason. Copyright 2003 IDC. Reproduction without written permission is completely forbidden. 9

10 IDC is a division of IDG, one of the world's top information technology media, research and exposition companies. Visit us on the Web at To view a list of IDC offices worldwide, visit IDC is a registered trademark of International Data Group.