Virtual Machine Backup Guide Virtual Infrastructure

Transcription

1 Virtual Machine Backup Guide Virtual Infrastructure

2 Please note that you will always find the most up-to-date technical documentation on our Web site at VMware, Inc Porter Drive Palo Alto, CA The VMware Web site also provides the latest product updates. Copyright VMware, Inc. All rights reserved. Protected by one or more of U.S. Patent Nos. 6,397,242, 6,496,847, 6,704,925, 6,711,672, 6,725,289, 6,735,601, 6,785,886, 6,789,156, 6,795,966, 6,880,022 6,961,941, 6,961,806 and 6,944,699; patents pending. VMware, the VMware boxes logo and design, Virtual SMP and VMotion are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions. Microsoft, Windows and Windows NT are registered trademarks of Microsoft Corporation. Linux is a registered trademark of Linus Torvalds. All other marks and names mentioned herein may be trademarks of their respective companies. Revision: Item: ESX-ENG-Q

3 Table of Contents Preface 1 About This Guide 2 Intended Audience 2 Document History 2 Related Documentation 3 Conventions 4 Abbreviations 4 Technical Support Resources 5 Self-Service Support 5 Online and Telephone Support 5 Support Offerings 5 Introduction 7 Why You Should Back Up Your Data 8 Preventing User Errors 8 Preventing Administrative Errors 8 Disk Structure of ESX Server System 10 Virtual Machine Disks 10 Snapshots 10 Disk Files and Redo Logs 11 Raw Device Mappings 11 Accessing and Managing Virtual Disk Files 12 What Backup Can Be Run Where 13 Traditional Versus Consolidated Backup 14 Quiescing the File System 14 Quiescing Applications 14 Backing up the Service Console 14 Backing up Virtual Machines 15 VMware Consolidated Backup 19 Overview 20 How Consolidated Backup Works 21 Integration with Third-Party Backup Software 21 Requirements 21 VMware Virtual Machine Snapshot Technology 22 iii

4 Setting up Consolidated Backup 24 Overview 24 VMware ESX Server and VirtualCenter Configuration 24 SAN Configuration 24 Configuring the Third-Party Software 25 Backup Proxy Configuration 25 Configuring Windows on the Backup Proxy 26 Configuring Virtual Machines for Consolidated Backup 29 Overview 29 Installing VMware Tools 29 Installing Third-Party Backup Software Agents 29 Using Consolidated Backup 31 Suggested Best Practice 31 Aliases 31 Configuring Backup Jobs 31 First Time Backup 32 Advanced Configuration 33 Quiescing 33 Capacity Planning 33 Canceling a Backup Job 36 Restoration and Disaster Recovery 37 Restoring Your Files Using Consolidated Backup 38 Restore 38 Restoring Your Files Using vcbrestore 40 Data Recovery 42 Troubleshooting 43 Backup Scenarios 45 Backup Usage Scenarios 46 A Typical Consolidated Backup Usage Scenario 46 Consolidated Backup for Full Virtual Machines 47 iv

5 Preface This preface describes the contents of this manual, lists related documentation, describes document conventions, and provides additional references for support. This preface contains the following topics: About This Guide on page 2 Related Documentation on page 3 Conventions on page 4 Technical Support Resources on page 5 1

6 Virtual Machine Backup Guide About This Guide This manual, the Virtual Machine Backup Guide, provides information for backing up your data. Intended Audience The information presented in this manual is written for system administrators who are experienced Windows or Linux system administrators and who are familiar with virtual machine technology and datacenter operations. Document History This is the Virtual Machine Backup Guide, Revision , Item No. ESX-ENG-Q This manual is revised with each release of the product or when deemed necessary. A revised version can contain minor or major changes. Release Date Description Release 3.0, Beta 2 February 16, 2006 PDF on Web 2

7 Preface Related Documentation ESX Server 3.0 and VirtualCenter 2.0 documentation consists of the following publications: Documentation Roadmap Introduction to Virtual Infrastructure Installation and Upgrade Guide Virtual Infrastructure User s Guide Virtual Machine Management Guide Virtual Machine Backup Guide SAN Configuration Guide Server Configuration Guide Web Access Administrator s Guide Resource Management Guide Setup for Microsoft Cluster Service Online Help Hardware Compatibility Guides I/O Compatibility Guide SAN Compatibility Guide Systems Compatibility Guide Backup Software Compatibility Guide 3

8 Virtual Machine Backup Guide Conventions This manual uses the following conventions. Style blue (online only) Courier Semi-Bold Bold Italic italic Purpose Cross references, links Commands, filenames, directories, paths, user input Interactive interface objects, keys, buttons Items of highlighted interest, terms Variables, parameters, emphasis in text Web addresses Abbreviations The graphics in this manual use the following abbreviations. Abbreviation VC VI server database disk-n.vmdk hostn VM# user#.vmdk datastore SAN tmplt Description VirtualCenter Virtual Infrastructure <product name> server <product name> database Redo log files <product name> managed hosts Virtual machines on a managed host Users with access permissions Storage disk for the managed host Storage for the managed host Storage area network type datastore shared between managed hosts Template 4

9 Preface Technical Support Resources The following sections describe various technical support resources available to you. Self-Service Support Online and Telephone Support Support Offerings Self-Service Support Use the VMware Technology Network for self help tools and technical information: Product Information Technology Information Documentation Knowledge Base Discussion Forums User Groups For more information about the VMware Technology Network, go to Online and Telephone Support Use online support to submit technical support requests, view your product and contract information, and register your products. Go to Use phone support for the fastest response on priority 1 issues for customers with appropriate support contracts. Go to Support Offerings Find out how VMware's support offerings can help you meet your business needs. Go to 5

10 Virtual Machine Backup Guide 6

11 CHAPTER1 Introduction Backup, restoration, and disaster recovery are among the most crucial elements of datacenter management. More and more critical information is stored in virtual machine disk images. This information should be backed up periodically to prevent data loss. This chapter includes the following information: Why You Should Back Up Your Data on page 9 Disk Structure of ESX Server System on page 11 What Backup Can Be Run Where on page 14 Traditional Versus Consolidated Backup on page 15 7

12 Virtual Machine Backup Guide Why You Should Back Up Your Data People back up their data for the simplest of reasons: they don t want to lose it. To an individual, loss of data can mean loss of personal memories; to a corporation, loss of data can mean the loss of money, jobs, and business, often with catastrophic consequences. Backing up your data means that you are protecting it from the following types of errors: User error, such as accidental file deletion. See Preventing User Errors. Administrative error, such as removing the wrong machine. See Preventing Administrative Errors. Hardware error, such as disk failure Some additional issues are the load on ESX Server, the load on the LAN, and no SAN tape library support. Preventing User Errors To prevent user errors, such as accidental file deletion, perform file-level backups in the virtual machine. You can treat virtual machines like physical machines by deploying backup agents in each virtual machine, running nightly backups, and moving data across the LAN to a tape server. There are some issues with file-level backup: An agent needs to be deployed in each virtual machine. This method of backup is resource intensive because of the heavy network load and because it uses valuable CPU and I/O cycles on production ESX Server systems. This method also has a backup impact. Preventing Administrative Errors To prevent administrative errors, such as removing the wrong machine, perform full virtual machine backups. The full virtual machine backups are snapshots, including disk files and configuration files, taken in two ways: VMFS snapshots Hardware-assisted snapshots (SAN) There are three issues with full virtual machine backup: 8

13 CHAPTER 1 Introduction Virtual machine export is slow. When it is done from within the service console, it slows down ESX Server as a whole. VMFS is optimized for virtual machine performance but not for access through the service console. Snapshots are only crash consistent for powered-on virtual machines. Crashconsistent snapshots look as though the virtual machine had crashed at the point in time when the snapshot was taken. 9

14 Virtual Machine Backup Guide Disk Structure of ESX Server System ESX Server uses VMware ESX Server File System (VMFS) for its storage needs. VMFS is a simple, high-performance file system on physical SCSI disks and partitions. VMFS is used for storing large files, such as the virtual disk images for ESX Server virtual machines and the memory images of suspended virtual machines. ESX Server 3 supports VMFS-3. A VMFS-3 volume can be a logical grouping of disk partitions, which are also called physical extents. Virtual Machine Disks In ESX Server 3, VMFS supports directories. Typically, there is one directory for each virtual machine on VMFS. This directory contains all the files that comprise the virtual machine, such as disk images,.vmx files, log files, and so on. The disk files are in a special format and generally use the.vmdk file extension. Snapshots Snapshot software allows you to make instantaneous copies of any virtual disk within the disk subsystem. Third-party backup software can be used for backup and disaster recovery. ESX Server 3 includes VMware Consolidated Backup (Consolidated Backup), a Windows solution for taking file- and image-level snapshots of your virtual machines. For more information on Consolidated Backup, see VMware Consolidated Backup on page 19. VMware offers a file system consistent backup. In most cases, a file system consistent backup allows you to completely recover from failure. If, however, you use applications with potential integration issues beyond the file system level, such as a database, the VMware solution may not offer enough for you. In that case, investigate a third-party backup solution to see whether it better suits your needs. File-based (VMware) Solution When considering a file-based solution using VMware Tools and VMFS, be aware of the following points: Using VMware tools and VMFS is better for provisioning: one large LUN is allocated and multiple.vmdk files can be placed on that LUN. With RDM, a new LUN is required for each virtual machine. Snapshotting and replication is included with your ESX Server host at no extra cost. The file-based solution is, therefore, a very cost-effective solution. 10

15 CHAPTER 1 Introduction Disk Files and Redo Logs Disk files and redo logs make performing backups much easier. In the default state, a virtual machine disk is a single file. All changes to that disk are written directly and immediately to that.vmdk file. However, ESX Server can instead use what is known as a redo log. When a redo log is added to a.vmdk file, that base disk file becomes static and unchanging. All writes are captured in the redo log. This state is represented by file name: if the base disk is called disk.vmdk, the redo log is called basen.vmdk. The redo log contains the most recent version of all blocks that have been changed by the virtual machine in this virtual disk since the redo log was created. Raw Device Mappings Raw Device Mappings (RDM) allows you to use all of the features of VMware Virtual Infrastructure in conjunction with raw SAN LUNs. The mapping file is the file that is used to connect the raw LUN to the virtual machine and is referenced in the virtual machine s configuration. There are two modes for RDM: virtual compatibility and physical compatibility. Virtual compatibility mode allows a mapping to act exactly as a virtual disk file does, including the ESX Server snapshots. Physical compatibility mode allows direct SCSI access to the device being mapped for those applications that need lower level disk access and control. In both cases, data is stored on the LUN or SCSI device, not on the disk file. In virtual compatibility mode, an RDM file in a VMFS volume manages metadata for its mapped device. There is a one-to-one mapping between mapping files and mapped devices. The mapping file is presented to the VMware Service Console as an ordinary disk file, available for file system operations, and can have redo logs. To the virtual machine, the ESX Server presents the mapped device as a locally attached SCSI device. In physical compatibility mode, RDM provides minimal SCSI virtualization of the mapped device. In this mode, the VMkernel passes all SCSI commands to the device with one exception: the Report LUNs command is virtualized so that the VMkernel can isolate the LUN for the virtual machine that owns it. Otherwise, all physical characteristics of the underlying hardware are exposed. Physical mode is useful when you need to run SAN management agents or other SCSI target-based software in the virtual machine. Physical mode is also used for virtual-to-physical clustering for costeffective high availability. 11

16 Virtual Machine Backup Guide Accessing and Managing Virtual Disk Files Most of the time you will be managing your data through the Virtual Infrastructure (VI) Client and the higher level operations it offers, such as creating virtual machines, cloning virtual machines, and deleting virtual machines, whenever possible. Direct manipulation of your virtual disk files on VMFS is possible through ESX Server service console and VMware SDKs although using the VI Client is the preferred method. From the service console, files can be viewed and manipulated in the /vmfs/volumes directory in mounted VMFS volumes with ordinary file commands, such as ls and cp. Although mounted VMFS volumes might appear similar to any other file system, such as ext3, VMFS is primarily intended to store large files, such as disk images. The ftp, scp, and cp commands can be used for copying files to and from a VMFS volume as long as the host file system supports these large files. Note: In Linux, importing a large disk from a Common Internet File System (CIFS) mount hangs the ESX Server, which must then be rebooted. If Consolidated Backup is not used, large files need to be moved from the service console to tape backup. To do this effectively, use supported programs with no reported known issues. As a workaround, use smbclient to copy the large file onto a local directory on the service console and then import from there. Additional file operations are enabled through the vmkfstools command. This command supports the creation of a VMFS on a SCSI disk and is used for the following: Creating, extending, and deleting disk images Importing, exporting, and renaming disk images Setting and querying properties of disk images Creating and extending a VMFS file system For more information on the vmkfstools command, see the Server Configuration Guide. 12

17 CHAPTER 1 Introduction What Backup Can Be Run Where The three components of backup software are as follows: Backup Client Media Server Scheduler Each component can be run in a virtual machine, on the service console, or on a physical machine (proxy) running Microsoft Windows While the location of the scheduler isn t important, the locations of media server and backup client are important. Deciding where to run each component enables certain types of backup (file-based versus image-based) with different trade-offs. 13

18 Virtual Machine Backup Guide Traditional Versus Consolidated Backup Traditional backup methods (backup agents in virtual machines) are slower, take a longer time, and have some performance degradation as many backup clients may be running on the ESX Server at once. Consolidated Backup is faster, shorter, and snapshot mode causes less degradation. Quiescing the File System Traditional backup methods (backup agents in virtual machines) require that the virtual machines are suspended and then the whole image is backed up for a quiesced file system; hence, this is rarely done. This takes a long time. Consolidated Backup is a short 1 to 2 second interruption similar to VMotion. Quiescing Applications Traditional backup methods (backup agents in virtual machines) cause a long slowdown during the entire backup process. Consolidated Backup is faster, shorter, and causes less degradation. In Consolidated Backup, quiescing applications is like taking a SAN snapshot. There is a slowdown or interrupt during snapshot creation, but not for the duration of the backup. Backing up the Service Console The VMware Service Console starts up and administers your virtual machines. It is a customized version of Linux based on the RHEL 3 distribution. It has been modified so that it can be managed by the VMkernel. The service console is scheduled by the VMkernel just as any virtual machine is. You should not attempt to run heavy workloads on the service console because it can take processor cycles away from your virtual machines. The state of your service console can be recovered without much trouble; however, having it safely backed up can save you time in a restore situation. You may not find it necessary to back up the service console as frequently as the virtual machines and their associated data. There are three approaches to backing up the service console: 1. Nothing Since not much state is stored in the service console, reinstalling ESX Server after a disaster makes more sense than restoring it from scratch. 2. File-Based 14

19 CHAPTER 1 Introduction The service console can be treated as a physical machine with a deployed backup agent. To restore the service console, first reinstall it, then reinstall the agent, and then restore the files that you backed up. This approach makes sense if management agents that are hard to set up have been deployed in the service console; otherwise, this approach provides no advantage over not backing up the service console. 3. Image-Based Use third-party software to create a backup image that you can restore quickly. Use your boot CD or whatever the backup software created to restore the service console. Backing up Virtual Machines Options for backing up your virtual machines are discussed in the following section. Table 1-1 on page 16 is an overview of the recommended traditional backup options. Table 1-1 Recommended Traditional Backup Types Media Server Virtual Machine Separate Physical Machine (not Consolidated Backup) Backup Client Virtual Machine Option 1 Option 2 Service Console Option 3 Option 4 Note: The backup server is not supported in the service console. Option 1: Media Server in a Virtual Machine The client is in the virtual machine and the media server is in the virtual machine. This option is not recommended in ESX Server except in a branch office scenario where no separate hardware for a proxy or media server is available. The backup agent does quiescing for option 1. Option 1 is file-level restore. This option has a hardware compatibility issue with SAN tape libraries. This option still deals with the usual backup impact. In addition, managing agents in every virtual machine is time consuming. Recommended: File-level restore: Full virtual machine restore: Quiescing: No Very easy No Excellent 15

20 Virtual Machine Backup Guide Load on ESX Server: LAN-free backup: Backup Impact: Manageability: Extremely high No No Very poor Option 2: Media Server in a Physical Machine This option is largely replaced by Consolidated Backup. Option 2 is the client in the virtual machine and the server in a separate physical machine. The backup agent does quiescing for option 2. Option 2 is a file-level restore. These options still deal with the usual backup impact. In addition, managing agents in every virtual machine is time consuming. Recommended: File-level restore: Full virtual machine restore: Quiescing: Load on ESX Server: LAN-free backup: Backup Impact: Manageability/Scalability: Yes Very Easy No Excellent High No No Very poor. Option 3: Media Server in a Virtual Machine The client is in the service console and the media server is in the virtual machine. This option is not recommended in ESX Server except in a branch office scenario where no separate hardware for a proxy or media server is available. The vmsnap, vmres, and other scripts are needed to do quiescing and take the snapshot. Option 3 is an image-level restore. These options have a hardware compatibility issue with SAN tape libraries. These options still deal with the usual backup impact. In addition, managing agents in every virtual machine is time consuming. Recommended: File-level restore: Full virtual machine restore: Quiescing: No No Very easy Excellent 16

21 CHAPTER 1 Introduction Load on ESX Server: LAN-free backup: Backup Impact: Manageability: Extremely high No No Very poor Option 4: Media Server in a Physical Machine This option is largely replaced by Consolidated Backup. Option 2 is the client in the virtual machine and the server in a separate physical machine. Option 4 is the client in the service console and the server in the physical machine. The vmsnap, vmres, and other scripts are needed to do quiescing and take the snapshot. Option 4 is an image-level restore. These options still deal with the usual backup impact. In addition, managing agents in every virtual machine is time consuming. Recommended: File-level restore: Full virtual machine restore: Quiescing: Load on ESX Server: LAN-free backup: Backup Impact: Manageability/Scalability: Yes No Very Easy Excellent High No No Very poor. Option 5: RDM and SAN Snapshots It is possible to create a SAN snapshot of RDM. It is possible to re-attach the snapshot to a physical server. Backups can then be performed from this physical proxy. The SAN snapshot could also be considered to be the backup if you trust your SAN (no more data movement), or you could create a remote replication of the snapshot or an image-based backup of it. For more information, see the SAN Configuration Guide. The issue here is again that this only gives crash consistent images of the virtual machines on the snapshot LUN unless your SAN does quiescing. Option 6: VMFS and SAN Snapshots This is a snapshot with a VMFS file, not a SAN snapshot. Use this option at your own risk. Call your service representative or storage vendor. 17

22 Virtual Machine Backup Guide Option 7: NFS and NDMP for Images This option is recommended for image backups of virtual machines stored on NAS. 18

23 CHAPTER2 VMware Consolidated Backup The VMware Consolidated Backup (Consolidated Backup) is the recommended way to perform daily backups for virtual machines residing on a SAN. This method performs backups using a dedicated physical host (proxy client). Backing up using a proxy rather than backing up from ESX Server itself takes the load off the ESX Server, makes your backups run faster, allows the use of fibre infrastructure, including fibreconnected tape devices, instead of forcing everything over the LAN, and eliminates the need for backup agents on each of the virtual machines. Consolidated Backup can be used with a single ESX Server host or with a VirtualCenter Management Server. This chapter includes the following information: How Consolidated Backup Works on page 21 Setting up Consolidated Backup on page 24 Configuring Virtual Machines for Consolidated Backup on page 29 Using Consolidated Backup on page 31 Advanced Configuration on page 33 19

24 Virtual Machine Backup Guide Overview VMware Consolidated Backup is a fast, efficient way of backing up data in virtual machines. Consolidated Backup reduces administrative overhead and cost, works with other industry-leading backup applications, and places no restrictions on Fibre Channel Tape usage, allowing LAN-free backup of virtual machines. Consolidated Backup is a new backup solution in ESX Server 3. Backup is offloaded to a dedicated physical host. Offloading to a dedicated physical machine solves many problems. For example, there is no resource contention on ESX Server host. Offloading also enables LAN-free backup and doesn t require backup agents in virtual machines. This results in better performance and a shorter time requirement for backup as well as lower costs and easier management. Consolidated Backup supports file-level backups for Windows guests and full virtual machine backup for all guests. Previously, to protect data stored in a virtual machine, a backup agent was typically deployed on each host or guest that contained important data. Backups were automatically conducted on a regular basis. In ESX Server 2, the backup agent scanned the file system for changes and sent the changed information across the network to a media server that wrote the data to backup media, such as a robotic tape library, during periods of low system use. The problem with this approach is that it only works well when: There is a window of low activity on the protected server during which backup can be performed or critical applications can be taken offline. Network bandwidth between the protected server and the media server is plentiful. Unfortunately, both conditions are usually not met in a modern datacenter environment. Global business does not permit services to be taken offline or slowed down at any time of day. Particularly with server consolidation, there is usually no window in time when network and computing resources are not being used so that the massive amounts of data that are being moved around during backup would not affect production tasks. Also, with an increasing number of servers (physical or virtual), the administrative overhead for managing backup software on each host can become prohibitively large. 20

25 CHAPTER 2 VMware Consolidated Backup How Consolidated Backup Works Consolidated Backup enables offloaded and impact-free backup for virtual machines running on an ESX Server system by allowing traditional file-based backup software to leverage VMware virtual machine snapshot technology and efficient SAN-based data transfer. Integration with Third-Party Backup Software In a typical Consolidated Backup scenario, backup scheduling is done by the thirdparty software. As a result, all the advanced scheduling and backup management functionality provided by the backup software is also available with Consolidated Backup. Backup jobs are configured and launched through the third-party backup software. The basic backup steps are: 1. The third-party backup software schedules the backup job, which represents a single virtual machine or a group of virtual machines. The job runs on the proxy. 2. Consolidated Backup s pre-backup script runs on the proxy, creating the file system s quiesced virtual machine snapshots (for Windows only) and mounting them on the proxy. 3. The third-party backup software performs the backup. 4. Consolidated Backup s post-backup script runs, removing the mount from the proxy and committing the backup snapshots. Consolidated Backup cannot alter any data in virtual disks on SAN; for example, Transient Writes are cached on the proxy (NT4 volumes). Requirements Microsoft Window 2003 server configured for Consolidated Backup, see Configuring the Consolidated Backup Framework on page 27. Windows virtual machines: Only required for file-level backup. Full virtual machine backup (image) is supported for all guest operating systems. SAN connected and LUN masked. Backup administrator account on the proxy with read access to the data. 21

26 Virtual Machine Backup Guide Note: The LUNs containing VMFS must be accessible by the proxy. To protect the VMFS volumes from accidental deletion, make sure that you restrict access to the proxy to trained personnel only. File system consistency is guaranteed for Windows guest operating systems only. VMware Virtual Machine Snapshot Technology VMware ESX Server system allows you to take snapshots of live virtual machines with any operating system. ESX Server 3 also guarantees file system consistency for snapshots of Windows only virtual machines. ESX Server can create file-system-consistent snapshots of Windows virtual machines that have VMware Tools installed so that no file system writes are pending at the time the snapshot of the virtual machine is taken. As a result, the snapshot represents a clean state of the virtual machine s disk images. Furthermore, system administrators can configure pre-freeze and post-thaw scripts in both Windows (application consistency) and Linux virtual machines to achieve even higher levels of data consistency. Once a snapshot of a virtual machine has been created, the frozen disk images can be accessed in different ways, as described in the remainder of this section. File-Level Backups For virtual machines running Windows, Consolidated Backup analyzes the disk images and mounts them on the proxy at predefined junction points that correspond to the 22

27 CHAPTER 2 VMware Consolidated Backup drive letter assignment for each volume in the virtual machine. This allows file-level backups of the file systems contained within the virtual disks belonging to the snapshot. In file-level backups: Volumes on virtual machine snapshots are analyzed. Discovered volumes are mounted on junctions. Junctions correspond to drive letters in the virtual machine, for example: C:\mnt\mytestvm.foo.com\letters\D The backup supports Windows LDM volumes. A file-level backup is ideal for preventing data loss due to errors, such as file corruption, caused by bugs in application programs, or user errors, such as accidental file deletion. Full Virtual Machine Backups Consolidated Backup can also export a virtual machine s disk images and configuration files to a local directory on the proxy. Third-party backup software picks up these files and moves them to the backup medium. Virtual machine disk images and configuration files exported to a local directory on the proxy Third-party backup software moves files to the backup medium Files located at: C:\mnt\mytestvm.foo.com\fullvm This type of backup is suitable for restoring entire virtual machines in the event of a hardware failure or a system administrator error, such as the accidental deletion of an entire virtual machine or a disk image. Note: Full virtual machine backups are operating-system neutral; that is, they can be performed regardless of the guest operating system. 23

28 Virtual Machine Backup Guide Setting up Consolidated Backup Overview Before you use Consolidated Backup, you should configure the following components: VMware ESX Server and VirtualCenter. See VMware ESX Server and VirtualCenter Configuration on page 24. The SAN fabric. See SAN Configuration on page 24. The third-party software. See Configuring the Third-Party Software on page 25. The backup proxy. See Backup Proxy Configuration on page 25. Windows on the backup proxy. See Configuring Windows on the Backup Proxy on page 26. VMware ESX Server and VirtualCenter Configuration You should already have an existing installation of an ESX Server and the Virtual Infrastructure (VI) Client or a multi-host configuration and VirtualCenter to manage them. SAN Configuration Before installing Consolidated Backup, configure the SAN fabric to which both the ESX Servers and the backup proxy are connected. Follow the documentation provided by your SAN storage array and switch vendors for details. See the SAN Configuration Guide for more information. Note: Before you install Consolidated Backup, complete the SAN configuration and set up your ESX Servers to use VMware file system (VMFS) or virtual compatibility raw device mappings (RDM). For Consolidated Backup, your SAN configuration must meet the following requirements: The backup proxy must have access to all the SAN arrays containing VMFS volumes with virtual disks on them. all the SAN arrays containing virtual compatibility RDMs that are supposed to be backed up using Consolidated Backup. For every LUN containing VMFS or RDM data, the LUN ID on the proxy server must match the LUN ID as seen by the ESX Server. 24

29 CHAPTER 2 VMware Consolidated Backup There is no support for EMC AX100 managed with NaviExpress. Full NaviSphere must be used. IBM ESS should be explicitly configured to have the consistency of LUN ID presentation. Usually, the SAN configuration on the storage array involves adding the backup proxy to all the host groups where the ESX Servers are located. Configuring the Third-Party Software You must configure third-party backup software for use with Consolidated Backup. This involves enabling the use of the Consolidated Backup pre-scripts and post-scripts for your backup software. Also, you might have to turn on the cross junctions (mount points) option for your backup software. For each supported third-party backup software, either the backup software vendor or VMware provides an Integration Module. This is a ZIP file containing all the required pre-backup and post-backup scripts. Also, this ZIP file contains a README.html file that describes how to use the Integration Module on the respective third-party backup software. Backup Proxy Configuration Hardware Requirements The backup proxy must be able to run Microsoft Windows Furthermore, the proxy requires the following hardware components: A network adapter (NIC) A Fibre Channel host bus adapter (HBA) Prerequisites To be able to install Consolidated Backup on the backup proxy, perform the following configuration steps: The proxy must be running Microsoft Windows Consolidated Backup doesn t support any other versions of Windows on the proxy. Networking on the backup proxy must be configured so that the Windows proxy can establish a connection to VirtualCenter or its ESX Server peer. If there is a firewall between the backup proxy and the VirtualCenter or ESX Server peer, the firewall must permit TCP/IP connections to VirtualCenter or the ESX Server peer. By default, VirtualCenter expects incoming connections at TCP/IP port

30 Virtual Machine Backup Guide The third-party backup software to be used with Consolidated Backup must be installed and correctly configured. You should verify the configuration of the third-party backup software at this time by running a backup and restoration job on a local directory on the proxy. Configuring Windows on the Backup Proxy Disabling Automatic Drive Letter Assignment All versions of Windows except for Windows 2003 Enterprise Edition and Windows 2003 Datacenter Edition automatically assign drive letters to each visible new technology file system (NTFS) and file allocation table (FAT) volume. For Consolidated Backup, you need to change this default behavior so that volumes are not automatically mounted on the proxy. Note: If you do not perform this configuration step, data corruption for virtual machines using RDM can occur. To prevent Windows from automatically assigning drive letters to RDM 1. Shut down the Windows proxy. 2. Disconnect the Windows proxy from the SAN or mask all the LUNs containing VMFS volumes or RDM for virtual machines. 3. Boot the proxy and log into an account with administrator privileges. 4. Open a command-line interface. 5. Run the diskpart utility by typing: diskpart The diskpart utility starts up and prints its own command prompt. 6. Disable automatic drive letter assignment to newly seen volumes by typing at the diskpart command prompt: automount disable 7. Clean out entries of previously mounted volumes in the registry by typing at the diskpart command prompt: automount scrub 8. Exit the diskpart utility by typing: exit 9. Shut down Windows. 26

31 CHAPTER 2 VMware Consolidated Backup 10. Reconnect the Windows proxy to the SAN or unmask all previously masked LUNs containing either VMFS volumes or RDM. 11. Boot the proxy. To install the basic Consolidated Backup framework 1. Log into the backup proxy using an account with administrator privileges. 2. Install the Consolidated Backup framework by running setup.exe from your CD ROM or electronic distribution. 3. During the installation, pick an install directory for Consolidated Backup or just accept the default one. Configuring the Consolidated Backup Framework Essential configuration for Consolidated Backup is stored in a configuration file called config.js. It is located in a subdirectory named config within the install directory for Consolidated Backup. The following table provides an overview of all the configuration settings in this file. Table 2-2 Configuration Settings Option Default Description BACKUPROOT C:\mnt Directory in which all the virtual machine backup jobs are supposed to reside. For each backup job, a directory with a unique name derived from the backup type and the virtual machine name is created here. Make sure this directory exists before attempting any virtual machine backups. Also, for full virtual machine backups, the volume containing this mount point must be large enough to hold the exported disk images of the largest virtual machine to be handled. HOST (no default) Host name/port of the VirtualCenter server or the ESX Server peer used by the proxy. PORT 902 Port number to connect to on the VirtualCenter or ESX Server peer. USERNAME (no default) User ID to be used for logging into the VirtualCenter host or ESX Server peer. PASSWORD (no default) Password to be used for logging into the VirtualCenter host or ESX Server peer. 27

32 Virtual Machine Backup Guide Option Default Description SNAPSHOT_POLICY automatic Valid options: automatic: The Consolidated Backup framework creates and deletes backup snapshots for virtual machines on demand. This is the default used most of the time. manual: The Consolidated Backup framework does not create or delete any snapshots but assumes that a backup snapshot named _VCB_BACKUP_ already exists and uses this snapshot for backup purposes. This option is useful for creative scripting. createonly: The Consolidated Backup framework creates a backup snapshot when the pre-backup script is being run, but it does not remove the snapshot after backup. This option is used if you need to run a verification job. Your verification script would then be responsible for tearing down the mount. deleteonly: The Consolidated Backup framework assumes that a backup snapshot named _VCB_BACKUP_ already exists and does not attempt to create one. However, the snapshot is deleted by the post-backup script. This option is useful for creative scripting. Installing a Backup Software Interoperability Module Finally, you must install a Consolidated Backup Interoperability module that matches your third-party backup software. For directions on how to install a particular interoperability module, please refer to the documentation provided with the interoperability module. 28

33 CHAPTER 2 VMware Consolidated Backup Configuring Virtual Machines for Consolidated Backup Overview In general, no particular configuration is required within the virtual machine to support Consolidated Backup. Installing VMware Tools You must install a new version of VMware Tools corresponding to ESX Server 3 in each protected virtual machine. Without installing VMware Tools, the snapshots that Consolidated Backup creates for backup will only be crash-consistent; that is, no file system synchronization will be performed. Installing Third-Party Backup Software Agents Depending on the backup and restoration scenarios that you are planning to support at your site, you might need to deploy a backup agent in some virtual machines. The backup agent should be supplied by your third-party backup software vendor. For details on installation of the backup agent, refer to the documentation provided with the interoperability module matching your backup software. Restrictions In general, Consolidated Backup can back up virtual machines on a backup proxy, thus offloading backup from the ESX Server and the protected virtual machine. However in some situations, Consolidated Backup cannot be used for backing up data in a virtual machine. If you encounter any of these situations, you should deploy a backup agent in the virtual machine and perform backup from within the virtual machine. You cannot use Consolidated Backup to do any of the following: Back up virtual machines with disk images stored on a storage device that the proxy cannot access. Back up virtual machines with virtual disks that are physical compatibility RDMs. Back up virtual machines that do not have an Internet protocol (IP) address or a domain name server (DNS) name associated with them. Perform a file-level backup of virtual machines running operating systems other than Windows NT 4.0, Windows 2000, Windows XP, or Windows

34 Virtual Machine Backup Guide When running file-level backup for disks in Windows virtual machines, make sure that each virtual disk partition has a drive letter assigned to it. Only partitions with drive letters can be visible to the proxy. NAS and iscsi are not supported for this release of Consolidated Backup with ESX Server

35 CHAPTER 2 VMware Consolidated Backup Using Consolidated Backup Suggested Best Practice Group virtual machines together as a single entity and manage that group as one entity in your backup software by configuring DNS aliases for the proxy. Once you have associated a group of virtual machines with one host name, you can set up a backup job for each alias using the alias as the client name for the job. Image-level backups should be performed periodically. Image-level backups are the only way to perform Consolidated Backup for Linux virtual machines. Note: There are examples in the readme files that come with your backup software. Aliases Setting up aliases means different permissions can be associated with these groups; for example, restore permissions are tied to these aliases, determining who can restore and which virtual machines can be restored from. If your department grows and you add another physical machine, aliases make it easy to move a group to a different proxy. New proxies can be added as the datacenter grows. Jobs for groups can be moved by pointing the alias to a different proxy. Configuring Backup Jobs Consolidated Backup is able to take backups of virtual machines on the backup proxy. There are several things to consider when configuring backup jobs: The jobs are assigned to the proxy. The jobs are specified by a directory set: file level: C:\mnt\mytestvm.foo.com\letters\D image level: C:\mnt\mytestvm.foo.com\fullvm The job has a time. Different products limit parallelism differently. Parallelism is defined by how many jobs are scheduled at the same time on the same proxy. For details on how to configure backup jobs for virtual machines, please refer to the documentation provided with the interoperability module for your third-party backup software. 31

36 Virtual Machine Backup Guide First Time Backup When you perform a first backup for a particular virtual machine, the virtual machine has to be powered on or backup fails on ESX Server. Using VirtualCenter mitigates this condition. Once you have completed the first backup of the virtual machine, Consolidated Backup can perform backups of the virtual machine regardless of its power state at backup time. 32

37 CHAPTER 2 VMware Consolidated Backup Advanced Configuration Note: There is a limit of 60 concurrently mounted volumes. That is, the 60 volumes equals the number of virtual machines multiplied by the number of drive letters per virtual machine. Quiescing Your virtual machines are automatically quiesced when you start the backup process. VMware provides a quiescing module in VMware Tools to provide file consistency. You can also run custom quiescing scripts. The optional pre-freeze and post-thaw scripts are picked up in the virtual machines here: In Windows Virtual Machines: c:\windows\pre-freeze-script.bat c:\windows\post-thaw-script.bat If a pre-freeze script exits with an exit code not equal to zero, the snapshot operation will fail. Capacity Planning Estimating Offloaded Backup Performance Since Consolidated Backup is snapshot-based, it mostly eliminates the backup impact for the protected virtual machines. Therefore, backup is not restricted to hours of low server activity but can occur around the clock. Capacity planning for backup is mainly influenced by the following factors: The amount of data to be backed up per virtual machine on each backup cycle. The throughput from the SAN storage array to the backup proxy. The throughput from the backup proxy to the backup device. The per virtual machine setup time required for creating or deleting virtual machine snapshots and for setup in the backup software. The following formula can be used to estimate the amount of time required for a virtual machine backup: 33

38 Virtual Machine Backup Guide t backup cost ds t snap t setup = x + + min ( tpread, tp write ) where t backup is the total amount of time (in seconds) required to back up a virtual machine. cost is the cost weight factor for the backup operation. The following heuristics can be used to estimate this factor: For full virtual machine backups, cost is always equal to 2. For file-level backups, it can be estimated as the average change rate between backups. For example, if on average 30% of the entire data in a virtual machine changes between two backups, then cost should be set to 0.3. For worst-case scenarios, it should be set to 1, assuming that 100% of the data in the virtual machine has changed since the prior backup. ds is the size of the virtual machine s data set in megabytes. This is the sum of the sizes of all disk images that comprise the virtual machine. tp read is the read throughput on the proxy when reading data off the virtual machine s disk images. This factor is determined by two components: The maximum read throughput that can be achieved when reading data off the SAN arrays storing virtual machine disk images. The overhead introduced by NTFS and the Consolidated Backup architecture. The actual value or tp read is gated by the maximum read throughput that can be achieved off the SAN array on the proxy. In the case of a full virtual machine backup, the actual value for tp read will be fairly close to this value. However, if a file-level backup is being performed, the value varies, depending on the data that is being backed up: For a file system containing few large files, the number is close to the maximum achievable read throughput. For a file system containing lots of small files, the value could be significantly smaller, as the file system overhead can dramatically increase in this case. tp write is the write throughput that can be achieved on the proxy writing to the backup device. This value is usually advertised by the vendor of the backup 34

39 CHAPTER 2 VMware Consolidated Backup device. In the case of lack of available bandwidth on the SAN fabric, this value could be lower. t snap is the amount of overhead introduced by Consolidated Backup. This is the time required to locate the virtual machine on the ESX peer and to create or delete the virtual machine s backup snapshot. t setup is the amount of time it takes for the backup software to launch a backup job. This value is highly dependent on the type of backup software used and might also depend on the type of backup medium being used, for example, tape positioning. Capacity Planning Example As an example for capacity planning, assume that a virtual machine with a total of 4GB disk space is to be backed up. A full backup of the virtual machine is done weekly and on average, 10% of the data in the virtual machine is modified each day. For this example we use the following estimates: ds is 4096MB since the virtual machine has 4GB of disk space. Since a full virtual machine backup is performed once a week, the cost factor is 1 on one day. On the other days in the week, it is 0.1 since we estimate that about 10% of the data in the virtual machine changes on a daily basis. cost = x Assume that raw data reads can be done from the SAN array on the proxy at a rate of 40MB/sec. Also, the file system overhead is estimated to be 15%: tp read = 40 MB/ sec x 0.85 = 34MB/ sec Assume that write performance to the backup device is 45MB/sec. This means that the I/O performance is determined by tp read rather than tp write. The time for snapshot creation and deletion and snapshot mount or unmount typically should be around 25 seconds. t snap = 25s Assume that the setup overhead for each backup job encountered by the backup software is about 30 seconds. 35

40 Virtual Machine Backup Guide Putting all these values together: t backup = 0.25 x 4096MB 34MB / sec + 25 sec + 30 sec 85 sec Note: These values are for full virtual machine backups. There is no data for incremental backups. Tape Drives Most tape drives are designed to work best at their native throughput speed. If data is supplied at a lower rate, the tape drive s write performance might drop even lower because more time is spent rewinding and repositioning, a phenomenon known as shoe-shining. Therefore, when doing backup capacity planning, you should make sure that the backup infrastructure can provide data at a rate equal to or greater than the advertised throughput of the tape drive used. Canceling a Backup Job If a backup operation is canceled from your third-party software while the backup is in process, the virtual machine might not be unmounted from the proxy server, and the snapshot might not be deleted. This is to be expected because the cleanup script was unable to run. To fix the problem, you must manually run the post-backup command for each virtual machine: 1. Check the folders in the C:\mnt directory to determine the virtual machine host names. 2. For each virtual machine host name, run the following command: cscript /nologo <VCB default installation path ["C:\program files\vmware\vmware Consolidated Backup Framework"]> post-command.wsf <virtual_machine_hostname> 36