Virtual Server Agent v9 with VMware. June 2011

Virtual Server Agent v9 with VMware June 2011

Contents Summary... 3 Backup Transport Methods... 3 Deployment Scenarios... 3 VSA Installation Requirements... 4 VSA Patch Requirements... 4 VDDK Installation... 4 VADP User Permissions... 5 Backup Permission Requirements... 5 Restore Permission Requirements... 5 VMware Environment Requirements... 6 Change Block Tracking... 6 VMware Port Requirements... 6 Configuring the VSA... 7 Subclient Configuration... 9 Considerations for Subclient Configuration... 11 Master VSA... 11 Running a Backup... 12 VSA with SnapProtect... 13 Requirements... 13 Backup Permission Requirements... 13 Restore Permission Requirements... 13 Snapshot Engine Requirements... 14 Configuration... 14 Entering the Array Information... 14 Enable SNAPPROTECT on the Client... 15 Create the Snapshot Copy... 16 Setting SnapProtect on the Subclient... 18 Running a VSA Snap Backup... 19 Running a Snap Backup Copy... 21 Running the Backup Copy with the Snap Job... 21 Running the Backup Copy Separately... 22 Restores... 24 Container Level Restores... 25

File / Folder Level Restores... 27 Considerations for VSA with VADP... 29 Considerations for VSA with SPE... 29 Summary vstorage APIs for Data Protection (VADP) available with VMware ESX 4 and later introduces several benefits over the VMware Consolidated Backup (VCB) method of protecting virtual machines. There are several requirements that must be met for VADP to function correctly. This document covers those requirements for both Commvault s Virutal Server Agent (VSA) and VMware infrastructure. Backup Transport Methods The three methods utilized by VADP to transport backup data are defined here. 1. SAN Data is read directly from the storage where virtual machines reside. This is also referred to as LANfree since no data is transferred over the network. The LUN(s) containing the virtual machine disks must be visible to the VSA server to support SAN mode backups. SAN mode is supported when utilizing fiber and iscsi storage. 2. Hot-Add The VSA is installed on a virtual machine residing on an ESX server. Hot-Add mode can achieve close to SAN mode performance if SAN is not available. The term Hot-Add refers to the way the backups are completed. In Hot-Add the data volumes containing the virtual machines to be included in the backup are automatically mounted to the VSA vm. 3. NBD & NBDSSL - NBD (network block device) & NBDSSL (encrypted NBD) transmit data over the ESX servers TCP/IP connection. This can be the production network or a dedicated backup network. Deployment Scenarios The VSA can be installed on a physical or virtual server and is typically deployed in one of the three following configurations in relation to the corresponding Media Agent (MA). 1. Virtualized VSA/Virtualized MA - VSA installed in Hot-Add mode along with MA software. The backup destination is typically network based (CIFS/NFS) or another vmfs datastore and tape out options are very limited. Shared storage is required for HotAdd mode backups of VMs living on other ESX hosts. When to Use All virtual environment where physical servers are not preferred. Data Stores are configured on NFS. Each MA can write directly to a mountpoint presented to the VM Able to segregate backup and production traffic on the network. When NOT to Use Direct backup to tape is required. Backup traffic cannot be segregated from production traffic.

2. Virtualized VSA/Physical MA VSA installed in Hot-Add mode. Data is transferred over the LAN to a physical MA. This model also allows for the use of a centralized and Linux MAs. Shared storage is required for HotAdd mode backups of VMs living on other ESX hosts. When to Use Direct backup to tape or SILO to cloud is required. Segregation of backup and production data is possible. Physical MA is required for other protection tasks and secondary operations. When NOT to Use Segregation of backup and production data is not possible. SAN based DataStores on FC or iscsi are in use. (use all physical configuration) 3. Physical VSA/Physical MA The VSA agent and MA are located on a physical server for SAN only backups with no data transferred over the LAN. A pure physical implementation will often provide the best performance and requires physical hardware with visibility into the storage network. When to Use Data Stores are configured on FC or iscsi SAN (No NFS). Able to provide Physical Windows server with SAN access to Data Store LUNs Direct tape copies are required or physical MAs required for secondary operations. When NOT to Use SAN access to Data Store LUNs from outside ESX environment is not possible. VSA Installation Requirements Windows Server 2003 SP 2 or later. Microsoft.NET Framework 2.0 or higher. (.NET will be automatically installed with the VSA) VSA Patch Requirements The latest VSA patches must be installed and are mandatory for VADP to function correctly. Always check Maintenance Advantage for recent updates when deploying Commvault products. The latest updates for v9 VSA can be found at the location below. https://services.commvault.com/mainadv.asp VDDK Installation The VDDK (Virtual Disk Development Kit) is now pushed with the installation of the VSA. No separate installation is required.

VADP User Permissions A dedicated account can be used for performing backups on vcenter/esx. The predefined VCB backup role can be copied and modified to perform VADP Backups. This role will require several additional permissions which are listed below. This account only needs to be created in the event that a restricted account is desired. Backup Permission Requirements Datastore Allocate Space Browse a datastore Low level file operations Virtual machine Configuration Disk Change Tracking Disk Lease Virtual machine Provisioning Allow read-only disk access Allow virtual machine download Clone virtual machine Virtual machine State Create Snapshot Remove Snapshot Restore Permission Requirements Restores are typically performed using the admin/root account. However a dedicated account can be created for this purpose. This account requires additional permissions over the account used for backups. These permissions are defined here. Resource Virtual machine Interaction Assign virtual machine to resource pool Assign vapp to resource pool Power Off Power On Virtual machine Configuration Add existing disk Add new disk Add or Remove device Advanced Change CPU count Change Resource Disk change Tracking Disk Lease Host USB device Memory Modify device setting Raw device Reload from path Remove disk Rename Reset guest information Settings Swapfile placement Upgrade virtual hardware Virtual machine Inventory Create new Register Remove Unregister Virtual machine Provisioning Allow disk access Allow read-only disk access Allow virtual machine download Virtual machine State Create snapshot Remove snapshot Revert to snapshot Network Assign network Datastore Allocate space

VMware Environment Requirements The vcenter server and all ESX servers should be at vsphere 5, 4.1 or 4.0 update 2 if possible. Should update 2 on 4.0 not be available, then a common user must be created on the vcenter server and all ESX servers to avoid vmx file download issues. VMware ESX Server Level - ESX(i) servers must be at version 4 or above. Disk Types - Independent disks and Physical RDM s are not supported. Virtual RDM s are supported. Stand Alone ESXi - Stand-alone ESXi is not supported without the standard license level. License levels are explained in more detail in the following article. http://blogs.vmware.com/esxi/2009/06/esxi-vs-esx-a-comparison-of-features.html vapp - Virtual machines configured as vapp are not supported. Fault Tolerant VMs - Fault Tolerant VM s are not supported for backup. Duplicate Disk Names - Virtual machines with duplicate disk names may fail to back up due to a duplicate index entry. Duplicate GUID - Virtual machines that have the same GUID may fail on restore. Change Block Tracking CBT must be enabled to support VADP backup. The VSA agent will automatically check and enable CBT at the time of backup. VSA may not be able to enable CBT for cloned and migrated VMs. CBT can be checked by following the steps documented in the following VMware article. http://kb.vmware.com/kb/1020128 VMware Port Requirements Prior to performing any backup or restore operations ensure the port requirements listed below are met. Virtual Center Ports for web service (default: 443) and TCP/IP (default: 902) must be opened. Should vcenter be configured to use non-default ports these must be opened. ESX Server Ports for web service (default: 443) and TCP/IP (default: 902) must be opened for the vstorage APIs for Data Protection.

Configuring the VSA 1. Right Click the VSA, Select All Tasks and then Create new Instance. Note Only one instance is needed per VSA. Multiple instances will only be required if multiple vcenters are being targeted by a single VSA. 2. Enter the instance name. Select VMware from the drop down Vendor Type list. 3. Select Virtual Center if you will be connecting through Virtual Center and enter the server address and enter the credentials or Select ESX Servers and click Configure ESX Servers if you will be directly targeting individual ESX servers. Note -Discover by vcenter whenever possible. Especially in environments that are leveraging DRS and have frequent vmotion operations.

4. Enter the name of the ESX server, User Name and Password. Click OK then OK. Click OK to close the Virtual Server Instance Property page. Note - The credentials entered here require specific privileges in vcenter. Please see the VADP User section of this document for the minimum permissions required. 5. Right click the Backup Set under the Instance you just created and select Properties. Note Only one backup set should be created per VSA. Multiple backup sets increase the possibility of double backups and/or missed guest machines. 6. Select vstorage on the General tab. Note - Setting vstorage is recommended when using VADP and will prevent failback to VCB. Automatic should only be selected in mixed environments.

7. Click the Guest Hosts Configuration tab. Click Discover. Set the selected VM(s) to the desired subclient. Click OK to close. Subclient Configuration 1. Right Click the Subclient under the Backup Set and select Properties.

2. Select the Backup Type. Select Transport Mode if desired. Click OK. Note - Setting the transport mode will force that method to be used or fail if not available. Auto will revert to NBD if the preferred method is not available. 3. Click the Storage Policy tab and select the desired Storage Policy. 4. Click the Content tab and selected desired machines for backup. Click OK to close.

Considerations for Subclient Configuration The default Subclient should be used and run on a regular basis as this performs the VM discovery operations. Creating one VM per Subclient is not a recommended configuration. This creates issues around scheduling and becomes unmanageable in larger environments. Typically only the default Subclient will be used in most deployments. Exceptions to this include; Multiple VSAs are deployed in the environment and the Master model will be in use. (See Master VSA below) Multiple Storage Policies are required for different VMs. Rule based discovery is in use and different Subclients will be used to target a particular affinity. Master VSA When multiple VSAs are in use a single VSA can be configured to control and cordinate the activities of all other proxies. This VSA is refered to as the Master VSA and all subclients are created on this client while the actual backup operations are offloaded to other proxies. This will allow all activities to be coordinated from a single interface and reduces the possibility of double backups and/or missed guest machines. 1. Create the subclients on the Master VSA. These can be automatically populated by leveraging one of the automatic discovery options or manually configured. 2. On the General tab of each subclient properties page select the Use Proxy dropdown box and select the desired VSA. Leaving the box blank will default to the proxy the subclient is being created on. Note Although the VSA agent needs to be installed on all of the proxies, you should only configure 1 instance on the master VSA server. The other proxy machines do not need to have an instance created.

Running a Backup 1. Right click the Subclient and select Backup. Select Backup Type and whether to Schedule or start job immediately. Click on the Advanced button. Click OK to start the Backup. 2. Enable Granular recovery is selected by default. Remove checkbox if this is not desired. Click OK to close. Click OK again to start/schedule the job.

VSA with SnapProtect The SnapProtect backup enables you to create a point-in-time snapshot of the data to be used for various data protection operations. SnapProtect backup works in conjunction with software and hardware snapshot engines to provide snapshot functionality for data protection operations. A dedicated ESX server can be used for selective copy to tier 2 storage. This completely removes any utilization on the ESX farm. Granular Recovery can be performed at this stage for individual file and folder recovery. Requirements VADP-SPE User Permissions A dedicated account can be used for performing VSA with SPE backups. This account requires additional permissions to be added to the VADP User role listed earlier in this document. This account only needs to be created in the event that a restricted account is desired. Backup Permission Requirements Datastore Allocate Space Browse a datastore Configure Datastore Remove Datastore Rename Datastore Host Configuration Storage partition configuration Virtual machine Configuration Disk Change Tracking Disk Lease Low Level file operations Virtual machine Provisioning Allow read-only disk access Allow virtual machine download Clone virtual machine Virtual machine State Create snapshot Remove snapshot Restore Permission Requirements Datastore Allocate space Host Configuration Storage partition configuration Network Assign network Resource Assign virtual machine to resource pool Assign vapp to resource pool Virtual machine Configuration Add existing disk Add new disk Add or Remove device Advanced Change CPU count Change Resource Disk change Tracking Disk Lease Host USB device Memory Modify device setting Raw device Reload from path Remove disk Rename Reset guest information Settings Swapfile placement Upgrade virtual hardware Virtual machine Interaction Power Off

Power On Virtual machine Inventory Create new Register Remove Unregister Virtual machine Provisioning Allow disk access Allow read-only disk access Allow virtual machine download Virtual machine State Create snapshot Remove snapshot Revert to snapshot Snapshot Engine Requirements Please see BOL 9 for the latest supported Snapshot Engines. http://documentation.commvault.com/commvault/release_9_0_0/books_online_1/english_us/features/s nap_backup/vs_vmware/setup_snap_engine.htm Configuration Entering the Array Information 1. From the Commcell Console select Control Panel and then Array Management. 2. Click Add on the Array Management menu. Note Ensure that the array information is entered in the same format that is presented to the ESX server. (for example should storage be presented by IP to the ESX servers it must also be entered by IP in Array Management)

The VSA will need visibility into the storage network for NFS and iscsi implementations. This may require a second network connection on the VSA for isolated storage networks. 3. Enter the Array configuration information. See BOL 9 for the correct format for your manufacturer, supported arrays and additional considerations. Click OK when finished. Click OK again to close the Array Management window. Enable SNAPPROTECT on the Client 1. Right click the Client with the VSA installed and select Properties.

2. Select the Advanced tab. Select Enable SnapProtect. Click OK to close. Create the Snapshot Copy 1. Under Policies > Storage Policies right click the desired storage policy. Select All Tasks > Create New Snapshot Copy.

2. On the General tab of the Copy Properties dialog box: Enter the Copy Name. Select the Library, MediaAgent, Drive Pool* and Scratch Pool* from the list. (*if required) 3. Click on the Retention tab, ensure Enable Data Aging is selected and adjust the retention to ensure Snapshot copies are kept a set amount of time. Note Please consult with your hardware array vendor to ensure that the appropriate amount of SnapShot reserve space is available for the desired retention time. These snapshots are managed by CommVault and will expire after their retention and subsequent data aging criteria has been met. The hardware array will not be managing the expiration of these snapshots.

4. Click OK to create the Snap Copy Storage Policy. The Copy will now be visible listed as a Primary Snap copy in the Storage Policy. Setting SnapProtect on the Subclient 1. Select the desired VSA Backupset. Right click the Subclient and select Properties. Select the Snap Backups tab. Note This tab is not visible if Snap Backups were not enabled at the client level 2. Select SnapProtect and select the appropriate Snap Engine. Note Array Management can also be accessed by clicking Enter Array Credentials

3. Select Select ESX server for snap mount. Highlight the desired server and then click OK. Click OK again to close. Note - This is the ES/ESXi server that the snap will be mounted to for performing backup copy operations. The same or a separate ESX/ESXi server that is utilized to create the Snapshot can be utilized to backup the snapshot. Running a VSA Snap Backup There are two primary steps involved with a VSA/SPE backup. These are the creation of the HW snap and then copying this snap to secondary media. The two scheduling options are listed below. Create a Snapshot copy without a backup copy. This is useful to create hardware based snapshots during the day without moving them off of the array. These snapshots can be backed up as part of a secondary job in the storage policy, or aged off through retention settings. Create a Snapshot copy with an inline Backup copy. This is useful for creating backup copies of the VMware data for protection outside of the array while leveraging the hardware capabilities of the snapshot engine. The backup copy will be created immediately after the HW snapshot has been completed. 1. Right Click the Subclient and select Backup.

2. Select Full, Immediate and then click the Advanced tab. Click OK to run the job. 3. Once the backup has completed right click the Subclient and select List Snaps.

4. The snaps created should be visible. Click Close. Running a Snap Backup Copy The Snap backup copy can be created immediately after the hardware snap is performed or scheduled separately in the storage policy. The steps required for both methods are listed below. Running the Backup Copy with the Snap Job 1. Right click the Subclient and select Backup. Click the Advanced button.

2. Select Create Backup Copy immediately. Select Enable Granular Recovery if desired. Click OK. Then OK. Note This will copy the Snapshot to the library immediately after completion of the HW snap. Running the Backup Copy Separately 1. Right click the Storage Policy and select Run Backup Copy. 2. Set the schedule if desired. Click OK to start or schedule the job. Note When implementing a multiple HW snap to single backup copy model the backup selection rule must be modified. Failure to do so will cause the backup copy to go into a pending state as it is waiting for earlier snaps to be backed up first. The required steps are listed below.

3. Right click the storage policy and select properties. Select the Snapshot tab. Select Chose the Backup Selection Rule. Click Advanced. 4. Select Manually Select Backups. Click OK. Then OK.

Restores The VSA supports three primary restore levels. These are file/folder, volume and container level. The procedure for file/folder and container restores will be shown below. 1. Right click the Backup Set or Subclient and select Browse Backup Data. 2. Select the desired time range for the restore or leave the default Browse latest Data. Select the intended restore type. Click OK.

Note When performing restores for VSA with SPE the Copy precedence must be changed to view the data from the backup copy. Data from the snapshot copy will be viewed by default. These steps can be skipped for non SPE backups. 1. On the browse options window select the Advanced button. 2. Check Browse from copy precedence. Set the value to 2. Click OK. Then OK again. Container Level Restores 1. Select the desired machine for the restore. Then click Recover all selected.

2. Select the desired restore options. 1. Select the appropriate restore as option. Virtual Machines should be selected when restoring a machine directly into vcenter/esx. VMDK Files should be selected when the virtual machine files will be restored to a secondary location outside of the VM environment. 2. Enter the Destination client and specify the Staging path. The Destination client can be the same machine that performed the backup or any client with the VSA installed. Note: The Staging path for VM restores is a temporary folder and will be deleted when the restore completes. This is a working folder and the entire VM will NOT be copied to this folder. When performing an out of place VMDK Filesrestore the entire machine WILL be copied to this folder. 3. Override Default Selection should be selected when restoring to a different vcenter/esx server.

4. Click Configure Password and set the password. Click Browse VC/ESX and select the desired ESX and datastore. Click OK. 5. Enter the desired name for the restored VM. Click OK to start the restore. File / Folder Level Restores 1. Select the desired machine and folders/files for the restore. Then click Recover All Selected.

2. Enter the desired restore options. Click OK to start the restore. Note: The destination client can be any client that has the FS ida installed.

Considerations for VSA with VADP Deploying the VSA for HotAdd mode Backups - When choosing the datastore on which the virtualized VSA will be installed, choose a datastore with the largest VMFS block size. This is necessary to ensure that the VSA can mount and back up virtual machines residing on all datastores. VDDK 1.2 is installed to the Base folder with the VSA. v8 VSA machines that were upgraded to v9 will need to have the Virtual Disk Development Kit registry key that was manually created during the installation of VDDK 1.1 deleted. Failure to do so will result in the VDDK 1.1 to still be utilized or fail if it has been uninstalled. (VDDK 1.1 does not support HotAdd mode on ESX 4.1) Fault Tolerant VM s are not supported for snapshot operations and cannot be backed up. The vstorage API can be used to restore backups performed with VCB. The backup data will first be staged to the proxy then vstorage will perform the restore. Backups performed via the vstorage API cannot be restored through the VMware converter. Converter does not understand the format of vstorage. When Change Block Tracking is unavailable backups will revert to CRC to determine changed blocks. Since the VSA needs to read the entire virtual machine disk, CRC incremental backups may take almost as long as full backups. However, the amount of data transferred and stored by an incremental backup is limited to the changed blocks within the virtual disk. Correcting CBT on the problematic VM is recommended as quickly as possible to take full advantage of VADP based incremental backups. Tools are available to check the correct installation of the VDDK and for verifying and correcting CBT issues. These can be found at the following location. http://cic/sitecollectiondocuments/virtualization/vsa%20tools/vmwarechangetrackingbatch.zip Helper VM s are not required for Hot-Add VSA servers utilizing VADP. VMware vcenter Converter Standalone 4.3 no longer supports VCB sources for restores. Version 4.0.1 must be used. HotAdd backups that experience a hard failure may result in mounted disks on the proxy that are not automatically cleaned up. To resolve this issue manually remove the disks from the proxy virtual machine and delete the \tmp\vmware directory created in the current user's Documents and Settings directory, typically found at C:\Documents and Settings\username\Local Settings\Temp\vmware-username\. The next time a snapshot is created or deleted for the virtual machine the redo logs will get cleaned up. Considerations for VSA with SPE A physical VSA/MA is strongly recommended for environments leveraging fiber channel storage (required for HDS). SnapProtect will perform a full backup. When switching to or from a snap backup will cause the next backup to be a full backup. Storage for iscsi and NFS must be entered in array management in the same format that it is presented to the ESX servers. (i.e. should storage be connected by IP on the ESX it must entered by IP in Array Management) When leveraging NFS storage an entry for each IP used will need to be entered into the array management. Entering only a single IP for a management interface is not sufficient. The VSA proxy must have visibility into the storage network. Should this be an isolated network an additional network connection will need to be added to the proxy.