Update on VMware vSphere Backup APIs & Backup Approaches

Pete Marfatia - Systems Engineer, VMware. VMUG 2010 June 03

2009 VMware Inc. All rights reserved

Agenda
Introduction RPO and RTO a quick explanation Where various VMware technologies fit Remind me, why are we backing up again? Backup Methods in a VMware Environment Quick recap of VMware Consolidated Backup (VCB) framework vStorage APIs for Data Protection (vSphere 4 )
vADP & VCB side by side comparison vADP ready partner solutions

Types of Data Deduplication

VMware Data Recovery Appliance Overview VMware Data Recovery Install, Config and FAQs (for reference later)

Introduction
RPO and RTO a quick explanation Where various VMware technologies fit Remind me, why are we backing up again?

Disaster Recovery Concepts

Disaster Strikes
Wks Days Hrs Mins Secs Secs Mins Hrs Days Wks

Recovery Point

Recovery Time

Recovery Point Objective (RPO)

Point in time to which data must be recovered (i.e. how old can the data be?)

Recovery Time Objective (RTO)

Time required to complete recovery of the application and its data

Recovery Point Objective (RPO)

Disaster Strikes
Wks Days Hrs Mins Secs Secs Mins Hrs Days Wks

Recovery Point
Tape Backup

Recovery Time

Periodic Replication Asynchronous Replication Synchronous Replication

Recovery Time Objective (RTO)

Disaster Strikes
Wks Days Hrs Mins Secs Secs Mins Hrs Days Wks

Recovery Point

Recovery Time
Automated Migration Manual Migration Tape Restore

Recovery Time includes:

Fault detection Recovering data Bringing apps back online

vCenter Site Recovery Manager

Introduction
RPO and RTO a quick explanation Where various VMware technologies fit Remind me, why are we backing up again?

Technology to help maintain availability and reduce downtime

Business Continuity
Maintain High Availability and Reduce Downtime

Planned Outage
Maintenance Tasks Hardware MM, vMotion & DRS Storage vMotion Update Manager

Unplanned Outage

Equipment Level
vCenter Server vCenter Heartbeat Host Level HA, HA, FT VM, O/S hang HA

Site Level (DR)

Backup Tape / Image
VCB vADP vCenter DR

Software Recovery Tasks vCenter Server vCenter Heartbeat VCB vADP vCenter DR Backup Tape / Image

Array Replication

Site Recovery Mgr

RPO/RTO = 0 with VMware Technology What we are going to discuss today

Site Level (FB/DC relocation) Site Recovery Manager

Backup and Restore in a virtual environment

Using in-guest agentswhy might you be using this? Existing backup solution does not support VCB / vADP (Really?) Carry over from P2Vtransitioning to solution leveraging VCB / vADP Support for specific NOS eg: File level back up of Linux (VCB limitation) Specific application level integration required eg: Domino, SAP For fine grain restore eg: Individual Exchange mailbox restores Source based de-duplication ie minimize data transferred over WAN Potential to restore to other systems (physical) for DR

As with physical, watch out for I/O bottlenecks

Backup and Restore in a virtual environment

Why might you increase the percentage of VM backup to a solution leveraging VCB / vADP vs in guest? Agent less => reduced operational touch / generic approach to VM backup80:20
rule. Use in-guest only where specifically required

VM snapshot technology to shrink effective backup window Offload backup from the LAN to optimized SAN => faster and more efficient File system-level consistent backup via Sync Driver or MS VSS

Application-level consistent backup via MS VSS

Ability to easily do image level backup for reduced RTO ie DR or change control

10

Introduction
RPO and RTO a quick explanation Where various VMware technologies fit Remind me, why are we backing up again?

11

Remember why we do backups in the first place

Restore requirements dictate backup methods RTO is impacted by the speed and ease with which you can recover the
system successfully AND consistently

Will the application be ready to run once restored or do you need to check
consistency/replay transaction logs first? How long?

Backup to/restore from disk vs tape maybe required Restore over SAN vs over LAN vs over WAN

Remote offices may use a Virtual Storage Appliance locally, which can dedupe
then replicate back to H/O more efficiently across the WAN to meet requirements of both offsite backup AND RTO

And so on

What about the ESX host config? COS based agent installsupported but less than ideal COS goes in 2011ESXi is like an appliance, just reload, restore config & go Other VMware features such as Host Profiles for config remediation &
consistency
12

Backup Methods in a VMware Environment

Quick recap of VMware Consolidated Backup (VCB) framework

vStorage APIs for Data Protection (vSphere 4 )

vADP & VCB side by side comparison vADP ready partner solutions

Types of Data Deduplication

13

VCB Framework Topography The old way of backing up VMs

First available in 2007 with VI3 Required a physical Proxy server with FC or iSCSI HBAs (virtual proxy in 1.5)

Support for backup on FC, iSCSI, NFS, LAN mode

Windows file and image level backup Image level for Linux

VMware Snapshot quiescing options

Nothing (not a good idea) File system-level consistent via Sync driver Pre- & Post-quiescing scripts VSS* for file system- & application-level

Early partner integration was varied but has improved significantly in experience, features and the number of participating partners:
IBM Tivoli Storage Manager CA BrightStor ARCserve EMC NetWorker & Avamar Symantec NetBackup & Backup Exec HP Data Protector CommVault Galaxy VizionCore vRanger Pro

Some security and retry mechanism improvements

14

* ESX(i) 3.5 U2 or later required

VCB Components Old way of backup software integration

VCB (since v1.1) can now reside on the same host as Virtual Center

mydata SAN-based VMFS datastore

15

VCB File-Level Backup: How it works

16

VCB VM Image Backup: How it works

17

The future for VCB

Its going away with vSphere 4.next (2010 timeframe) per the email many of you received earlier this year This allows us to focus resources on further enhancing vADP--OK?

Still cant see a way out? Speak with your VMware Partner or VMware Rep for a Professional Services
engagement to help square this away

18

Backup Methods in a VMware Environment

Quick recap of VMware Consolidated Backup (VCB) framework

vStorage APIs for Data Protection (vSphere 4 )

vADP & VCB side by side comparison vADP ready partner solutions

Types of Data Deduplication

19

vStorage APIs for Data Protection

Next evolution of VMware Data Protection Part of vSphere 4no extra install VADP Enables native integration between vSphere 4 and backup application Backup server is now proxy equivalent Supports Windows and Linux file-level backup Supports MS VSS VADP Supports FC, iSCSI, NAS or local storage Improved Functionality Supports full, differential, and incremental image backup and restore of virtual
machines

Efficient backups Easy restores Goal is to make virtual backup better than physical

vADP works with Change Block Tracking

VMware has introduced Changed Block Tracking with vSphere 4.0 where
the ESX(i) host stores the last changed time for each block in a VMDK

Applications (like VMware Data Recovery & Storage VMotion) can access this
information and find out know which blocks have changed and which have not been touched

vADP APIs may now access this last changed time data and determine if the
block needs to be backed up or notfaster backup, less to transfer

Storage I/O is reduced with this algorithm as we only touch changed blocks for
backup

Included in the API to access storage on VMFS volumes Changed block tracking is only available with VMs running with hardware
version 7
Change Block Tracking (CBT) on virtual machines http://kb.vmware.com/kb/1020128 Cannot enable FT on backed up virtual machine http://kb.vmware.com/kb/1015300 Reverting to a pre-existing snapshot can cause incremental backups based on

Changed Block Tracking to become inconsistent

21

http://kb.vmware.com/kb/1021607

vADP & VCB side by side comparison

vADP VCB
Yes
Yes, with two step copy source to VCB proxy and VCB proxy to the target No Yes, Windows only Yes, by using VMware Converter

Requires additional download & install

Full VM image backup

No, built into the data protection software

Yes, single step copy source to target Yes using change block tracking Yes, both Windows and Linux Yes

Incremental VM image backup File level backup Full VM image restore

Incremental VM image restore

File level restore CLI for image backup CLI for file backup

Yes
Yes, using restore agents No Yes

No
Yes, using restore agents Yes Yes

Introduction to vStorage APIs for Data Protection

22

http://blogs.vmware.com/storage/2010/02/introduction-to-vstorage-apis-for-data-protection---vstorageapis-for-data-protection-were-introduced-in-vsphere-40-to-facil.html

vADP ready Partner Solutions

Symantec Backup Exec 2010 & Net Backup 7.0 Vizioncore vRanger Pro 4.2

EMC Avamar 5.0

Commvault Simpana 8 with

post SP4 update VEEAM 4.0

Data Recovery 1.x

CA ArcServe 12.5

Tivoli TSM v 6.2

vStorage APIs for Data Protection (VADP) FAQ

http://kb.vmware.com/kb/1021175

23

Backup Methods in a VMware Environment

Quick recap of VMware Consolidated Backup (VCB) framework

vStorage APIs for Data Protection (vSphere 4 )

vADP & VCB side by side comparison vADP ready partner solutions

Types of Data Deduplication

24

Data Deduplication - Concepts

Deduplication is where only one unique instance of the data is actually

retained on the storage media, such as disk or tape. Redundant data is replaced with a pointer to the unique data copy. Similar concept to ESX(i) transparent page sharing!

Deduplication helps reduce the amount of space consumed by full backups Initially during the first full backup all the blocks on the VM are backed up
(deduplicated accordingly), and subsequent full backups only need to store the blocks which have changed since the previous backup
25

Types Data Deduplication (source based & target based)

Source (agent) based data deduplication Reduces data to be transferred to backup server / target

Target (backup server) based data deduplication

Data common across multiple guests/hosts can be deduplicated, further reducing storage required

In this example, we are backing up multiple hosts which are all running the
same OS and similar applications

26

VMware Data Recovery Appliance Overview

27

Overview of VMware Data Recovery Functionality

Backups:
Disk based backup, not file level (fast backup and quick restore) Any Guest OS can be backed up. A VM can be backed up even if it is suspended or powered off. Leverages VM snapshot technology and uses the SYNC driver or Microsoft VSS to quiesce the Guest OS on a running VM. VMware Tools must be installed and running. Leverages VM Change Block Tracking (very efficient storage of backup)

Restores:
Can overwrite the current VM. Restore the VM if it no longer exists. Restore a test VM which is identical to the original VM, but does not have networking enabled this is called a Restore Rehearsal.

File level restore Technical Note

http://www.vmware.com/pdf/vdrFileRestore_Usage.pdf

28

VMware Data Recovery

1. Backup vCenter 4.0 VirtualCenter

1. Schedule backups via VC 2. Snapshots taken 3. Data de-duped and stored
De-duplicated Storage

Agent-less, disk-based backup and recovery of VMs on ESX(i) 4.0 hosts VM or file level restore Incremental backups plus data de-dupe and compressed to save disk space Quick, simple and complete data protection for your VMs Ease of deployment

2. Restore

vCenter 4.0
1. VM goes down 2. Select VM images/files to recover 3. Fast restore of VMdowntime minimized

Centralized Management through vCenter Server

Cost Effective Storage Management

29

VMware Data Recovery Key Components

Backup and Recovery Appliance
Linux appliance in OVF format - leverages vStorage API for Data Protection to discover, manage backup and restore First backup is full VM, then incremental forever VM or file level restore

VMware Infrastructure
VSS support via VMware Tools Changed block tracking functionality allows backups to be more efficient

Destination Storage
Any VMFS storage: DAS, NFS, iSCSI or Fibre Channel storage plus CIFS shares as target All backed up virtual machines are stored in a deduplicated datastore

vCenter integration VMware vCenter

vSphere Client Plug-in Wizard driven backup and restore job creation Automatically import virtual machine inventory Awareness of HA/VMotion/DRS

30

VDR Appliance Hot-Add Mechanism

Virtual Infrastructure VDR Appliance

Snapshot

Shared Storage

Changed
Blocks

VMDK

Snapshot

31

VDR Appliance Hot-Add Mechanism

Effectively the VDR appliance attaches to a linked-clone disk of the original VM
during a hot-add backup operation to access the read-only base disk for backup purposes

VM being backed up

Appliance VM

Delta Disk

Delta Disk

Base Disk

32

Overview of Data Recovery Components (continued)

VMware Data Recovery uses a variation of the new vStorage API called
vcbAPI.

Only works with vSphere vCenter 4.0 & ESX(i) 4.0. Data Recovery VM must have a store for dedupe purposes. This can be:
Additional local VMDK assigned to the appliance (Recommended for performance reasonscan leverage SCSI Hot-Add feature vs using the LAN) RDM (iSCSI or FC) mapped to the appliance SMB/CIFS shared storage configured via the VDR GUI NFS storage supportability: VMDK on an NFS share presented to the ESX(i) is supported. An NFS share mapped to the appliances is not supported.

VDR does a full backup, only the used blocks on the disk are obtained. If the
disk is lazy zeroed or a thin disk, then not all of the disk needs to be read.
Storage guidelines http://viops.vmware.com/home/docs/DOC-11532

33

VMware Data Recovery - Licensing

Data Recovery is included in the following editions:
vSphere Essentials Plus vSphere Enterprise vSphere Advanced vSphere Enterprise plus

Existing Enterprise customers are entitled to Data Recovery as part of SnS. VI3 Foundation and VI3/vS4 Standard Edition customers can purchase as an Addon. See vSphere Pricing, Packaging and Licensing - VMware vSphere 4 The VMs being protected must exist on ESX(i) hosts that have valid licenses. VDR can protected up to a maximum of 100 Virtual Machines SMB market. A host that is licensed can run multiple Data Recovery appliances. Currently only tested to one VDR appliance per vCenter Server instance. Does not support vCenter Linked Mode.

34

VMware Data Recovery - Limitations

There is currently no integration with tape.
See appendix slides on Archiving the Backups to backup dedupe destinations to tape or replicate. VMs.

VDR cannot backup Fault Tolerant VMs as there is no way to snapshot FT VDR appliance cannot reside in a vApp. 8 concurrent jobs running on the appliance at any time (backup & restore). Maximum of 100 Virtual Machines can be backed up using the appliance
(maximum of 100 backup jobs). This is a hard limit.

An appliance can have at the most two store destinations (up to 1TB max
each) due to the overhead involved in deduping. However this is a soft limit and is not strictly enforced, and is unsupported if the limit of two is exceeded.

Multiple backup appliances can be deployed but they do not interact with
each other. Hence it is up to the user to make sure that the same VM is not used as a source in both the appliances.

There is no support to backup up VMs on ESX(i) 3.x hosts.

35

VMware Data Recovery Install, Config and FAQs

(for reference)

36

Installing
ALWAYS use latest code! Download (see resources section) Install vSphere Client plug-in Then File \ Deploy OVF Template to deploy VDR appliance

Add your storage destination VMDK

Now turn on the appliance, and get ready to do configuration

Installing Distribution Binaries

Install VDR Plug-in and Appliance

Installing File / Deploy OVF Template

Deploy From Downloaded OVF

Configuration Outline
Open vSphere console to the appliance
Turn on the appliance IP / TZ Change root account password from vmw@re

Use plug-in
Credentials Destinations may need to use IP Create a backup job!

Configuration

Configure Network Settings of VDR Appliance

Your First Backup Start

Create a Backup Job

Your First Backup Step 1

Select Virtual Machine to Backup

Your First Backup Step 2

Select Destination Dedupe Store

Your First Backup Step 3

Select Backup Window

Your First Backup Step 4

Select Retention Policy

Your First Backup Step 5

Review Backup Job

Your First Backup - Configured

Backup Job Configured

Your First Backup

Bring To Compliance

Your First Restore

Restore From Restore Points

Your First Restore Step 1

Select Restore Point

Your First Restore Step 2

Select Restore Destination

Your First Restore Step 3

Review Restore Operation

Your First Restore In Operation

Restore In Progress

VDR Technology Preview

File Level Restore Client GUI

File Level Restore Client Step 1

Mount Restore Point

File Level Restore Client Step 2

VM VMDK Mounted on E:

Can Mount Multiple VMDKs

Technical Details
Know your retention needs, and what you chose! Test carefully! Actually do a restore! Restore rehearsal is with new datastore and no network!

File level restore available with experimental utility (command line tool with a
GUI version soon)

VDR is NOT backward compatible NO ESX 3.5 or VC 2.5!

Technical Details
Virtual Hardware 7 is needed for best performance! It provides changed block tracking to
shorten backup for VDR,

First image backup is full, all the rest are incremental, but restores are always from full. The de-duplication is block level, not file and it happens within the VMDK even though it
is a file. The de-duplication can happen across VMs in the destination de-dupe store so it is VERY efficient

Each job runs once per 24 hour period If your job is at folder / host / resource pool new VMs back up without a configuration
change!

Technical Details - Retention

Decide How Many Backups to Retain

Technical Details Virtual Hardware

Virtual Hardware v.4 vs. v.7

Technical Details Virtual Hardware

Upgrading to Virtual Hardware v.7

Tip: use vCenter Upgrade Manager achieve this across all your VMs, easily.

Technical Details Virtual Hardware

Upgraded to v.7

Things To Know

Archiving the Backups

You need to make a copy of your back up destination offsite for the best protection; you
can easily do this with VDR

You first select your destination and than press Unmount No need to backup appliance (jobs, catalog for restores, and back ups all
in appliance)

Copy it to tape, to another location, or whatever works for you You can mount it to another appliance as necessary. Suggest you save the .OVF of the
appliance with the destination in case of version issues in the future

Things To Know

Unmount Destination Before Archiving

Things To Know

Extending Your Destination

In the VS Client, Grow the VMDK file in the VM hardware settings Restart the appliance Now back in the Data Recovery client, unmount the destination, and
use the Extend option

You may be prompted for an Integrity Check so let it start.

If not, select it on the menu bar it is right beside Extend

When it is ready you can mount it and it will be ready for use!

Things To Know

Changing Virtual Disk Size to 60 GB

Things To Know

Extending Destination Dedupe Store after Size Change

Best Practices and Guidelines

Use the ovf to install is smoother and avoids human error Use a static IP that is added to DNS At most two destinations per appliance

At most 100 VMs backed up per appliance hard limit

Suggest Thick destination disk to start, not Thin. Thin works but Thick
provides a small improvement in performance

Destination can be VMDK or CIFS, but strongly recommend VMDK for

performance

VMs with RDMs in virtual compatibility mode are supported. VM snapshots

not possible on RDM physical compatibility, so not supported.

Best Practices and Guidelines Must have 5 8 GB in destination to start for system use, so start with
at least 20 GB Most space efficient to have less destinations with more VMs rather than opposite And group like VMs together to maximize dedupe!

Resources
Community - http://www.vmware.com/products/data-recovery/community.html Resources - http://www.vmware.com/products/data-recovery/resources.html Storage guidelines - http://viops.vmware.com/home/docs/DOC-11532 File Level Restore Usage - http://communities.vmware.com/docs/DOC-10670 Eval guide http://www.vmware.com/resources/techresources/10055 Docs http://www.vmware.com/support/pubs/vdr_pubs.html Release notes http://www.vmware.com/support/vdr/doc/vdr_10_releasenotes.html Admin guide - http://www.vmware.com/pdf/vdr_10_admin.pdf Email alerts - http://communities.vmware.com/thread/252977?tstart=0 Bits - http://www.vmware.com/download/vsphere/
Technical Resources http://www.vmware.com/products/data-recovery/ http://www.vmware.com/technical-resources/virtual-storage/resources.html http://www.vmware.com/technical-resources/high-availability/best-practices.html http://www.vmware.com/technical-resources/high-availability/virtualization-backup.html http://www.vmware.com/a/webcasts/details/257/

Summary
Know your retention needs, and what you chose! Test carefully! Move dedupe store destination off site! Keep an eye on the last backup date of the VMs! Lots of resources available Make sure your VMs are VH7!

Q and A
Questions Please!
You cannot change the URL port from 5480! Change network settings via console instead of web

VDR will backup a VM that has been VMotioned

VDR will backup a VM that has been Storage VMotioned Version number can be found on the Configuration tab in the client. If you appliance doesnt keep its name, make sure you have a forward, and reverse DNS
entry for it.

You do not need to upgrade the VMware Tools in the appliance The backups will not run if the host CPU utilization is 90% or higher, or if free space in
destination is less than 5 GB per VM in job

Only 8 backup or restore jobs can run at the same time How do I see the other logs? Use <shift _ logs> when in the Log view. You do not have to change the hardware version for the appliance!

Background Material
Error The operation is not allowed on non-connected sockets means IP is good but the
engine in the appliance is not listening on the socket that the UI is using. Install again but dont delete!

Yes, you can point a new appliance at old destination all will be seen To be able to use DNS for destinations
/etc/hostname with FQDN /etc/hosts with IP and FQDN /etc/sysconfig/network (HOSTNAME=, DOMAIN=) /etc/resolve.conf with DNS server /etc/sysconfig/network-scritps/ifcfg-eth0 with IP info Hostname F /etc/hostname Service network restart

Reset an appliance to defaults

Service datarecovery stop Rm /var/vmware/datarecovery/Config10.* Service datarecovery start

Background Material #2
Are you suspicious of your appliance and want to confirm something? You can turn off
your appliance, and install a new one. It should use a different datastore destination and you can do a backup. In this way you eliminate your appliance and destinations as potential issues! Most important, dont let a single VM be backed up by both of these appliances!

Did you know that when VDR starts, it polls all VMs that are part of its backup jobs, and
removes any stale SCSI Hot-Add associations to the appliance and will delete any VDR snapshots (_datarecovery_) in any of these VMs. This is in case a crash occurred in the middle of a backup operation.

Did you know when VDR starts a backup of a VM it will look for and delete any snapshot
that VDR created. This is in case a crash occurred in the middle of a backup operation.

How often does the retention policy apply? It is applied once per day if the dedupe store
is 80% full and has less than 250 GB free space, otherwise it is applied once per week.

Background Material #3 datarecovery.ini

You can make adjustments to VMware Data Recovery through the use of an ini file.
These require 1.0.1 of VDR.

You will generally need to create the file and populate it. Log into the appliance, and stop Data Recovery (service
section in it
datarecovery stop)

Create a file called datarecovery.ini in \var\vmware\datarecovery, and create an [Options] Some of the more useful options include:

IntegrityCheckInterval=x is default every day with a once per week deep check. If
x=2, then the deep check would be every two days.

BackupRetryInterval=x the default is 30 minutes if possible, and the x value is in

minutes and has no max value.

RetentionPolicyInterval=x if there is more than 20% of storage available, or more

than 256 GB available, there will be no retention policy grooming. So if your dedupe store is 81% full and with less than 256 GB free, and this is set to =3, then every three days it will perform a groom.

Background Material - Screens

VDR Task Progress

Background Material - Screens

View Logs

Background Material - Screens

Destination Dedupe Store Configuration

Background Material Screens

Unmount Destination Dedupe Store Before Archiving

Thank You

81