RMAN Backup-Recovery-Oracle-Clariion-Data-Domain-Psg PDF

EMC Backup and Recovery for
Oracle 11g OLTP

Enabled by EMC CLARiiON,
EMC Data Domain, EMC NetWorker,
and Oracle Recovery Manager
using Fibre Channel
Proven Solution Guide
Copyright 2010 EMC Corporation. All rights reserved.

Published June, 2010
EMC believes the information in this publication is accurate as of its publication date. The information
is subject to change without notice.
Benchmark results are highly dependent upon workload, specific application requirements, and
system design and implementation. Relative system performance will vary as a result of these and
other factors. Therefore, this workload should not be used as a substitute for a specific customer
application benchmark when critical capacity planning and/or product evaluation decisions are
contemplated.
All performance data contained in this report was obtained in a rigorously controlled environment.
Results obtained in other operating environments may vary significantly.
EMC Corporation does not warrant or represent that a user can or will achieve similar performance
expressed in transactions per minute.
No warranty of system performance or price/performance is expressed or implied in this document.
Use, copying, and distribution of any EMC software described in this publication requires an applicable
software license.
For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on
EMC.com.
All other trademarks used herein are the property of their respective owners.
Part number: H6834.1
Table of Contents
Table of Contents
Chapter 1: About this Document ............................................................................................... 4
Overview ............................................................................................................................ 4
Audience and purpose ....................................................................................................... 5
Business challenge ............................................................................................................ 6
Technology solution ........................................................................................................... 6
Objectives .......................................................................................................................... 7
Reference Architecture ...................................................................................................... 8
Validated environment profile ............................................................................................. 9
Hardware and software resources ..................................................................................... 9
Prerequisites and supporting documentation ................................................................... 11
Terminology ..................................................................................................................... 12
Chapter 2: Use Case Components ......................................................................................... 13
Chapter 3: Storage Design ...................................................................................................... 17
Overview .......................................................................................................................... 17
CLARiiON storage design and configuration ................................................................... 18
Data Domain .................................................................................................................... 23
SAN topology ................................................................................................................... 25
Chapter 4: Oracle Database Design ....................................................................................... 28
Overview .......................................................................................................................... 28
Chapter 5: Installation and Configuration ................................................................................ 33
Overview .......................................................................................................................... 33
Navisphere ....................................................................................................................... 34
PowerPath ........................................................................................................................ 37
Install Oracle clusterware ................................................................................................. 41
Data Domain .................................................................................................................... 47
NetWorker ........................................................................................................................ 51
Multiplexing ...................................................................................................................... 57
Chapter 6: Testing and Validation ........................................................................................... 59
Overview .......................................................................................................................... 59
Section A: Test results summary and resulting recommendations .................................. 60
Chapter 7: Conclusion ............................................................................................................. 76
Overview .......................................................................................................................... 76
Appendix A: Scripts ................................................................................................................. 78
EMC Backup and Recovery for Oracle 11g OLTP Enabled by EMC CLARiiON, EMC Data Domain, EMC NetWorker,
and Oracle Recovery Manager using Fibre Channel Proven Solution Guide
3
Chapter 1: About this Document

Overview
Introduction to
about this
document
This Proven Solution Guide summarizes a series of best practices that were
discovered, validated, or otherwise encountered during the validation of the EMC
Data Domain Backup and Recovery for an Oracle 11g OLTP environment enabled
by EMC CLARiiON, EMC Data Domain, EMC NetWorker, and Oracle Recovery
Manager.
EMC's commitment to consistently maintain and improve quality is led by the Total
Customer Experience (TCE) program, which is driven by Six Sigma methodologies.
As a result, EMC has built Customer Integration Labs in its Global Solutions Centers
to reflect real-world deployments in which TCE use cases are developed and
executed. These use cases provide EMC with an insight into the challenges currently
facing its customers.
Use case
definition
A use case reflects a defined set of tests that validates the reference architecture for
a customer environment. This validated architecture can then be used as a reference
point for a Proven Solution.
Contents
The content of this chapter includes the following topics.

Topic
See Page
Audience and purpose
Business challenge
Technology solution
Objectives
Reference Architecture
Validated environment profile
Hardware and software resources
Prerequisites and supporting documentation
11
Terminology
12
Audience and purpose

Audience
The intended audience for the Proven Solution Guide is

internal EMC personnel
EMC partners, and
customers
Purpose
The purpose of this proven solution for deduplication is to define a working

infrastructure for an Oracle RAC environment with an Oracle 1 TB OLTP database
on CLARiiON storage infrastructure using a Data Domain appliance to show the
following:
Benefits of deduplication in an Oracle OLTP environment.
Backup and restore times for a 1 TB OLTP database, using EMC NetWorker,
Recovery Manager (RMAN) backups, and EMC Data Domain with inline
deduplication.
Utilization of SnapView clones to facilitate backup and restore, with minimum
impact to production and ensuring business continuity during backup and recovery.
This document provides a specification for the customer environment (storage
configurations, design, sizing, software and hardware, and so on) that constitutes an
enterprise Oracle 11g RAC backup and recovery solution in an Oracle OLTP
environment, deployed on the CX4-960.
In addition, this use case provides information on:
Building an enterprise Oracle 11g RAC environment on an EMC CLARiiON
CX4-960.
Identifying the steps required to design and implement an enterprise-level Oracle
11g RAC solution around EMC software and hardware.
Deployment of a Data Domain DD880 appliance as a Virtual Tape Library.
5
Business challenge
Overview
Today's IT is being challenged by the business to solve the following pain points
around the backup and recovery of the business critical data:
Protect the business information as an asset of the business defined

recovery point objective (RPO - amount of data to recover) and recovery
time objective (RTO - time to recover)
Efficient use of both infrastructure and people to support the business
Difficulties around the backing up of large enterprise-critical systems multiterabyte systems
Exponential data growth, changing regulatory requirements, and increasingly

complex IT infrastructure all have a major impact on data managers data protection
schemes. RTO continue to decrease while the precision of the RPO increases. In
other words, IT managers must be able to recover from a given failure quicker than
ever and with less data loss. It is not uncommon for organizations to routinely
exceed their backup window or even have a backup window that takes up most of
the day. Such long backup operations leave little margin for error and any disruption
can place some of the data at risk of loss. Such operations also mean that a
guaranteed RPO cannot be met.
Because of the demands generated by data growth and the RTO/RPO requirements
in Oracle database environments, it is critical that robust, reliable, and tested backup
and recovery processes are in place. Backup and recovery of Oracle databases are
a vital part of IT data protection strategies. To meet these backup and recovery
challenges enterprises need proven solution architectures that encompass the best
of what EMC and Oracle can offer.
Technology solution
Overview
This solution describes a backup and recovery environment of an Oracle 11g OLTP
database. The database was deployed on a CLARiiON CX4-960 using Oracle ASM
in a two-node RAC configuration. Backup and recovery was implemented using
RMAN, EMC NetWorker, and an EMC Data Domain DD880 appliance.
The backup process was offloaded to a NetWorker proxy node using Navisphere
SnapView clones to minimize the impact to the production environment. The
DD880 appliance enabled an 84 percent saving on the storage capacity required by
the backup process. Reference the Storage saving % after 5 weeks of backup
cycle chart in the Testing and Validation section.
6
The following table describes the key components and their configuration details
within this environment.
Component
Configuration
Software
CLARiiON CX4960
Four BE 4 Gb FC ports,
eight FE 4 Gb FC ports
per storage processor,
nine DAEs with five 146
GB, and 130 x 300 GB
disk drives
FLARE
04.29.000.5.003
Data Domain
DD880
Four 4 Gb FC ports VTL SAN connectivity,

two SAS HBAs - disk
connectivity, three EOS
disk shelves with 48
disks
DDOS 4.7.1.3
Oracle 11g OLTP

database system
1 TB Oracle 11g OLTP

database using ASM on
a two-node RAC
Oracle 11g
Database/Cluster/
ASM versions
11.1.0.7
NetWorker
NetWorker Management
Console, storage nodes,
clients
NetWorker 7.6
NMO 5.0
Objectives
This document provides guidelines on how to configure and set up an Oracle 11g
OLTP database with Data Domain deduplication storage systems. The solution
demonstrates the benefits of deduplication in an Oracle backup environment. The
backup schedule used level 0 (full backups), level 1 (incremental differential
backups), and Oracle Block Change Tracking (BCT).
This document is not intended to be a comprehensive guide to every aspect of an
Enterprise Oracle 11g solution. The information that is to be obtained and reported
from this document is described in the following list:
Installation and build of the infrastructure

Configure and test CLARiiON storage
Configure the Oracle 11g environment
Configure a Data Domain virtual tape library
Configure NetWorker
7
Reference Architecture
Corresponding
Reference
Architecture
This use case has a corresponding Reference Architecture document that is

available on Powerlink and EMC.com. Refer to EMC Backup and Recovery for
Oracle 11g OLTP Enabled by EMC CLARiiON, EMC Data Domain, EMC NetWorker,
and Oracle Recovery Manager using Fibre Channel Reference Architecture for
details.
If you do not have access to the following content, contact your EMC representative.
Reference
Architecture
diagram
The following diagram depicts the overall physical architecture of the use case.
8
Validated environment profile

Profile
characteristics
The use case was validated with the following environment profile.
Profile characteristic
Database characteristic
Benchmark profile
Value
OLTP
Swingbench OrderEntry - TPC-C-like
benchmark
< 10 ms
70 / 30
A Swingbench load that keeps the system
running within agreed performance limits
1 TB
1
300 GB 15k rpm
Response time
Read / Write ratio
Database scale
Size of databases
Number of databases
Array drives: size and speed
Hardware and software resources

Hardware
The hardware used to validate the use case is listed below.
Equipment
Storage array
Quantity
1
Configuration
CLARiiON CX4-960:
Nine DAEs
Five 146 GB FC drives
130 x 300 GB FC drives
SAN
4 Gb capable FC switch, 64 port
Oracle database node
Four Quad-Core Xeon E7330 processors, 2.4 GHz, 6 MB,

1066 FSB, 32 GB RAM. Two 73 GB 10k internal disks. Four
4 Gb Emulex LP11002E HBAs.
Proxy node (mount host)
Four Quad-Core Xeon E7330 processors, 2.4 GHz, 6 MB,

1066 FSB, 32 GB RAM. Two 73 GB 10k internal disks. Four
4 Gb Emulex LP11002E HBAs.
Navisphere management
server
Two Quad-Core processors, 1.86 GHz, 16 GB RAM. Two 4

Gb Emulex LP11002E HBAs.
Cisco Catalyst 3750G
NetWorker server
Network
9
Software
The software used to validate the use case is listed below.
Software
RedHat Linux
Microsoft Windows
5.3
2003 SP2
Version
Oracle Database/Cluster/ASM
11g Release 1 (11.1.0.7.0)
Oracle ASMLib
Swingbench
Orion
2.0
2.3
10.2
FLARE operating environment

Navisphere Management Suite
04.29.000.5.003
Navisphere Analyzer
SnapView
PowerPath
DDOS
NetWorker
NetWorker Module for Oracle
Cisco IOS
Fabric OS
6.29.0.6.34
6.29.0.6.34.1
5.3
4.7.1.3
7.6
5.0
12.2
6.2.0g
Comment
OS for database nodes
OS for Navisphere
Management Server
Database/cluster
software/volume management
Support library for ASM
OLTP database benchmark
Orion is the Oracle I/O
Numbers Calibration Tool
designed to simulate Oracle
I/O workloads
Includes:
Access Logix
Navisphere Agent
Multipathing software
Data Domain OS
Back up and recover suite
NetWorker Oracle integration
Network
SAN
10
Prerequisites and supporting documentation

Technology
It is assumed the reader has a general knowledge of:
Supporting
documents
The following documents, located on Powerlink.com, provide additional, relevant

information. Access to these documents is based on your login credentials. If you do
not have access to the following content, contact your EMC representative.
Third-party
documents
EMC CLARiiON
EMC Data Domain
EMC NetWorker
Oracle Database
Red Hat Linux
EMC CLARiiON CX4-960 Setup Guide

EMC Navisphere Manager Help (html)
EMC PowerPath Product Guide
EMC CLARiiON Database Storage Solution: Oracle 10g/11g with CLARiiON
Storage Replication Consistency
EMC CLARiiON Server Support Products for Linux Servers Installation
Guide
EMC Support Matrix
Data Domain OS Initial Configuration Guide
Data Domain OS Administration Guide
NetWorker Installation Guide
NetWorker Administration Guide
NetWorker Module for Oracle Administration Guide
NetWorker Module for Oracle Installation Guide
The following documents are available on third-party websites.
Oracle Database Installation Guide 11g Release 1 (11.1) for Linux

Oracle Database Backup and Recovery User's Guide
Orion: Oracle I/O Numbers Calibration Tool
11
Terminology
Terms and
definitions
This section defines the terms used in this document.
Term
Definition
ASM
Automatic Storage Management
BCT
Block Change Tracking
BE
Back End
DAE
Disk Array Enclosure
DBCA
Database Configuration Assistant
FE
Front End
NMO
RAC
Real Application Cluster
RPO
Recovery Point Objective
RTO
Recovery Time Objective
SAS
Serial Attached SCSI
SISL
Streams Informed Segment Layout
VTL
Virtual Tape Library
12
Chapter 2: Use Case Components

Introduction
This section briefly describes the key solutions components. For details on all of the
components that make up the solution architecture, refer to the hardware and
software sections.
CLARiiON
CX4-960
The EMC CLARiiON CX4 model 960 enables you to handle the most data-intensive
workloads and large consolidation projects. CLARiiON CX4-960 delivers innovative
technologies such as Flash drives, Virtual Provisioning, a 64-bit operating system,
and multi-core processors.
The CX4 new flexible I/O module design, UltraFlex technology, delivers an easily
customizable storage system. Additional connection ports can be added to expand
connection paths from servers to the CLARiiON. The CX4-960 can be populated with
up to six I/O modules per storage processor.
The CX4-960 also uses a new generation of storage processor CPUs, memory, and
PCI bus architecture.
EMC Data
Domain DD880
EMC Data Domain deduplication storage systems dramatically reduce the amount of
disk storage needed to retain and protect enterprise data. By identifying redundant
data as it is being stored, Data Domain systems provide a storage footprint that is
five to 30 times smaller, on average, than the original dataset. Backup data can then
be efficiently replicated and retrieved over existing networks for streamlined disaster
recovery and consolidated tape operations.
The Data Domain DD880 is the industrys highest throughput, most cost-effective,
and scalable deduplication storage solution for disk backup and network-based
disaster recovery (DR).
The high-throughput inline deduplication data rate of the DD880 is enabled by the
Data Domain Streams Informed Segment Layout (SISL) scaling architecture. The
level of throughput is achieved by a CPU-centric approach to deduplication, which
minimizes the number of disk spindles required.
Navisphere
Management
Suite
The Navisphere Management Suite of integrated software tools allows you to

manage, discover, monitor, and configure EMC CLARiiON systems as well as
control all platform replication applications from an easy-to-use, secure, web-based
management console.
Navisphere Management Suite enables you to access and manage all CLARiiON
advanced software functionalityincluding EMC Navisphere Quality of Service
Manager, Navisphere Analyzer, SnapView, SAN Copy, and MirrorView. When
used with other EMC storage management software, you gain storage resource,
SAN, and replication management functionalityfor greater efficiency and control
over CLARiiON storage infrastructure.
13
EMC
PowerPath
EMC PowerPath is a server-resident software that enhances performance and

application availability. PowerPath works with the storage system to intelligently
manage I/O paths, and supports multiple paths to a logical device. In this solution,
PowerPath manages I/O paths and provides:
Automatic failover in the event of a hardware failure. PowerPath automatically
detects path failure and redirects I/O to another path.
Dynamic multipath load balancing. PowerPath intelligently distributes I/O requests
to a logical device across all available paths, thus improving I/O performance and
reducing management time and downtime by eliminating the need to configure
paths statically across logical devices.
PowerPath enables customers to standardize on a single multipathing solution
across their entire environment.
EMC NetWorker
EMC NetWorker software comprises a high-capacity, easy-to-use data storage

management solution that protects and helps to manage data across an entire
network. NetWorker simplifies the storage management process and reduces the
administrative burden by automating and centralizing data storage operations.
NMO provides the capability to integrate database and file system backups, to
relieve the burden of backup from the database administrator while allowing the
administrator to retain control of the restore process. NMO includes the following
features:
Automatic database storage management through automated scheduling,

autochanger support, electronic tape labeling, and tracking.
Support for backup to a centralized backup server.
High performance through support for multiple, concurrent high-speed
backup devices.
Together with the NetWorker server, NMO augments the backup and recovery
system provided by the Oracle server and provides a storage management solution
that addresses the need for cross-platform support of enterprise applications.
EMC SnapView
SnapView is a storage-system-based software application that allows you to create a

copy of a LUN by using either clones or snapshots. A clone is an actual copy of a
LUN and takes time to create, depending on the size of the source LUN. A snapshot
is a virtual point-in-time copy of a LUN and takes only seconds to create. SnapView
has the following important benefits:
It allows full access to a point-in-time copy of your production data with modest
impact on performance and without modifying the actual production data.
For decision support or revision testing, it provides a coherent, readable and
writable copy of real production data.
For backup, it practically eliminates the time that production data spends offline or
in hot backup mode, and it offloads the backup overhead from the production
server to another server.
14
It provides a consistent replica across a set of LUNs. You can do this by

performing a consistent fracture, which is a fracture of more than one clone at the
same time, or a fracture that you create when starting a session in consistent
mode.
It provides instantaneous data recovery if the source LUN becomes corrupt. You
can perform a recovery operation on a clone by initiating a reverse synchronization
and on a snapshot session by initiating a rollback operation.
Oracle
Database 11g
Enterprise
Edition
Oracle Database 11g Enterprise Edition delivers industry-leading performance,

scalability, security, and reliability on a choice of clustered or single servers running
Windows, Linux, and UNIX. It provides comprehensive features easily managing the
most demanding transaction processing, business intelligence, and content
management applications.
Oracle Database 11g Enterprise Edition comes with a wide range of options to help
grow your business and meet users' performance, security, and availability service
level expectations.
Oracle Database 11g RAC
Oracle Real Application Clusters (RAC) is an optional feature of Oracle Database
11g Enterprise Edition. Oracle RAC supports the transparent deployment of a single
database across a cluster of servers, providing fault tolerance from hardware failures
or planned outages. If a node in the cluster fails, Oracle continues running on the
remaining nodes. If more processing power is needed, new nodes can be added to
the cluster providing horizontal scaling.
Oracle RAC supports mainstream business applications of all kinds. This includes
Online Transaction Processing (OLTP) and Decision Support System (DSS).
Oracle ASM
Oracle Automatic Storage Management (ASM) is an integrated database filesystem
and disk manager. It can reduce the complexity of managing the storage for the
database. The ASM filesystem and volume management capabilities are built into
the Oracle database kernel.
In addition to providing performance and reliability benefits, ASM can also increase
database availability because disks can be added or removed without shutting down
the database. ASM automatically rebalances the database files across an ASM
diskgroup after disks have been added or removed.
Oracle ASMLib
ASMLib is a support library for the ASM feature of Oracle Database. It is an add-on
module that simplifies the management and discovery of ASM disks. The ASMLib
provides an alternative to the standard operating system interface for ASM to identify
and access block devices.
ASMLib is composed of the actual ASMLib library, which is loaded by Oracle at
Oracle startup, and a kernel driver that is loaded into the OS kernel at system boot.
The kernel driver is specific to the OS kernel.
ASMCMD
The asmcmd utility can be used by Oracle database administrators to query and
manage their ASM systems. ASM-related information can be retrieved easily for
15
diagnosing and debugging purposes.

Oracle Recovery Manager
Oracle Recovery Manager (RMAN) is a command-line and Enterprise Managerbased tool for backing up and recovering an Oracle database. It provides block-level
corruption detection during backup and restore. RMAN optimizes performance and
space consumption during backup with file multiplexing and backup set
compression, and integrates with Oracle Secure Backup and third-party media
management products for tape backup.
Oracle Block Change Tracking
This database option causes Oracle to track datafile blocks affected by each
database update. The tracking information is stored in a block change tracking file.
When block change tracking is enabled, RMAN uses the record of changed blocks
from the change tracking file to improve incremental backup performance by only
reading those blocks known to have changed, instead of reading datafiles in their
entirety.
Swingbench
Swingbench is a publicly available load generator (and benchmark tool) designed to

stress test Oracle databases. Swingbench consists of a load generator, a
coordinator, and a cluster overview. The software enables a load to be generated
and the transactions/response times to be charted.
Swingbench is provided with four benchmarks:
OrderEntry TPC-C-like workload.

Calling Circle Telco-based self-service workload.
Stress Test Performs simple insert/update/delete/select operations.
DSS A DSS workload, based on the Oracle Sales History schema.
The Swingbench workload used in this testing was Order Entry. The Order Entry
(PL/SQL) workload models the classic order entry stress test. It has a profile similar
to the TPC-C benchmark. It models an online order entry system, with users being
required to log in before purchasing goods.
16
Chapter 3: Storage Design

Overview
Introduction to
storage design
The environment consists of a two-node Oracle 11g RAC cluster that accesses a
single production database. Each cluster node resides on its own server, which is a
typical Oracle RAC configuration. The two RAC nodes communicate with each other
through a dedicated private network that includes a Cisco Catalyst 3750G-48TS
switch. This cluster interconnection synchronizes cache across various database
instances between user requests. A Fibre Channel SAN is provided by two Brocade
4900 switches. EMC PowerPath is used in this solution and works with the storage
system to intelligently manage I/O paths. In this solution, for each server, PowerPath
manages four active I/O paths to each device and four passive I/O paths to each
device.
Contents
This chapter contains the following topics:

Topic
See Page
CLARiiON storage design and configuration
18
Data Domain
23
SAN topology
25
17
CLARiiON storage design and configuration

Design
CLARiiON CX4-960 uses UltraFlex technology to provide array connectivity. This

approach is extremely flexible and allows each CX4 to be tailored to each users
specific needs.
In the CX4 deployed for this use case each storage processor was populated with
four back-end buses to provide 4 Gb connectivity to the DAEs and disk drives. Each
storage processor had eight 4 Gb front-end Fibre Channel ports for SAN
connectivity. There were also two iSCSI ports on each storage processor which were
not used.
Nine DAEs were populated with 130 x 300 GB 15k drives, and five 146 GB drives
were also used for the vault. The CLARiiON was configured to house a 1 TB
production database and two clone copies of that database. The clone copies were
utilized as follows:
Gold copy
Backup copy
Gold copy
At various logical checkpoints within the testing process the gold copy was refreshed
to ensure there was an up-to-date copy of the database available at all times. This
was to ensure that an instantaneous recovery image was always available in the
event that any logical corruption occurred during, or as result of, the testing process.
If any issue did occur then a reverse synchronization from the SnapView clone gold
copy would have made the data available immediately, thereby avoiding having to
rebuild the database.
Backup copy
The backup clone copy was used for NetWorker proxy backups. The clone copy of
the database was mounted to the proxy node and the backups were executed on the
proxy node. This is also referred to as the clone mount host.
Configuration
It is a best practice to use ASM external redundancy for data protection when using
EMC arrays. CLARiiON will also provide protection against loss of media, as well as
transparent failover in the event of a specific disk or component failure.
The following image shows the CLARiiON layout; the CX4-960 deployed for this
solution had four 4 Gb Fibre Channel back-end buses for disk connectivity. The
back-end buses are numbered Bus 0 to Bus 3. Each bus connects to a number of
DAEs (disk array enclosures). DAEs are numbered using the Bus X Enc Y
nomenclature, so the first enclosure on Bus 0 is therefore known as Bus 0 Enc 0.
Each bus has connectivity to both storage processors for failover purposes.
Each enclosure can hold up to 15 disk drives. Each disk drive is numbered in an
extension of the Bus Enclosure scheme. The first disk in Bus 0 Enclosure 0 is known
as disk 0_0_0.
18
The following image shows how ASM diskgroups were positioned on the CLARiiON
array.
The first enclosure contains the vault area. The first five drives 0_0_0 through 0_0_4
have a portion of the drives reserved for internal use. This reserved area contains
the storage processor boot images as well as the cache vault area. Disks 0_0_11 to
0_0_14 were configured as hot spares.
Disks 0_0_5 to 0_0_9 were configured as RAID Group 0 with 16 LUNs used for the
redo logs. These LUNs were then allocated as an ASM diskgroup, named the redo
diskgroup. RAID Group 0 also contained the OCR disk and the Voting disk.
The next four enclosures contain three additional ASM diskgroups. The following
section explores this in more detail.
ASM diskgroups
The database was built using four distinct ASM diskgroups:
The Data diskgroup contains all datafiles and the first control file.
The Online Redo diskgroup contains online redo logs for the database and a
second control file. Ordinarily, Oracles best practice recommendation is for
the redo logs files to be placed in the same diskgroup as all the database
files (the Data diskgroup in this example). However, it is necessary to
separate the online redo logs from the data diskgroup when planning to do
recovery from split mirror snap copies since the current redo log files cannot
be used to recover the cloned database.
The Flash Recovery diskgroup contains the archive logs.
The Temp diskgroup contains tempfiles.
19
ASM data area

MetaLUNs were chosen for ease of management and future scalability. As the data
grows, and consequently the number of ASM disks increases, ASM will have an
inherent overhead managing a large number of disks. Therefore, metaLUNs were
selected to allow the CLARiiON to manage request queues for large number of
LUNs.
For the Data diskgroup four striped metaLUNs were created, each containing four
members. The selection of members for each metaLUN was chosen to ensure that
each member resided on a different back-end bus to ensure maximum throughput.
The starting LUN for each metaLUN were also carefully selected to avoid all the
metaLUNs starting on the same RAID group. This selection criterion was to avoid
starting all the ASM disks on the same set of spindles, and alternating the metaLUN
members to even out the LUN residence. This methodology was used to ensure that
ASM parallel chunk IOs will not be hitting the same spindles at the same time within
the metaLUNs when, or if, Oracle performs a parallel table scan.
20
EMC SnapView
SnapView clones were used to create complete copies of the database. A clone
copy was used to offload the backup operations to the proxy node. A second clone
copy was used as a Gold copy. The following graphic shows an example of a clone
LUN's relationship to the source LUN, in this example the clone information for one
of the LUNs is contained in the ASM datagroup. SnapView clones create a full bitfor-bit copy of the respective source LUN. A clone was created for each of the LUNs
contained within the ASM diskgroups, and all clones were then simultaneously split
from their respective sources to provide a point-in-time content consistent replica set.
The command naviseccli h arrayIP snapview listclonegroup data1 was used
to display information on this clone group. Each of the ASM diskgroup LUNs is
added to a clone group becoming the clone source device. Target LUN clones are
then added to the clone group. Each clone group is assigned a unique ID and each
clone gets a unique clone ID within the group. The first clone added has a clone ID
of 010000000000000 and for each subsequent clone added the clone ID increments.
The clone ID is then used to specify which clone is selected each time a cloning
operation is performed.
As shown above there are two clones assigned to the clone group. Clone ID
01000000000000000 was used as the gold copy and clone ID 0200000000000000
was used for backups. (The Navisphere Manager GUI also shows this information.)
When the clones are synchronized they can be split (fractured) from the source LUN
to provide an independent point-in-time copy of the database.
21
The LUNs used for the clone copies were configured in a similar fashion to the
source copy to maintain the required throughput during the backup process. The
image below shows the clone relationship for two of the metaLUNs.
22
Data Domain
Overview
The following sections describe how Data Domain systems ensure data integrity and
provide multiple levels of data compression, reliable restorations and multipath
configurations. The Data Domain operating system (DD OS) Data Invulnerability
Architecture protects against data loss from hardware and software failures.
Data integrity
When writing to disk, the DD OS creates and stores checksums and self-describing
metadata for all data received. After writing the data to disk, the DD OS then
recomputes and verifies the checksums and metadata. An append-only write policy
guards against overwriting valid data.
After a backup completes, a validation process looks at what was written to disk to
see that all file segments are logically correct within the file system and that the data
is the same on the disk as it was before being written to disk.
In the background, the Online Verify operation continuously checks that data on the
disks is correct and unchanged since the earlier validation process.
The back-end storage is set up in a double parity RAID 6 configuration (two parity
drives). Additionally hot spares are configured within the system. Each parity stripe
has block checksums to ensure that data is correct. The checksums are constantly
used during the online verify operation and when data is read from the Data Domain
system. With double parity, the system can fix simultaneous errors on up to two
disks.
To keep data synchronized during a hardware or power failure, the Data Domain
system uses NVRAM (non-volatile RAM) to track outstanding I/O operations. An
NVRAM card with fully-charged batteries (the typical state) can retain data for a
minimum of 48 hours. When reading data back on a restore operation, the DD OS
uses multiple layers of consistency checks to verify that restored data is correct.
Data
compression
DD OS stores only unique data. Through Global Compression, a Data Domain

system pools redundant data from each backup image. Any duplicate data is stored
only once. The storage of unique data is invisible to backup software, which sees the
entire virtual file system.
DD OS data compression is independent of data format. This can be structured, for
example, databases, or unstructured, for example, text files. Data can be from file
systems or raw volumes. Typical compression ratios are 20:1 on average over many
weeks. This assumes weekly full and daily incremental backups. A backup that
includes many duplicate or similar files (files copied several times with minor
changes) benefits the most from compression. Depending on backup volume, size,
retention period, and rate of change, the amount of compression can vary.
Data Domain performs inline deduplication only. Inline deduplication ensures:
Smaller footprint
Longer retention
Faster restore
Faster time to disaster recovery
23
SISL
Inline deduplication is enabled by Streams Informed Segment Layout (SISL). SISL

identifies 99 percent of duplicate segments in RAM and ensures that all related
segments are stored in close proximity on disk for optimal reads.
Multipath and
load-balancing
configuration
Multipath configuration and load balancing are supported on Data Domain systems
that have at least two HBA ports. In a multipath configuration on a Data Domain
system, each of two HBA ports on the system is connected to a separate port on the
backup server.
24
SAN topology
SAN topology
Oracle layout
The two-node Oracle 11g RAC cluster nodes and the proxy node were cabled and
zoned as shown in the following image. Each node contained four two-port HBAs.
Four of the available ports were used to connect the nodes to the CX4-960.
CLARiiON best practice dictates that single initiator soft zoning is used. Each HBA is
zoned to both storage processors. This configuration offers the highest level of
protection and may also offer higher performance. It entails the use of full-feature
PowerPath software. In this configuration, there are multiple HBAs connected to the
host; therefore, there are redundant paths to each storage processor. There is no
single point of failure. Data availability is ensured in event of an HBA, cable, switch,
or storage processor failure. Since there are multiple paths per storage processor,
this configuration benefits from the PowerPath load-balancing feature and thus
provides additional performance.
The connectivity diagram below shows the two-node Oracle 11g RAC cluster nodes.
25
The remaining four HBA ports in each node were zoned to the Data Domain DD880
appliance. This zoning approach ensured that the primary storage
CLARiiON CX4-960 and the backup storage DD880 were on different HBA ports.
This ensures that the traffic is segregated during backups as recommended by EMC
best practice.
Single initiator zoning was also used when zones were created for the DD880. The
zoning of the nodes to the DD880 also had redundancy built-in to ensure that tape
drives were available in the event of a cable, switch, or HBA failure.
26
NetWorker
topology
The EMC NetWorker environment provides the ability to protect your enterprise
against the loss of valuable data. In a network environment, where the amount of
data grows rapidly, the need to protect data becomes crucial. The EMC NetWorker
product gives you the power and flexibility to meet such a challenge.
A Data Domain system integrates into a NetWorker environment as the storage

destination for directed backups. In this solution the Data Domain system was
configured as a VTL. This takes advantage of the speed of disk and easily integrates
with a previously configured NetWorker environment.
27
Chapter 4: Oracle Database Design

Overview
Introduction to
Oracle
database
design
This chapter provides guidelines on the Oracle database design used for this
validated solution. The design and configuration instructions apply to the specific
revision levels of components used during the development of the solution.
Before attempting to implement any real-world solution based on this validated
scenario, gather the appropriate configuration documentation for the revision levels
of the hardware and software components. Version-specific release notes are
especially important.
ASM
diskgroups
The database was built with four distinct ASM diskgroups (+DATA, +FRA, +REDO,
and +TEMP).
ASM Diskgroup
Contents
DATA
Data and index tablespaces, controlfile
FRA
Archive logs
REDO
Online redo log files, controlfile
TEMP
Temporary tablespace
The ASMCMD CLI lists the diskgroups, showing the state of each one.
28
Control files
The Oracle database, in this solution, has two control files, each stored in different
ASM diskgroups.
Redo logs
All database changes are written to the redo logs (unless logging is explicitly turned
off) and are therefore very write-intensive. To protect against a failure involving the
redo log, the Oracle database was created with multiplexed redo logs so that copies
of the redo log can be maintained on different disks.
Archive log mode was enabled which automatically created database-generated
offline archived copies of online redo log files. Archive log mode enables online
backups and media recovery.
Note
Oracle recommends that archive logging is enabled; the following graphic shows
how to check that the database is in archive log mode.
29
The previous graphic shows that once archive log mode is enabled, the archive logs
were written out to the FRA diskgroup.
Parameter files
A centrally located server parameter file (spfile) was used to store and manage the
database initialization parameters persistently by all RAC instances. Oracle
recommends that you create a server parameter file as a dynamic means of
maintaining initialization parameters.
30
Swingbench
TPC-C-like
toolkit
The order entry wizard that is used to create the SOE schema in Swingbench 2.3
limits its size to 100 GB. The reason for this is that it executes as a single-threaded
operation and would take an unreasonable time to create a schema any larger.
However, Datagenerator is capable of creating larger schemas that would generate
much higher levels of I/O (index lookups). The script alterations specific to this
solution environment to create a 1 TB database can be seen in the appendix.
31
The Configuration dialog box (see the following image) enables you to change all of
the important attributes that control the size and type of load placed on your server.
Four of the most useful are:
Number of Users: This describes the number of sessions that Swingbench will
create against the database.
Min and Max Delay Between Transactions (ms): These values control how long
Swingbench will put a session to sleep between transactions.
Benchmark Run Time: This is the total time Swingbench will run the bench for.
After this time has expired Swingbench will automatically log off the sessions.
This graphic shows a typical example with 120 concurrent sessions.
32
Chapter 5: Installation and Configuration

Overview
Introduction to
installation and
configuration
This chapter provides procedures and guidelines for installing and configuring the
components that make up the validated solution scenario. The installation and
configuration instructions presented in this chapter apply to the specific revision
levels of components used during the development of this solution.
Before attempting to implement any real-world solution based on this validated
scenario, gather the appropriate installation and configuration documentation for the
revision levels of the hardware and software components planned in the solution.
Version-specific release notes are especially important.
Contents
This chapter contains the following topics:

Topic
See Page
Navisphere
34
PowerPath
37
Install Oracle clusterware
41
Data Domain
47
NetWorker
51
Multiplexing
57
33
Navisphere
Overview
Navisphere Management Suite enables you to access and manage all CLARiiON
advanced software functionality.
Register hosts
The Connectivity Status view in Navisphere, seen in the image below, shows the
new host as logged in but not registered. Install the Navisphere host agent on the
host and reboot, and the HBAs will then automatically register.
The Hosts tab shows the host as unknown and the host agent is unreachable; this is
because the host is multi-homed, that is, the host has multiple NICs configured, see
the following image.
34
A multi-homed host machine has multiple IP addresses on two or more NICs. The
host can be physically connected to multiple data links that can be on the same or
different networks. When you install Navisphere Host Agent on a multi-homed host,
the host agent, by default, binds to the first NIC in the host. If your host is multihomed, for the host agent to successfully register with the desired CLARiiON storage
system, you need to configure the host agent to bind to a specific NIC. This is
rectified by setting up an agentID.txt file. To do this stop the Navisphere agent, then
rename or delete the HostIdFile.txt file located in /var/log, as shown in the following
image.
Create agentID.txt in root; this file should only contain the fully qualified hostname of
the host and the IP address HBA/NIC port that the Navisphere agent should use.
The agentID.txt file should contain only these two lines and no special characters,
as shown in the following image.
35
Then stop and restart the Navisphere agent; this re-creates the HostIdFile.txt file
binding the agent to the correct NIC. The host now shows correctly on the Hosts tab.
The host is now shown as registered correctly with Navisphere, as shown in the
previous image.
36
PowerPath
Overview
EMC PowerPath provides I/O multipath functionality. With PowerPath, a node can
access the same SAN volume via multiple paths (HBA ports), which enables both
load balancing across the multiple paths and transparent failover between the paths.
PowerPath
policy
After PowerPath has been installed and licensed it is important to set the PowerPath
policy to CLARiiON-Only. The following image shows the powermt display output
prior to setting the PowerPath policy.
The I/O Path Mode is shown to be unlicensed.
37
Once the PowerPath Policy has been set correctly all paths are now alive and
licensed. The previous image shows the powermt set policy command and the
powermt display command output for CLARiiON LUN 80. It lists the eight paths for
this device. These paths are managed by PowerPath. The LUN is owned by SPA,
therefore the four paths to SPA are active and the remaining paths to SPB are
passive.
All ASM diskgroups were then built using PowerPath pseudo names.
Note
A pseudo name is a platform-specific value assigned by PowerPath to the
PowerPath device.
Because of the way in which the SAN devices were discovered on each node, there
was a possibility that a pseudo device pointing to a specific LUN on one node might
point to a different LUN on another node. The emcpadm command was used to
ensure consistent naming of PowerPath devices on all nodes.
The following image shows how to determine the available pseudo names.
38
The next image shows how to change the pseudo names using the following
command:
emcpadm renamepseudo s <xxx> t <yyy>
This table shows the PowerPath names associated with the LUNs used in the ASM
diskgroups.
Diskgroup
Purpose
Diskgroup Name
Path
CLARiiON
LUN
Data files
DATA
/dev/emcpowerac
10
/dev/emcpowerad
/dev/emcpowerae
/dev/emcpoweraf
/dev/emcpowere
65
/dev/emcpowerf
64
/dev/emcpowerg
63
/dev/emcpowerh
62
/dev/emcpoweri
61
/dev/emcpowerj
60
/dev/emcpowerk
59
/dev/emcpowerl
58
/dev/emcpowerm
57
/dev/emcpowern
56
/dev/emcpowero
55
/dev/emcpowerp
54
/dev/emcpowerq
53
/dev/emcpowerr
52
/dev/emcpowers
50
/dev/emcpowert
51
/dev/emcpoweru
22
Online Redo Logs
Temp/Undo
REDO
TEMP
39
Flash Recovery
High
availability
health check
FRA
/dev/emcpowerv
20
/dev/emcpowerw
16
/dev/emcpowerx
18
/dev/emcpowery
23
/dev/emcpowerz
21
/dev/emcpoweraa
19
/dev/emcpoweab
17
To verify that the hosts and CLARiiON are set up for high availability, install and run
the naviserverutilcli utility on each node to ensure that everything is set up correctly
for failover. To run the utility, use the following command:
naviserverutilcli hav upload ip 172.<xxxxxxx>
In addition to the standard output, the health check utility also uploads a report to the
CLARiiON storage processors that can be retrieved and stored for reference.
40
Install Oracle clusterware

Overview
Oracle 11g clusterware was installed and configured for both production nodes.
Below are a number of screenshots taken during the installation, showing the
configuration of both RAC nodes.
Specify cluster
The image below shows the installation summary screen.
Configure ASM
and Oracle 11g
software and
database
Before configuring Oracle and ASM, it is recommended to review the Oracle

Database Installation Guide 11g Release 1 (11.1) for Linux.
The following general guidelines apply when configuring ASM with EMC technology:
Use multiple diskgroups, preferably a minimum of four, optimally five. Place

the Data, Redo, Temp, and FRA in different (separate) diskgroups.
Use external redundancy instead of ASM mirroring.
Configure diskgroups so that each contains LUNs of the same size and
performance characteristics.
Distribute ASM diskgroup members over as many spindles as is practical for
the sites configuration and operational needs.
41
Partition the disks

In order to use either file systems or ASM, you must have unused disk partitions
available. This section describes how to create the partitions that will be used for
new file systems and for ASM.
When partitioning the disks it is important that the partition is aligned correctly. Intelbased systems are misaligned due to the metadata written by the BIOS. To correctly
align the partition to ensure improved performance, an offset of 64 KB (128 blocks)
was used.
This example uses /dev/emcpowera (an empty disk with no existing partitions) to
create a single partition for the entire disk.
42
ASM diskgroup
creation
The Oracle DBCA creates the ASM data diskgroup and the init.ora for the ASM
instance. Additional diskgroups can then be created.
ASM uses mirroring for redundancy. Three types of redundancy are supported by
ASM. They are:
External redundancy.
Normal redundancy: 2-way mirrored. At least two failure groups are needed.
High redundancy: 3-way mirrored. At least three failure groups are needed.
EMC recommends using external redundancy as provided by the CLARiiON CX4940. Refer to the CLARiiON configuration setup.
43
Database
installation
Once the ASM diskgroups were created, Oracle Database 11g 11.1.0.6.0 was
installed.
44
The Oracle environment was patched to 11.1.0.7.0.
45
Block change
tracking
The block change tracking feature for incremental backups improves incremental
backup performance by recording changed blocks in each datafile in a block change
tracking file. This file is a small binary file stored in the database area. RMAN tracks
changed blocks as redo is generated.
If you enable block change tracking, RMAN uses the change tracking file to identify
changed blocks for an incremental backup, thus avoiding the need to scan every
block in the datafile. RMAN only uses block change tracking when the incremental
level is greater than 0 because a level 0 incremental backup includes all blocks.
46
Data Domain
Introduction
Data Domain DD880 integrates easily into existing data centers and can be
configured for leading backup and archiving applications using NFS, CIFS, OST, or
VTL protocols. This solution focuses on VTL over Fibre Channel SAN.
The Data Domain appliance was configured with four Fibre Channel connections to
the SAN. The largest number of tape drives assigned per channel for this solution
was two, giving a maximum number of eight RMAN channels per node.
VTL option
The following image shows the Data Domain Enterprise Manager. The VTL option
requires an additional Data Domain license.
To add the license click the Maintenance tab and launch the Configuration Wizard
from the Tasks drop-down, as shown in the following image.
47
Select option 1, Licenses.
Add the VTL license and save the configuration.
After the VTL license has been applied, the VTL service can then be enabled. Go to
the Data Management page and select VTL Actions Enable.
48
After the VTL license was added, the virtual tape library was created.
Virtual tapes were created in a tape pool. For this solution the default pool was
selected.
The tapes were then imported into the library.
49
Access groups were created within the DD880 appliance to allow access to
individual tape drives and the media changer. The HBA WWNs of the nodes will be
present in the Physical Resources tab only if they are correctly zoned on the FC
switches. The HBAs are then available to be added to the access groups as
initiators. Tape drives are added to the access group as LUNs. The LUNs are
assigned primary and secondary ports on the appliance.
50
NetWorker
NetWorker
introduction
NetWorker
configuration
The following NetWorker components were installed:

NetWorker Server
NetWorker Server
NetWorker Management Console
RAC nodes
NetWorker storage node
NetWorker Client
NMO
Proxy node
NetWorker storage node
NetWorker Client
NMO
After zoning the SAN switches correctly and creating the access groups on the Data
Domain appliance, the servers can pick up the tape devices. A SCSI bus rescan is
required to achieve this.
When selecting tape devices for use it is important to select the SCSI non-rewind
devices. The st driver provides the interface to a variety of SCSI tape devices under
Linux.
First (auto-rewind) SCSI tape device name: /dev/st0
Second (auto-rewind) SCSI tape device name: /dev/st1
First the non-rewind SCSI tape devices: /dev/nst0
Second the non-rewind SCSI tape devices: /dev/nst1
The following image shows the devices selected are all non-rewind devices.
51
The three servers were added to NetWorker as storage nodes. The devices can then
be made available to NetWorker by performing a scan for devices.
To change device properties, right-click on the device and select Properties. The
parameters used for this solution are described in the following section.
The NetWorker device Target sessions and Max sessions parameters were both set
to 1. This effectively disabled NetWorker multiplexing, which is important when using
data deduplication.
Multiplexing, and its effect on data duplication, is described in detail in the
Multiplexing section.
52
The NetWorker Wizard was used to configure the client backups on each node.
NetWorker Module for Oracle NMO was installed on each node to enable NetWorker
integration with Oracle RMAN.
Client configuration, user, and connection details to the database to be backed up

are entered along with the RMAN catalog user and connectivity details, as shown in
the next graphic.
53
The following parameters were also modified:
Number of channels
Backup level
Control file backup
Archive redo log backup
Filesperset
Testing was conducted using different numbers of RMAN channels. Backup levels
were set at level 0, or full backups, on the weekend and level 1, or differential
incremental backups, on weekdays. The control file and archive redo logs were
included in the backups. Refer to the Testing and Validation section for more
details.
54
The filesperset parameter was set to 1; this is explained in detail in the next section.
55
The wizard creates the RMAN script, as shown below, which can be modified if
required.
56
Multiplexing
Overview to
NetWorker
multiplexing
Today multiplexing is a common practice used in many backup operations.

Multiplexing became widely used as traditionally most tape drives had a greater
bandwidth than an individual backup client could fill. To ensure a tape drive
bandwidth was efficiently utilized, several channels are multiplexed together to keep
the tape drive busy.
The following image shows six 10 MB/s clients writing to a 60 MB/s device. Without
multiplexing the tape device is not used to its capacity, as one client writes the others
have to wait for completion, therefore the tape device is underutilized. In this
example, the tape device has a bandwidth of 60 MB/s and each backup client has a
bandwidth of 10 MB/s.
The image below shows six 10 MB/s clients writing to a 60 MB/s device, with
multiplexing enabled, all clients are interleaved together to take advantage of the
extra bandwidth of the tape device.
57
RMAN
multiplexing
When using a deduplication appliance, such as a DD880, it is best practice to ensure

that multiplexing is disabled.
When creating backup sets, RMAN can simultaneously read multiple files from disk
and then write their blocks into the same backup set. For example, RMAN can read
from two datafiles simultaneously, and then combine the blocks from these datafiles
into a single backup piece. The combination of blocks from multiple files is called
RMAN multiplexing. Similar to NetWorker multiplexing, RMAN multiplexing has the
same negative effect on deduplication
The parameter that sets up multiplexing within Oracle is filesperset. The filesperset
parameter specifies the number of files that will be packaged together and sent on a
single channel to a tape device. This has the same effect as mixing bits from many
files, and again makes it more difficult to detect segments of data that already exist.
Therefore, to take full advantage of data deduplication it is important to have the
filesperset parameter set to 1.
58
Chapter 6: Testing and Validation

Overview
Introduction to
testing and
validation
Storage design is an important element to ensure the successful development of the

EMC Backup and Recovery for Oracle 11g OLTP Enabled by EMC CLARiiON, EMC
Data Domain, EMC NetWorker, and Oracle Recovery Manager using Fibre Channel
solution.
Contents
This section contains the following topic:

Topic
See Page
Section A: Test results summary and resulting

recommendations
60
59
Section A: Test results summary and resulting recommendations

Description of
the results
summary and
conclusions
Testing was conducted using two, four, six, and eight RMAN channels.
Backups were run using a backup cycle consisting of level 0, or full backups, on the
weekend and level 1, or differential incremental backups, Monday through Thursday.
For the purposes of this solution Friday COB was deemed to be the start of the
weekend.
The starting point for an incremental backup strategy is a level 0 incremental backup,
which backs up all blocks in the database. An incremental backup at level 0 is
identical in content to a full backup but, unlike a full backup, the level 0 backup is
considered part of the incremental backup strategy.
A level 1 incremental backup contains only blocks changed after a previous
incremental backup. A level 1 backup can be a cumulative incremental backup,
which includes all blocks changed since the most recent level 0 backup, or a
differential incremental backup, which includes only blocks changed since the most
recent incremental backup. Incremental backups are differential by default.
Note
When using a deduplication appliance such as Data Domain it is often
recommended to run only full backups as only the unique data will be stored.
However, this was outside the scope of the solution.
Archived redo logs and the control file were also backed up as part of each backup
that occurred during this solution. Backing up the archived redo logs had a significant
impact on the overall change rate of the database. The change rate of the database
was 2 percent. However, because the archived log files were backed up on every
backup, the change rate observed during incremental backups was actually much
higher, closer to 10 percent.
Both NetWorker and RMAN multiplexing (filesperset=1) were disabled for all
backups, as this has a negative effect on deduplication rates achieved.
For this use case a number of tests were carried out on the Oracle 11g OLTP
backup and recovery infrastructure. At a high level the tests performed were:
Orion validation
RMAN channel configuration
Swingbench
Validate Swingbench profile
Backup from production
SnapView clone copy from production
Backup from proxy
Data Domain deduplication
Restore
60
Orion
validation
Once the disk environment was set up on the CLARiiON CX4-960, the disk
configuration was validated using an Oracle toolset called Orion. Orion is the Oracle
I/O Numbers Calibration Tool designed to simulate Oracle I/O workloads without
having to create and run an Oracle database. It utilizes the Oracle database I/O
libraries and can simulate OLTP workloads (small I/Os) or data warehouses (large
I/Os). Orion is useful for understanding the performance capabilities of a storage
system, either to uncover performance issues or to size a new database installation.
Note
This is a destructive tool so it should only be run against raw devices prior to
installing any database or application.
This graph shows total throughput on a single node, with four metaLUNs, consisting
of 40 disks.
This demonstrates the desired scaling.
61
RMAN channel
configuration
This image shows the average MB/s throughput when backing up the database
using different numbers of RMAN channels. Eight RMAN channels proved to be the
fastest in this environment.
Shown here is a typical example of NetWorker backing up the database, in this case
using four RMAN channels.
All subsequent tests were run using the following base parameters. The
methodology for selecting these parameters was explained earlier in this document.
Backup parameters
RMAN Channels
Filesperset
NetWorker Target Session
NetWorker Max Session
Archived Redo Logs
Control File
8
1
1
1
Included in all backups
Included in all backups
62
Validate
Swingbench
profile
The following image shows the processor activity on Node1 with the Swingbench
load running against the cluster.
The next image shows the response time of the data metaLUNs (Data ASM
diskgroup) on the CLARiiON array during the same period.
Note
The array time is one hour behind the server time.
63
Backup from
production
The following graph shows the processor activity on Node 1. Similar to the previous
example the Swingbench load is running against the cluster. In addition, an RMAN
backup initiated by NetWorker is also running against Node 1. The backup is running
against the same LUNs that are serving the Swingbench load. There is an increase
in CPU utilization and iowait.
The following graph shows the response times of Data metaLUNs (Data ASM
diskgroup) on the CLARiiON. The response time is higher for the duration of the
backup.
64
Clone copy
from
production
The following graph again shows the processor activity on Node 1 under the
Swingbench load. In this example, a sync of the proxy clone is started and finished
during the time window.
The following graph shows the response time on the Data metaLUNs (Data ASM
diskgroup) on the CLARiiON. There is an initial increase in response time and iowait
(highlighted) when the sync is started. But the duration of the impact to the
production nodes is much shorter when compared to running the backup from one of
the nodes.
65
The following graph shows processor activity on Node 1 under Swingbench load.
Again, a clone sync is initiated. In this example, the database is also put into hot
backup mode after the completion of the sync. No desirable effect can be seen on
the RAC node when in hot backup mode.
The following image shows the response time on the metaLUNs (Data ASM
diskgroup) on the CLARiiON. The first increase in response time, circa 09:47,
corresponds to the start of the clone sync.
66
The second larger increase in response time, between 9:59 and 10:03, occurs when
the database is put into hot backup mode. The spike occurs when the database is
put into hot backup mode because:
Any dirty databuffers in the database buffer cache are written out to files and
the datafiles are checkpointed.
The datafile headers are updated to the system change number (SCN)
captured when the begin backup command is issued. The SCN is not
incremented with checkpoints while a file is in hot backup mode. This lets the
recovery process understand which archive redo log files may be required to
fully recover this file from that SCN onward.
The datablocks within the database files continue to be read and written to.
During hot backup, an entire block is written to the redo log files the first time
the data block is changed. Subsequently, only redo vectors (changed bytes)
are written to the redo logs.
When the database is taken out of hot backup mode the datafile header and SCN
are updated.
This shows that there is less impact on the RAC nodes when using SnapView clones
to offload the backup to the proxy host, compared to running the backup directly from
the RAC node.
67
Backup from
proxy node
The following graph shows processor activity on the proxy node during a backup.
The subsequent graph shows the response time on the proxy clone Data metaLUNs
(Data ASM diskgroup) on the CLARiiON.
68
The following two graphs show a complete level 0 backup from start to finish. They
show both production RAC nodes under Swingbench load. A clone sync is initiated
at approximately 2:50. Again, there is a short period at the beginning of the sync
during which an impact is observed. The sync completed at approximately 2:58. The
database was put into hot backup mode prior to fracturing the clones; there was no
noticeable effect on the production nodes.
69
The following graph shows the proxy node during the same time interval. There was
no load running against the proxy node. To facilitate backing up the clone copy of the
database, the instance was started and database mounted on the proxy node.
The next graph shows the response times from the CLARiiON for the duration of the
backup. The response time of the production data metaLUNs (the ASM data
diskgroup), is tracked on the upper metric. The first peak is the clone sync and the
second peak is when the database is in hot backup mode. The lower metric tracks
the proxy clone data metaLUNs during the same period; the backup is taken from
these LUNs on the proxy node, taking the additional overhead of the backup away
from the production nodes.
70
Deduplication
The following image shows the data stored on the DD880 after the first weekly
backup cycle. The backup cycle consisted of an RMAN level 0 (full) backup on the
weekend and RMAN level 1 backups Monday through Thursday. Oracle Block
Change Tracking was enabled to improve incremental backup performance. The
database daily change rate is 2 percent. However, because the archived log files
were also backed up, the change rate observed during incremental backups was
actually much higher, closer to 10 percent.
Note
The graphs show the total Data Written to the DD880 increasing over time; this is
also described as the logical capacity. The Data Stored refers to the unique data
that is stored on the appliance. The % Reduction shows the storage savings gained
from using Data Domain.
By eliminating redundant data segments, the Data Domain system allows many
more backups to be stored and managed than would normally be possible for a
traditional storage server. While completely new data has to be written to disk
whenever discovered, the variable-length segment deduplication capability of the
Data Domain system makes finding identical segments extremely efficient.
71
The following graph shows the deduplication after two weeks; there is an increase at
the weekend, when the level 0 full backups are run.
A falloff in the deduplication factor can be seen on weekdays when the backup policy
is incremental and archived redo logs are contained within every backup.
The graph above shows the backup cycle trend and deduplication factor after three
weeks.
72
The graph above illustrates a deduplication factor of 6:1 after five weeks, which
yields a storage capacity saving of 84 percent (see the following graph).
73
Restore
Data Domain Streams Informed Segment Layout (SISL) technology ensures

balanced backup and restore speeds.
The following image shows the duration of a backup of the database compared to a
restore of the database. Both the backup and restore processes were initiated on the
proxy node using eight RMAN channels.
74
The following image shows the restore throughput of the DD880 appliance. A
sustained read throughput of over 630 MB/s was achieved. Comparing this to the
data in the subsequent image, which shows the throughput achieved by the DD800
appliance during a backup, this further demonstrates the balance between backup
and restore speeds.
The following image shows the DD880 appliance during a backup with a sustained
write throughput of over 790 MB/s.
75
Chapter 7: Conclusion
Overview
Introduction to
conclusion
This proven solution Reference Architecture details an Oracle infrastructure design

leveraging an EMC CLARiiON CX4-960 array, EMC Data Domain DD880, and EMC
NetWorker. Also included are various test results, configuration practices, and
recommended specific Oracle storage design layouts that meet both capacity and
consolidation requirements. Described in this document are many of the
technologies that enable the benefits outlined below.
Conclusion
Traditional hardware compression provides substantial cost savings in Oracle

environments. However, in this solution data deduplication has been shown to
significantly reduce the amount of data that needs to be stored over an extended
period of time. This offers cost savings both from a management standpoint and in
the numbers of disks or tapes required by a customer to achieve their long-term
backup strategy.
Data deduplication can fundamentally change the way organizations protect backup
and nearline data. Deduplication changes the repetitive backup practice of tape, with
only unique, new data written to disk. The test results show that, in an environment
utilizing RMAN incremental backups, a data deduplication ratio of over 6.3:1,
resulting in an 84 percent saving in the storage required to accommodate the backup
data, makes it economically practical to retain the savesets for longer periods of
time. This reduces the likelihood that a data element must be retrieved from the
vault. Both of these factors can significantly improve the RTO.
Although cost savings are generally not the initial reason to consider moving to disk
backup and deduplication, financial justification is almost always a prerequisite. With
the potential cost savings of disk and deduplication, the justification statement
becomes, we can achieve all of these business benefits and save money. That is a
compelling argument.
The solution meets the business challenges in the following manner:
Ability to keep applications up 24x7

Faster backup and restores meet more aggressive backup windows,
and restore your key applications in minutes, not days
Reduced backup windows minimize backup windows to reduce
impact on your application and system availability
Protect the business information as an asset of the business

Reduced business risk restore data quickly and accurately with builtin hardware redundancy and RAID protection
Reduced backup windows minimize backup windows to reduce
impact on your application and system availability
Efficient use of both infrastructure and people to support the business

Improved IT efficiency save hours of staff time and boost user
productivity
Correct costs / reduce costs match infrastructure costs with changing
information value via efficient, cost-effective tiered storage
76
In summary, utilizing the solution components, in particular CLARiiON technology,

EMC Data Domain, and EMC NetWorker software, provides customers with the best
possible backup solution to prevent both user and business impact. Business can
continue as usual, as if there were no backup taking place at all. In customer
environments where, more than ever, there is a trend toward 24x7 activity, this is a
critical differentiator that EMC can offer.
Next steps
EMC can help to accelerate assessment, design, implementation, and management

while lowering the implementation risks and costs of a backup and recovery solution
for an Oracle Database 11g environment.
To learn more about this and other solutions contact an EMC representative or visit
www.EMC.com/solutions/oracle.
77
Appendix A: Scripts
Appendix A: Scripts
Datagenerator
.env file
Alterations specific to use case environment were entered in to the

Datagenerator.env file.
#!/bin/bash
# Set the following to reflect the root directory of your Java
installation
export JAVAHOME=/usr/java/jdk1.5.0_19/
# Set the following to the directory where you installed
datagenerator
export DATAGENHOME=/Datagenerator/datagenerator040198/datagenerator
# Set the following to the location of your TimesTen install
(optional)
#export TTHOME=/opt/TimesTen/tt70
#
export
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/lib:$TTHOME/lib
export
CLASSPATH=$JAVAHOME/jre/lib/rt.jar:$JAVAHOME/lib/tools.jar:$DATAGENHO
ME/lib/datagenerator.jar
# The following is needed for 10g environments
#export CLASSPATH=$CLASSPATH:$ORACLE_HOME/jdbc/lib/ojdbc14.jar
# The following is only needed for 11g environments
export CLASSPATH=$CLASSPATH:$ORACLE_HOME/jdbc/lib/ojdbc5.jar
# The following is only needed for TimesTen environments
#export CLASSPATH=$CLASSPATH:$TTHOME/lib/classes15.jar
Datagenerator
SOE schema
creation script
The following Datagenerator script was used to create and populate the SOE
schema used by Swingbench when running a typical TPC-C-type load against the
database.
-- Datagenerator "SOE" benchmark schema creation
-- Author : Dominic Giles
-- Created : 1/9/2008
-set verify off
define tablespace=SOE
variable tablespace_size number
define tablespace_size=1024
define datafile='+DATA'
define indextablespace=SOE_INDEX
define indextablespace_size=2048
define indexdatafile='+DATA'
define username=SOE
define password=SOE
define indexprefs=nologging
define parallelism=16
define parallelclause=''
define usecompression=''
-- Uncomment the following line to enable compression on the sales
and customer
-- define usecompression='COMPRESS'
define connectstring='//tce-r900-enc01/orcl'
variable scale number
define scale=1
accept username default &username prompt 'Order entry username
[&username] : '
78
Appendix A: Scripts
accept password default &password prompt 'Order entry password

[&password] : '
accept tablespace default &tablespace prompt 'Default tablespace
[&tablespace] : '
accept datafile default &datafile prompt 'Datafile for tablespace
&datafile [&datafile] : '
accept indextablespace default &indextablespace prompt 'Default index
tablespace [&indextablespace] : '
accept indexdatafile default &indexdatafile prompt 'Datafile for
tablespace &indexdatafile [&indexdatafile] : '
accept scale default &scale prompt 'Enter scale. 1 = 1GB, 100=100GB
etc [&scale] : '
accept connectstring default &connectstring prompt 'Enter
connectstring for database [&connectstring] : '
accept parallelism default &parallelism prompt 'Enter level of
parallelism [&parallelism] : '
pause Generation will now begin. Press return to continue or control
C to exit
variable t1 number;
set termout off
col new_tablespace_size new_value tablespace_size
col new_indextablespace_size new_value indextablespace_size
col new_parallelclause new_value parallelclause
select (round(&scale*&tablespace_size))||'M' new_tablespace_size,
(round(&scale*&indextablespace_size))||'M'
new_indextablespace_size,
case when &parallelism <= 1 then ''
else 'parallel '||&parallelism
end new_parallelclause
from dual
/
set timing on
set termout on
timing start stage_timer
prompt **************************************
prompt Starting tablespace creation
prompt **************************************
@@soecreatetablespace.sql
prompt **************************************
prompt Completed tablespace creation
timing show stage_timer
prompt **************************************
@@soecreateuser.sql
connect soe/SOE
--@@soedrop.sql
@@soecreatetables.sql
prompt **************************************
prompt Starting data generation
prompt **************************************
!cd ../..;./datagenerator -c soe.xml -scale &scale -nodrop -ni -nc cl -db -u &username -p &password -cs &connectstring -tc &parallelism
-async -bs 100 -commit 100000
prompt **************************************
prompt Completed data generation
prompt **************************************
prompt **************************************
prompt Creating constraints
prompt **************************************
@@soeconstraints.sql
prompt **************************************
79
Appendix A: Scripts
prompt Creating indexes

prompt **************************************
prompt Creating indexes
@@soeindexes.sql
prompt **************************************
prompt Completed index and constraint creation
prompt **************************************
@@soeviews.sql
@@soepackage.sql
prompt **************************************
prompt Started schema analysis
prompt **************************************
set timing off
exec
dbms_stats.gather_schema_stats(SYS_CONTEXT('USERENV','CURRENT_SCHEMA'
),degree =>&parallelism);
@@soeinvalid.sql
prompt **************************************
prompt Completed analysis
prompt **************************************
prompt
prompt Completed building Order Entry schema
prompt
prompt **************************************
exit
SOE schema
installation
The SOE schema was installed manually using the scripts located in the
$DATAGEN_HOME/bin/scripts/soe directory.
These scripts can be run directly from SQL*Plus as the SYS or SYSTEM user.
The following command is an example of how to start the installation using the
soe_install script:
[oracle@TCE-R900-ENC01 soe]$ sqlplus /nolog
SQL*Plus: Release 11.1.0.7.0 - Production on Thu Aug 20 10:47:32 2009
Copyright (c) 1982, 2008, Oracle.
All rights reserved.
SQL> connect /as sysdba

Connected.
SQL> @soe_install
Order entry username [SOE] :
Order entry password [SOE] :
Default tablespace [SOE] :
Datafile for tablespace +DATA [+DATA] :
Default index tablespace [SOE_INDEX] :
Datafile for tablespace +DATA [+DATA] :
Enter scale. 1 = 1GB, 100=100GB etc [1] : 200
Enter connectstring for database [//tce-r900-enc01/orcl] :
Enter level of parallelism [16] : 12
Generation will now begin. Press return to continue or control C to
exit
This script was used to create the schema of the testing user (in this case, soe) and
load all of the data necessary to run the tests.
80
Appendix A: Scripts
NetWorker
RMAN backup
script
The RMAN script below is a typical example of one used to generate backups
through the NetWorker console.
This example shows an eight-channel incremental level 0 backup to tape. Each
backup was assigned a tag ID, which was later used as part of the restore process.
RUN {
ALLOCATE CHANNEL CH1 TYPE 'SBT_TAPE';
BACKUP
INCREMENTAL LEVEL 0
FILESPERSET 1
FORMAT '%d_%U'
TAG= 'RUN529'
DATABASE PLUS ARCHIVELOG;
backup controlfilecopy '+FRA/ORCL/control_backup'
RELEASE CHANNEL CH1;
}
Oracle RMAN
restore script
tag= 'RUN529_CTL';
The restore process consisted of first allocating eight channels then restoring the
controlfile, mounting the database, and performing the restore database command
using the tag ID assigned earlier. Below is a sample restore script.
RUN
{
restore controlfile from tag'RUN529_CTL';
alter database mount;
restore DATABASE from tag'RUN529';
}
81

RMAN Backup-Recovery-Oracle-Clariion-Data-Domain-Psg PDF

Enviado por

Dados do documento

Descrição original:

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

RMAN Backup-Recovery-Oracle-Clariion-Data-Domain-Psg PDF

Enviado por

Direitos autorais:

Formatos disponíveis

EMC Backup and Recovery for

Oracle 11g OLTP

Copyright 2010 EMC Corporation. All rights reserved.

Chapter 1: About this Document

The content of this chapter includes the following topics.

Audience and purpose

Validated environment profile

Hardware and software resources

Prerequisites and supporting documentation

Chapter 1: About this Document

Audience and purpose

The intended audience for the Proven Solution Guide is

The purpose of this proven solution for deduplication is to define a working

Chapter 1: About this Document

Protect the business information as an asset of the business defined

Exponential data growth, changing regulatory requirements, and increasingly

Chapter 1: About this Document

Four 4 Gb FC ports VTL SAN connectivity,

Oracle 11g OLTP

1 TB Oracle 11g OLTP

Installation and build of the infrastructure

Chapter 1: About this Document

This use case has a corresponding Reference Architecture document that is

Chapter 1: About this Document

Validated environment profile

Hardware and software resources

The hardware used to validate the use case is listed below.

Five 146 GB FC drives

130 x 300 GB FC drives

4 Gb capable FC switch, 64 port

Oracle database node

Four Quad-Core Xeon E7330 processors, 2.4 GHz, 6 MB,

Proxy node (mount host)

Four Quad-Core Xeon E7330 processors, 2.4 GHz, 6 MB,

Two Quad-Core processors, 1.86 GHz, 16 GB RAM. Two 4

Cisco Catalyst 3750G

Chapter 1: About this Document

The software used to validate the use case is listed below.

11g Release 1 (11.1.0.7.0)

FLARE operating environment

Chapter 1: About this Document

Prerequisites and supporting documentation

It is assumed the reader has a general knowledge of:

The following documents, located on Powerlink.com, provide additional, relevant

EMC CLARiiON CX4-960 Setup Guide

The following documents are available on third-party websites.

Oracle Database Installation Guide 11g Release 1 (11.1) for Linux

Chapter 1: About this Document

This section defines the terms used in this document.

Automatic Storage Management

Block Change Tracking

Disk Array Enclosure

Database Configuration Assistant

NetWorker Module for Oracle

Real Application Cluster

Recovery Point Objective

Recovery Time Objective

Serial Attached SCSI

Streams Informed Segment Layout

Virtual Tape Library

Chapter 2: Use Case Components

Chapter 2: Use Case Components