D96091GC10 SG

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2019, Oracle and/or its affiliatesฺ
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake
Zero Data Loss Recovery
Appliance Overview
Student Guide
D96091GC10
Edition 1.0 | November 2016 | D98586
Learn more from Oracle University at oracle.com/education/

Authors Copyright © 2016, Oracle and/or its affiliates. All rights reserved.
Hector Navarro Disclaimer
Alessandro Colonna This document contains proprietary information and is protected by copyright and
other intellectual property laws. You may copy and print this document solely for your
own use in an Oracle training course. The document may not be modified or altered
Technical Contributors in any way. Except where your use constitutes "fair use" under copyright law, you
and Reviewers may not use, share, download, upload, copy, print, display, perform, reproduce,
publish, license, post, transmit, or distribute this document in whole or in part without
Jeff Ferreira the express authorization of Oracle.
Kelly Smith
The information contained in this document is subject to change without notice. If you
Tim Chien find any problems in the document, please report them in writing to: Oracle University,
500 Oracle Parkway, Redwood Shores, California 94065 USA. This document is not
Marco Calmasini warranted to be error-free.
Restricted Rights Notice

Editors
Nikita Abraham If this documentation is delivered to the United States Government or anyone using
the documentation on behalf of the United States Government, the following notice is
a ble
Kavita Saini applicable:
fe r
U.S. GOVERNMENT RIGHTS
an s
Graphic Editor - t r
The U.S. Government’s rights to use, modify, reproduce, release, perform, display, or
n
Prakash Dharmalingam
a no
disclose these training materials are restricted by the terms of the applicable Oracle
license agreement and/or the applicable U.S. Government contract.
h a s
Publishers
Trademark Notice
m ) Oracle iand/or
d e ฺ
Pavithran Adka Oracle and Java are registered
ฺ c oowners. Gu its affiliates. Other names
trademarks of
Syed Ali
n k ng ent
may be trademarks of their respective
Raghunath M
n b a
S t ud
Jobi Varghese
u nio this
l l o @ use
a k be e to
l o (a cens
b el li
ak eem
Contents
1 Introduction
About This Course 1-2
Course Overview 1-3
Additional Resources 1-4
How Prepared Are You? 1-5
Introductions 1-6
a ble
fe r
2 Zero Data Loss Recovery Appliance Solution
ans
Objectives 2-2
n - t r
o
Agenda 2-3
s an
What Are Engineered Systems? 2-4
) ha eฺ
ฺ c om Guid
Engineered Systems: Oracle Zero Data Loss Recovery Appliance X6-2 2-5
n k ng ent
Oracle Zero Data Loss Recovery Appliance: Overview 2-6
ud
Engineered System: Simple to Install, Manage, and Maintain 2-9
n b a t
Agenda 2-10
nio this S
u
Oracle Recovery Appliance Cloud-Scale Solution 2-11
@ use
Agenda 2-12 l l o
a k be e to
Oracle Recovery Appliance Hardware Specifications 2-13
l o (a cens
Oracle Recovery Appliance X6-2 Hardware Details 2-14
bel li
Oracle Recovery Appliance X6-2: Compute Server 2-17
em
ake
Oracle Recovery Appliance X6-2: Storage Server 2-18
Oracle Recovery Appliance X6-2: InfiniBand Switches 2-19
Oracle Recovery Appliance X6-2: Management Switch 2-20
Agenda 2-21
Oracle Recovery Appliance Base Software 2-22
Oracle Integrated Lights Out Manager (Oracle ILOM) 2-24
Agenda 2-25
Implementation Workflow 2-26
Summary 2-28
3 Recovery Appliance Functions and Features

Objectives 3-2
Agenda 3-3
Why CIOs Care 3-4
Why Customers Care 3-5
iii
Inadequate Data Recovery = Big Business Impacts 3-6
Key Database Protection Goals 3-7
Agenda 3-8
Backup Appliances Are Not Designed for Database 3-9
Oracle Deduplication and Validation Challenges 3-11

Agenda 3-12
Recovery Appliance: Unique Benefits for Business and IT 3-13
Oracle Zero Data Loss Recovery Appliance: Overview 3-15
Key Architecture Components 3-16
Data Guard-Like Protection for All Databases 3-18
Data Loss Protection from Site Disasters 3-19
a ble
fe r
Data Loss Protection from Users or Errors 3-20
ans
Data Loss Protection from Corruptions 3-22
n - t r
o
an
Space-Efficient “Virtual” Full Backups 3-25 s
ha eฺ
Incremental Forever Backup Strategy 3-26 )
Delta Push 3-27 ฺ c om Guid
Real-Time Redo Transport 3-29
n k ng ent
n b a
Delta Store: Incremental Backups 3-31
S t ud
u nio this
Delta Store: Virtual Full Backup Creation 3-33
l l o @ use
Delta Store: Efficient Indexing and Storage Management 3-34
k be e to
Delta Store: Virtual Full Backup Optimization 3-35
a
l o (a cens
Delta Store: Virtual Full Backup Purging 3-36
bel li
Delta Store: How It Works 3-38
em Delta Store: Receiving Backup Pieces 3-40
ake Delta Store: Container Files and Delta Pools 3-41
Delta Store: Delta Pool Plans 3-43
Internal Tasks 3-44
Recovery Appliance Metadata Database: CATALOG 3-46
RMAN Recovery Catalog and Virtual Private Catalogs 3-48
Data Encryption 3-50
Protected Databases 3-51
Recovery Appliance Storage 3-52
Recovery Appliance Storage Space Management 3-54
End-to-End Data Protection Visibility 3-57
Fast Recovery with Oracle Recovery Appliance 3-58
Policy-Based Cloud Scale Protection 3-61
Dynamic Policy-Based Space Allocation 3-63
iv
Modern Cloud-Scale Database Protection 3-64
Recovery Appliance Replication 3-65
Media Manager Integration 3-67
Enterprise Manager Cloud Control 3-68
Reporting 3-70
Summary 3-72
4 Recovery Appliance Installation

Objectives 4-2
Agenda 4-3
Oracle Exadata Deployment Assistant (OEDA) 4-4
OEDA 4-6
a ble
fe r
OEDA Configuration: Welcome Screen 4-8
ans
OEDA Configuration: Customer Details Screen 4-9
n - t r
o
an
OEDA Configuration: Hardware Selection Screen 4-10
OEDA Configuration: Rack Details Screen 4-11 s
ha eฺ
)
OEDA Configuration: Customer Network Requirements 4-12
ฺ c om Guid
OEDA Configuration: Customer Network Screen 4-17
k ng ent
OEDA Configuration: Admin Network Screen 4-19
n
n b a t ud
OEDA Configuration: Ingest Network Screen 4-21
S
u nio this
OEDA Configuration: InfiniBand Network Screen 4-22
l l o @ use
OEDA Configuration: Replication Network Screen 4-23
k be e to
OEDA Configuration: Compute Node OS Screen 4-24
a
l o (a cens
OEDA Configuration: Review and Edit Screen 4-25
bel li
OEDA Configuration: Cluster Info Screen 4-26
em OEDA Configuration: Cluster Details Screen 4-27
ake OEDA Configuration: Ingest Cluster Networking Screen 4-29
OEDA Configuration: Replication SCAN Screen 4-30
OEDA Configuration: Cell Alerting Screen 4-31
OEDA Configuration: Platinum Support Screen 4-32
OEDA Configuration: Auto Service Request Screen 4-33
OEDA Configuration: Configuration Manager Screen 4-35
OEDA Configuration: Tape Library Screen 4-36
OEDA Configuration: Comments Screen 4-38
OEDA Configuration: Generate Files Screen 4-39
OEDA Configuration: Finish Screen 4-40
OEDA Configuration: Installation Template 4-42
OEDA Installation: Steps Overview 4-43
OEDA Installation: Get the Current Software - Step 1 4-44
OEDA Installation: Pre-Install Check - Step 2 4-46
OEDA Installation: Installation Script - Step 3 4-48
v
OEDA Installation: RA Installation Script - Step 4 4-50
OEDA Installation: EM Discovery - Step 5 4-52
Network Test Throughput 4-53
Agenda 4-54
Expanding the Recovery Appliance 4-55

Extending the Recovery Appliance 4-57
Summary 4-58
5 Zero Data Loss Recovery Appliance Configuration

Objectives 5-2
Agenda 5-3
Recovery Appliance Configuration Tasks 5-4
a ble
fe r
Agenda 5-5
ans
Recovery Appliance Administrative Interfaces 5-6
n - t r
o
an
Agenda 5-7
Recovery Appliance Discovery 5-8 s
ha eฺ
Agenda 5-12 )
ฺ c
Recovery Appliance Administration Roles 5-13 om Guid
Cloud Control User Accounts 5-14
n k ng ent
Agenda 5-22 n b a
S t ud
u
Protection Policies 5-23nio this
l l
Remote Access 5-35 o @ use
Agenda 5-36
a k be e to
l o (a cens
Recovery Appliance Metadata 5-37
bel li
em Recovery Appliance Metadata 5-46
ake Agenda 5-58
Summary 5-84
6 ZDLRA Protected Databases Configuration

Objectives 6-2
Agenda 6-3
Oracle Database 12c 6-4
Protected Database Administration 6-6
Agenda 6-10
Recovery Appliance Backup Module 6-11
Agenda 6-16
Real-Time Redo 6-17
Agenda 6-20
vi
Enrolling Protected Databases 6-21
Configuring Protected Databases 6-26
Agenda 6-29
Backing Up Protected Databases 6-30
Monitoring Protected Databases 6-35

Migration 6-41
Agenda 6-50
Recovering Protected Databases 6-51
Summary 6-56
7 Optimizations and Performance Tuning

a ble
fe r
Objectives 7-2
ans
Agenda 7-3
n - t r
o
an
Recovery Appliance Sizing 7-4
Optimization 7-6 s
ha eฺ
Agenda 7-7 )
Optimization 7-8 ฺ c om Guid
Tasks 7-14
n k ng ent
n
Recovery Appliance Users 7-15b a
S t ud
Migration 7-16 u nio this
l l o @ use
Agenda 7-19
a k be e to
l o (a cens
rastat.pl 7-20
bel li
network_throughput_test.sh 7-23
em Agenda 7-25
ake Oracle Enterprise Manager: Performance Pages 7-26
Summary 7-27
8 Disaster Recovery with Replication

Objectives 8-2
Agenda 8-3
Data Loss Protection from Site Disasters 8-4
Recovery Appliance Replication Concepts 8-6
Agenda 8-7
Recovery Appliance Replication Process 8-8
Agenda 8-9
Recovery Appliance Replication Interfaces 8-10
Recovery Appliance Replication Status 8-11
Agenda 8-13
Recovery Appliance Replication Configuration 8-14
vii
Recovery Appliance Replication Status 8-18
Summary 8-20
9 Recovery Appliance Integrated Media Management

Objectives 9-2
Agenda 9-3
Tape Archival 9-4
Recovery Appliance Tape Configuration 9-6
Recovery Appliance Alternative Tape Configurations 9-8
Agenda 9-9
Integrated Media Manager Components 9-10
Integrated Media Manager Concepts 9-12
a ble
fe r
Integrated Media Manager Interfaces 9-13
an s
Media Manager Implementation in the Recovery Appliance 9-14
n - t r
o
an
Oracle Clusterware Media Manager Services in the Recovery Appliance 9-16
s
Oracle Clusterware Media Manager Directories in the Recovery Appliance 9-17
ha eฺ
)
Copy-to-Tape Backup Operations in the Recovery Appliance 9-18
ฺ c om Guid
Integrated Media Manager Concepts in the Recovery Appliance 9-19
k ng ent
Integrated Media Manager in the Recovery Appliance 9-21
n
Integrated Media Manager 9-22 n b a
S t ud
Summary 9-28 u nio this
l l o @ use
10 Maintenance
a k bande Patching
e to
(a 10-2
Objectives
l o c e ns
b el
Agenda li
10-3
em Recovery Appliance Patch Overview 10-4
ake Recovery Appliance Patch Planification 10-6
Recovery Appliance Patching 10-8
Agenda 10-18
Recovery Appliance Oracle Enterprise Manager Plug-In 10-19
Agenda 10-21
Recovery Appliance Powering Off, Powering On 10-22
Agenda 10-25
Replacing a Damaged Physical Disk 10-26
Safe Disk Removal 10-28
Replacing a Damaged Flash Card 10-29
Moving All Disks from One Cell to Another 10-30
Using the Cell Software Rescue Procedure 10-32
Summary 10-35
viii
11 Diagnostics and Troubleshooting
Objectives 11-2
Agenda 11-3
Recovery Appliance Main Areas for Issues 11-4
Generic Troubleshooting Questions 11-6

Agenda 11-7
Diagnostic Collection 11-8
Other Useful Diagnostic Tools: Exachk 11-10
Other Useful Diagnostic Tools: ExaWatcher 11-13
Items to Collect 11-15
Agenda 11-18
OEM Monitoring Home Page 11-19
a ble
fe r
OEM Protected Database Metrics 11-20
ans
OEM Incidents 11-21
n - t r
o
an
imageinfo and imagehistory 11-22
Agenda 11-23 s
ha eฺ
Automatic Service Request 11-24 )
ASR Process 11-25 ฺ c om Guid
Agenda 11-26
n k ng ent
n b a t ud
Oracle Configuration Manager Overview 11-27
S
Configuring OCM 11-28 u nio this
Summary 11-29
l l o @ use
a k be e to
s
(a withceBInPublisher
12 Reporting
el l o li
m b Objectives 12-2
e Agenda 12-3
ake Introduction to BI Publisher 12-4
Recovery Appliance Reporting Workflow 12-5
Agenda 12-6
Oracle-Provided Reports with the Recovery Appliance 12-7
Agenda 12-9
Reporting on Recovery Appliance Components 12-10
Agenda 12-13
Reporting on Protected Databases 12-14
Summary 12-17
ix
ake
me
bel
l o
a k
li
l l o
(a cens
u
be e to
n
@ use
b a n
nio this
k
S t
ฺ c
ud
ng ent
)
s
om Guid
ha eฺ
an
o n - t r an
sfe r a b
le
1
Introduction r a ble
s fe
Oracle Zero Data Loss Recovery Appliance - t r an
o n
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.
a k
l o (a cens
bel li
em
ake
About This Course
After completing this course, you should be able to:

• Gain an understanding of the Recovery Appliance architecture
and components, including Delta Push and Delta Store

• Compare and understand the benefits of the Recovery
Appliance versus traditional backup systems
• Describe Recovery Appliance tools applicable for the
database administrator
r a ble
• Employ best practices to implement the Recovery Appliance
n s fe
in your data center
n - tra
no
• Deploy Replication for Disaster Recovery configurations
a
h a s
• Integrate the Recovery Appliance withmTape d e ฺ
) iLibraries for
Archival protection o
ฺc t G u
n g n
ankinterested
• Create reports for sharingbwith tu d e people
n i on is S
@ u e th
b e llo to us Copyright © 2016, Oracle and/or its affiliates. All rights reserved. 2
( a ak nse
e l lo l i ce
m b
e
ake
Zero Data Loss Recovery Appliance Overview 1 - 2

Course Overview
This course comprises the following lessons:

1. Introduction
2. The Zero Data Loss Recovery Appliance Solution

3. Recovery Appliance Functions and Features
4. Recovery Appliance Installation
5. Recovery Appliance Configuration
6. Protected Databases Configuration
7. Optimizations and Performance Tuning
r a ble
8. Disaster Recovery with Replication
n s fe
9. Media Management with Oracle Secure Backup
n - tra
10. Maintenance and Patching a no
11. Diagnostics and Troubleshooting h a s
m ) ideฺ
12. Reporting with BI Publisher
g ฺ co t Gu
Appendix A: RA View Reference
a n kn den
Appendix B: DBMS_RA PL/SQL Package
nio this Stu
nb Reference
u
@ use
l l o
be e to
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. 3
a k
l o (a cens
bel li
em
ake

Additional Resources
• Oracle Recovery Appliance Product Page

– https://www.oracle.com/engineered-systems/zero-data-loss-
recovery-appliance/index.html
• My Oracle Support
– http://support.oracle.com
• Oracle Technology Network Data Sheet
– http://www.oracle.com/technetwork/database/availability/recove
le
ry-appliance-ds-2297776.pdf rab fe
• Oracle Documentation Library n s
– http://docs.oracle.com/cd/E55822_01/index.htm n - tra
n o
h a sa
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li
m be
e
ake

How Prepared Are You?
To assess your preparedness to take this course, answer the

following questions. A “Yes” answer to both will confirm that you
are prepared to take this course.

• Are you familiar with Oracle Database?
• Are you familiar with the Oracle Recovery Manager (RMAN)
utility?
If you have answered “No” to either of these questions, take the
r a ble
appropriate Oracle education classes pertaining to Oracle fe
n s
n - tra
Database. You will find them here: http://education.oracle.com
• After you have completed the appropriate course,a noyou will be
ready to take this course. has ฺ ) ide
o m
c t Gu
g ฺ
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li
m be
e
ake

Introductions
Now that you have been introduced to the course, introduce

yourself to the other students and the instructor. Provide the
following information:
• Name
• Company affiliation
• Title, function, and job responsibility
• Experience related to topics presented in this course
r a ble
• Reasons for enrolling for this course n s fe
• Expectations from this course n - tra
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

2
Zero Data Loss Recovery Appliance Solution able

fer s
- t r an
o n
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake
Objectives
After completing this lesson, you should be able to:

• Identify the various Oracle Engineered Systems
• Explain the benefits of the Oracle Recovery Appliance

• List the Recovery Appliance hardware components
• List the Recovery Appliance software components
• Describe the Recovery Appliance implementation steps
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Agenda
• Engineered Systems: Overview

• Cloud-Scale Recovery Appliance Solution
• Oracle Recovery Appliance Hardware

• Oracle Recovery Appliance Software
• Implementation: Overview
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

What Are Engineered Systems?
Engineered Systems are purpose-built hardware and software

that are sold and supported as a single system. Engineered
Systems provide:
• Optimal performance and high availability
• Less deployment and operating risk
• End-to-end single-vendor support
• End-to-end unified monitoring of all
r a ble
components sfe n
n - tra
ENGINEERED TO an
o
s
WORK TOGETHERm) ha eฺ
ฺ c o Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
Building
b eand li a custom platform requires top talent. Traditional custom systems that
l operating
k e emused to run Oracle Database backup solutions do not deliver the performance or reliability
are
a that an Engineered System can deliver. The components that are used in custom database
backup systems are often not balanced and are often misconfigured, thus creating
bottlenecks and reducing overall system performance. Complex inter-component failure
modes are not tested and components are not engineered together as a unit in custom
database backup systems.
The hardware components, database software and libraries, operating system and device
drivers, firmware, network configuration, and all other components in the Engineered System
are optimized to work together. Oracle Engineered Systems are built to provide the highest
performance and are the most widely available platform for running Oracle software
components.
Engineered Systems include a full range of specialized services to simplify deployment,
maintenance, and support. Engineered Systems are best suited for companies that want the
highest system performance, rapid deployment, reliability, and security, with the lowest total
cost of ownership.

Engineered Systems: Oracle Zero Data Loss Recovery
Appliance X6-2
The Recovery Appliance is an engineered system of software, servers, storage,
and networking that offers a data protection solution that is completely integrated
with Oracle Database and eliminates data loss exposure for all databases
without impacting production environments.
• Integrated real-time Oracle redo transport eliminates data loss exposure.
• Incremental forever backups eliminate backup windows and database server
overhead.
• Continuous database-aware validation ensures data integrity and database
recovery to any point in time.
• Cloud-scale capacity with online expansion a ble
protects thousands of databases, enabling fe r
database protection as a service. an s
n - t r
• Autonomous tape archival enhances
tape resource utilization and investment a no
protection. h a s
• Unified control and automation simplifies m ) ideฺ
end-to-end management of the entire g ฺ co t Gu
n kn den
data protection lifecycle – from data creation
a
to disk to tape.
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Oracle’s l l o li Recovery Appliance (Recovery Appliance) is a ground-breaking data
eZero Data Loss
b
k e em solution that tightly integrates with the Oracle Database in order to address these
protection
a requirements head-on. It eliminates data loss exposure and dramatically reduces data
protection overhead on production servers. In addition, the Recovery Appliance scales to
protect thousands of databases, ensures end-to-end data validation, and implements full
lifecycle protection including disk backup, tape backup, and remote replication.

Oracle Zero Data Loss Recovery Appliance: Overview
Oracle Recovery Appliance Design Goals Tape Library

Protected Databases Recovery Appliance
Delta Push
• DBs access and send only changes Tape Archival
• Minimal impact on production • Offloads Tape Backup
• Real-time redo transport instantly • Tapes utilized all day
protects ongoing transactions
a ble
fe r
an s
n - t r
Protects all DBs in your Data Center
Delta Store
a no
• Stores validated, compressed DB changes on disk
• Petabytes of data
• Oracle 10.2-12c, any platform
h a s
• Fast restores to any point-in-time using deltas
• No expensive DB backup agents

m ) ideฺ
• Built on Exadata scaling and resilience
• Enterprise Manager end-to-end control
• Scales to thousands of DBs
g ฺ co t Gu
a n kn den Replicate to Remote
n b S t u Recovery Appliance
n i o i s
@ u e th
b e llo to us Copyright © 2016, Oracle and/or its affiliates. All rights reserved.
( a ak nse
Oracle’s e l
Zero
lo Data Loss
l i ceRecovery Appliance tightly integrates with the new Recovery
m b
k e e
Appliance specific capabilities in the Oracle Database and the Recovery Manager (RMAN)
a backup tool to provide data protection capabilities and performance that are not possible with
any other data protection solution.
The principal design goal for the Recovery Appliance is to eliminate the loss of critical
database data that is still possible by using existing data protection solutions.
The second design goal for the Recovery Appliance is to reduce backup-related processing
on production database systems to the absolute minimum, transmitting only the changed
data. With unnecessary backup processing eliminated, production systems can now focus on
their primary goal of serving business-critical workloads.
The third design goal for the Recovery Appliance is to provide a cloud-scale database
protection service for tens to thousands of databases in a data center.

The Recovery Appliance tightly integrates advanced data protection technologies with Oracle
Database to address these challenges head-on. It redefines the database protection
landscape with an innovative, never-before-seen approach that offers the following benefits:
• Data-Loss Elimination: As backups go, you are only protected up to your last backup.
In the event of a full restore and recovery operation, any changed blocks and archived
logs not backed up result in some amount of data loss spanning sometimes hours and
sometimes even days. The Zero Data Loss Recovery Appliance completely changes
this paradigm by supporting the reception of real-time redo blocks from all the hundreds
to thousands of databases across an enterprise, thereby dramatically reducing data loss
exposure from hours/days to sub-seconds.
• Zero-Impact Backups: With the Recovery Appliance, customers simply run the RMAN
in an incremental-forever strategy – that’s it. It means no more recurring full backups, no
more tape backups, and no more RMAN backup deletion and validation commands on
the production server. All expensive backup workloads are offloaded to the appliance,
where dedicated hardware and storage handle these tasks in the most efficient manner. a ble
fe r
• Restore to Any Point-in-Time: Each incremental block received is validated,
ans
- t r
compressed, and then indexed as a part of a new virtual full backup, which appears as a
n
no
normal full backup in the RMAN catalog. Essentially, this means validated full backups
a
h a s
are now available at any point in time, without ever incurring the cost of running full
backup operations.
m ) ideฺ
ฺ
• Cloud-Scale Protection: It has the ability togmassively co t G u at a cloud level by
scale
supporting from tens to thousands of databases
a n kn dacross
e n a data center, essentially
forming a private Data Protectionn
o b within
Cloud
S tuthe enterprise. There are several
technology components within
u i s
i Appliance that makes this possible.
n the Recovery
h
e t
• End-to-End Data Protection
e l lo@ o uVisibility:
s The Recovery Appliance provides a complete,
end-to-end view
a a kbinto thes e t protection lifecycle by using Oracle Enterprise Manager
data
l o ( –cfrom
Cloud Control n the time the backup is created by RMAN on the database to the
edisk,
b e
time l
it is l
stored i on on tape, and/or replicated to another Recovery Appliance in a
e e m remote data center.

a k • Secure Replication: Backups on a local Recovery Appliance can be easily and quickly
replicated via secure transport to a remote Recovery Appliance for protection against
disasters such as site outages or regional disasters. The replication topology can be
tailored to match the data center’s requirement.
• Autonomous Tape Archival: Optional tape integration sends backups directly from the
appliance to tape, offloading resource-intensive full-backup-to-tape operations from
production database servers. This also increases efficiency and reduces costs by
enabling tape systems to be used 24 x 7 instead of only during off-peak backup
windows.

• Scales Efficiently: Built on a massively scalable architecture, the Zero Data Loss
Recovery Appliance enables users to expand capacity seamlessly to petabytes of
storage, with no down time.
• Secures Enterprise Data: The ability to replicate change data in real-time to another
Zero Data Loss Recovery Appliance, along with end-to-end data validation, protects
business-critical data from site disasters or software/hardware malfunctions.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

Engineered System: Simple to Install, Manage, and
Maintain
Installation
Expertise
Optimization
Expertise
Build Your Own
Network
Administration
SAVINGS
Storage
Administration
Time
Cost
System
a b le
Administration fe r
an s
Enroll Protected
n - t r
Databases
a no
Disaster Recovery
h a s
Implementation
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Building l l o li system for an Oracle Database backup system requires a diverse
eyour own custom
b
k e emset and expertise that cover many hardware and software components, often which are
skill
a from different vendors. The initial deployment of a custom build-your-own system might
encompass tasks such as sizing, ordering, researching best practices, assembling, installing,
patching, and configuring the software. If you take into consideration all the activities that are
needed to put together a new system, it can take weeks to perform.
Because Oracle Recovery Appliance is an engineered system that has been purpose-built, it
greatly reduces the time and cost that are associated with the integration of the hardware and
software components that have been optimized for running the Oracle Database Backup
solution. All the hardware and software components are provided by Oracle, thereby enabling
customers to benefit from streamlined single-vendor support. With Oracle Recovery Appliance
X6-2, a single DBA can deploy a high-availability clustered database backup system in about
one hour.

Agenda


a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Oracle Recovery Appliance Cloud-Scale Solution
Private Data Protection Cloud supports up to thousands of databases

across a data center.
• Policy-Based Data Protection: Groups databases by the recovery service

tier based on Protection Policies
• Database Aware Space: Database-oriented space management done
according to recovery window goals, and not according to storage-volume
levels
• Scalable Architecture: Adding compute and storage resources in a simple
and modular fashion, and not in an “add more devices” approach
a ble
fe r
an s
n - t r
no
Fibre Channel
connectivity scales
a
with capacity
s
Connectivity scales h a
) ideฺ Tape Archival
with capacity m
co Replica
Remote
u
g ฺ t G
a n kn den
nb s Stu
Protected Databases
n i o i
u e t h
e l lo@ o us Copyright © 2016, Oracle and/or its affiliates. All rights reserved.
a a kb se t
l o ( cen
e l
OraclebRecovery li
Appliance easily protects all databases in the data center by using a
k e em scalable service. The following technology components within the Recovery

massively
a Appliance make this possible:
Policy-Based Data Protection: A protection policy that defines recovery window goals that are
enforced for each database for backups to the Recovery Appliance or a tape device, in
addition to improving manageability
Database Aware Space: Manages backup storage space according to the recovery window
goal for each protected database
Scalable Architecture: Can scale to accommodate increases in backup traffic, storage usage,
and the number of databases

Agenda


a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Oracle Recovery Appliance Hardware Specifications
U42 U42
Storage Server (18)
U41 U41
ZDLRA X4-2 ZDLRA X5-2 ZDLRA X6-2 U40

U39
Storage Server (14)
U40
U39
U38 U38
Storage Server (13)
Processor Intel E5-2697 v2 Intel E5-2699 v3 Intel E5-2699 v4 U37 U37
U36 U36
Storage Server (12)
U35 U35
Cores/Node 2x12 (48) 2x18 (72) 2x22 (88) U34

Storage Server (11)
U34
U33 U33
(Total) U32 U32
Storage Server (10)
U31 U31
Memory/Node 256GB (512GB) 256GB (512GB) 256GB (512GB) U30
U29
Storage Server (9)
U30
U29
(Total) DDR3 DDR4 DDR4 U28
Storage Server (8)
U28
U27 U27
U26 U26
Boot disks 4x600GB 4x600GB 4x600GB U25
Storage Server (15)
U25
U24 U24
Storage Server (16)
U23 U23
U22 36-Port IB Switch U22
Networking 4 x 10GbE Copper 4 x 10GbE Copper 4 x 10GbE Copper
le
U21 Cisco 4948E-F U21
NICs (opt fiber) NICs (opt fiber) NICs (opt fiber) U20
U19
36-Port IB Switch U20
r a
U19 b
Interconnect 2 InfiniBand QSFP+ 2 InfiniBand QSFP+ 2 InfiniBand QSFP+
U18
Storage Server (17)
s fe U18
an
U17 Compute Server U17
U16
U15
n - t r Compute Server
Storage Server (7)
U16
U15
no
U14 U14
FC SAN adapter 2 x 16 Gb QLogic 2 x 16 Gb QLogic 2 x 16 Gb QLogic U13 U13
sa
Storage Server (6)
(optional) FC SFP+ PCIe FC SFP+ PCIe FC SFP+ PCIe U12 U12
h a
) Servers ฺ
U11
U10
Storage Server (5)
U11
U10
Storage Servers 14 Storage Servers 18 Storage Servers 18 Storage
m i d e U09
Storage Server (4)
U09
co t Gu
U08 U08
(max conf) (168 Data Drives) (216 Data Drives) (216 Data Drives)
g ฺ U07
Storage Server (3)
U07
knto dMulti n Rack (up to U06 U06
Storage Multi Rack (up to

a n
Multi Rack (up e U05
Storage Server (2)
U05
nb18) s Stu
U04 U04
Expansion 18) 18) U03 U03
n i o i
U02
Storage Server (1)
U02
u e t h U01 Solid filler U01
a a kb se t
l o ( cen
Zero Datab l li Appliance is a backup solution that has been specifically designed
e Loss Recovery
m Oracle Database files. The Recovery Appliance can also recover any database to
etoeprotect
ak any point in time as well as automate tape archival of data to protect against site disasters.
Virtual full backups are possible to remote backup appliances. The Recovery Appliance is a
single, standalone, rack solution that contains Compute Servers and Storage Servers, all
housed within a customized Sun Rack 1242 enclosure. Compute Servers are based on the
Oracle Server X6-2 (8-drive) server and Storage Servers are based on the Oracle Server X6-
2L (12-drive) server. Engineered Systems, like this Recovery Appliance, are designed such
that specifically chosen hardware and software components are pre-integrated in the rack to
operate as a cohesive whole, reducing the cost and complexity of IT infrastructures while
increasing productivity and performance compared to standalone systems.

Oracle Recovery Appliance X6-2 Hardware Details
Components Component Base Rack Full Rack
Breakdown
Servers Included (2) X6-2 Compute Servers (total) (2) X6-2 Compute Servers
(total)
Compute Servers (88) Cores (total) (88) Cores (total)

Processor Cores
Memory 512 GB Memory (total) 512 GB Memory (total)
Servers Included (3) X6-2L Storage Servers (18) X6-2L Storage Servers
(upgradeable to 18) (total)
Raw Capacity 288 TB 1.73 PB

Storage Servers
Usable Capacity ~94 TB 1st Base Rack ~580 TB 1st Full Rack
a ble
~32 TB each additional Storage Server
fe
~580 TB each additional Full Rack r
~94 TB each additional Base Rack
a n s
Internal Network (2) 36-port InfiniBand Switch with (2) 36-portn
r
-t Switch
InfiniBand
QSFP Ports n o
with QSFP Ports
(Leaf Switches)
a
s (1) 48-Port Ethernet Switch
(1) 48-Port Ethernet Switchha
Switches
Management Switch
m ) ideฺ
Multi-Rack Network ฺ coSwitcht with
(1) 36-Port InfiniBand
g G u (1) 36-Port InfiniBand Switch
(Spine Switch -
n k n en
QSFP Ports with QSFP Ports
optional)
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
The hardware
b li
el components of Zero Data Loss Recovery Appliance X6 consists of (2) 1U
em Server X6-2s (Compute Servers) and multiple Oracle Server X6-2Ls (Storage
eOracle
ak Servers). The Recovery Appliance includes InfiniBand and Ethernet switches, and is available
configured as a full rack or as a base rack that can be upgraded with additional Storage
Servers when required.

The main components that make up Zero Data Loss Recovery Appliance X6 include:
• Compute Servers: Compute Servers handle all processing required by the application
software, except for any storage- or disk-related processing (which is handled by Storage
Servers). They are also used to provide host network connectivity to the Recovery
Appliance.
• Storage Servers: Storage Servers are highly optimized for use with database software
and are employed to dramatically accelerate database storage and to speed I/O
operations. Database processing is pushed to Storage Servers, allowing all the disks to
operate in parallel, reducing CPU consumption on Compute Servers.
• Switches: InfiniBand and Ethernet switches are used for the various internal and external
networks as well as for the management of the Recovery Appliance. These switches
include:
- Leaf: (2) 36-port InfiniBand switches used for high-speed internal communication
between internal hardware components (Compute Servers, Storage Servers, fabric
a ble
interconnects, and so on)
fe r
ans
- Spine: (1) A 36-port InfiniBand switch used for high-speed external communication
between multiple Recovery Appliances (optional) n - t r
o
an
- Management: (1) A 48-port 1 GbE switch used to provide easy management of all
s
ha eฺ
internal hardware components (Compute Servers, Storage Servers, and so on)
)
• Networks
ฺ c om Guid
n k ng ent
- Private: (2) 36-port InfiniBand switches (commonly referred to as Leaf Switches)
n b a
S t ud
used for high-speed internal communication between internal hardware components
nio this
(Compute Servers, Storage Servers, system disk, and fabric interconnects)
u
l l o @ use
- Ingest: A required 10 GbE network that connects the protected Oracle Database
a k be e to
servers to the Recovery Appliance for backup within the same data center (also
l o (a cens
referred to as the backup network). This is a high-speed, private Ethernet network
using (2) 10 Gb connections to each of the (2) Compute Servers in the rack. These
bel li
connections may be configured as active/passive or as active/active. Compute
e m Servers support channel bonding, providing higher bandwidth and availability.
ake - Replication: An optional 10 GbE network used to connect local Recovery
Appliance(s) with remote Recovery Appliance(s)
• Fibre Channel (FC) SAN: An optional 16 Gb Fibre Channel network used to connect the
Recovery Appliance to the datacenter storage area network (SAN) for backups to tape

• Multi-Rack Interconnect: (1) A 36-port InfiniBand switch (commonly referred to as a
Spine Switch) used to connect the Recovery Appliance to up to (18) total Recovery
Appliances. The Spine Switch is optional and can be ordered for factory installation.
• Management: (1) A 48-port 1 GbE switch used to provide easy management of all
internal hardware components (Compute Servers, Storage Servers, fabric interconnects,
etc.) in the Recovery Appliance. This 1 GbE network connects to the customer’s existing
management network and cannot be used for client or application traffic.
• Rack: A single, standalone rack based on Sun Rack 1242 that houses and provides AC
power to all internal components (Compute Servers, Storage Servers, and switches)
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

Oracle Recovery Appliance X6-2: Compute Server
Oracle Server X6-2: Designed for Clustered Databases

• Compact 1U enterprise-class server
• 2 Intel Xeon processor E5-2699 v4 processors

• 44 cores (88 threads)
• 256 GB memory (8 x 32GB DDR4-2400 DIMMs)
• 2.4 TB HDD capacity (4 x 600GB 10000 rpm SAS-3 HDs)
• 4 PCIe Gen 3 I/O expansion slots
– 1 Dual-port InfiniBand QDR (40 Gb/sec) PCIe HCA (internal Private Network)
a ble
– 1 Dual-port 10 GbE SFP+ PCIe adapter (10 GbE Private Network) fe r
an s
• r
4 10gBase-T ports (1 GbE Management and 10 GbE Private Networks)
- t
• 2 Redundant hot-swappable power supplies n on
a
• as(FC SAN network)
0 or 1 Dual-port 16 Gb QLogic FC SFP+ PCIe adapter
) h eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
Compute
b l
eServers li
included with the Zero Data Loss Recovery Appliance X6 are Oracle Server
k e emservers, are only available in limited configurations, and optimized for the Zero Data
X6-2
a Loss Recovery Appliance X6. They handle all the processing required by the database
application software, except for any storage- or disk-related processing (which is handled by
Storage Servers). They are also used to provide host network connectivity to the Zero Data
Loss Recovery Appliance X6.
The dual-port 16 Gb QLogic FC SFP+ PCIe adapters are not installed at the factory and
shipped for installation with the Recovery Appliance. No other cards (FC or otherwise) are
allowed in the Recovery Appliance and the firmware version of the QLogic adapters cannot
be changed.

Oracle Recovery Appliance X6-2: Storage Server
Oracle Server X6-2L: Designed for Enterprise Storage

• Compact 2U enterprise-class server
• 2 Intel Xeon processor E5-2630 v4 processors

• 20 cores (40 threads)
• 128 GB memory (8 x 16GB DDR4-2400 DIMMs)
• 96 TB HDD capacity (12 x 8TB 7200 rpm SAS-3 HDs)
• 6 PCIe Gen 3 I/O expansion slots
– 12.8 TB SSD capacity (4 x 3.2 TB Sun Flash Accelerator F320 PCIe adapters
(NVMe))
a ble
– 1 Dual-port InfiniBand QDR (40 Gb/sec) PCIe HCA (internal Private Network) fe r
– 1 Dual-port 10 GbE SFP+ PCIe adapter (10 GbE Private Network) an s
n - t r
• 1 Internal SAS-3 12 Gb RAID HBA with 1 GB Supercap-backed
o cache
n
• s a Networks)
4 10gBase-T ports (1 GbE Management and 10 GbE Private
h a
• 2 Redundant hot-swappable power supplies ) eฺ
om Guid
ฺ c
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
Storage
b l
eServers li with the Zero Data Loss Recovery Appliance X6 are Oracle Server
included
k e em servers, are only available in limited configurations, and optimized for the Zero Data
X6-2L
a Loss Recovery Appliance X6.
Storage Servers are the building blocks of the Recovery Appliance. They free up CPU
resources in Compute Servers and are highly optimized for use with database software to
dramatically accelerate database processing and to speed I/O operations. Database
processing is pushed to Storage Servers, allowing all disks to operate in parallel, thereby
reducing CPU consumption on Compute Servers.
The Recovery Appliance uses Storage Servers as the main storage for backups. This is the
destination for protected database backups. Redo staging areas for redo streams reside on
Recovery Appliance Storage Servers disks too. To expand past all the resources provided in
a Zero Data Loss Recovery Appliance X6 (Base Rack), customers can order additional
Storage Servers when required.

Oracle Recovery Appliance X6-2: InfiniBand Switches
Sun Datacenter InfiniBand Switch 36: Designed for High-Performance

Storage
• 1RU form factor

• 36 ports supporting Quad Data Rate (QDR) InfiniBand
• Embedded management module for InfiniBand fabric management
• Data throughput: 2.3 Tb/sec
• Port-to-port latency: 100 ns QDR
• Redundant power and cooling
r a ble
• Leaf switches to interconnect Compute Servers and Storage Servers sfe
• Spine switch to interconnect multiple Oracle Recovery Appliances
- t r an
(Multi-
Rack) non
h a sa
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
InfiniBand l l o i used for high-speed internal communication among components and
e switches lare
b
k e em
for inter-rack communication.
a There are (2) Leaf Switches included with each Zero Data Loss Recovery Appliance. The
Leaf Switches are Sun Datacenter InfiniBand Switch 36 and are used to interconnect the
compute servers and storage servers to provide extremely scalable, reliable and high-
performance connectivity among all internal components within the Zero Data Loss Recovery
Appliance.
Leaf Switches also connect to Spine Switches when multiple Recovery Appliances are
connected together. Client connectivity is not recommended via InfiniBand connections. For
best performance, it is highly recommended that you use 10 GbE for all client connectivity.
The (2) ports on each InfiniBand HCA within each Compute and Storage Server connect to
different Leaf Switches to provide redundant communications between Compute Servers and
Storage Servers.
The Spine Switch is not included with the Zero Data Loss Recovery Appliance X6 by default.
A spine switch is required for interconnecting the Zero Data Loss Recovery Appliance X6 with
other Recovery Appliances. The Spine Switch used for interconnecting Recovery Appliances
together is the Sun Datacenter InfiniBand Switch 36, which is used to connect up to (18)
Recovery Appliances together.

Oracle Recovery Appliance X6-2: Management Switch
Cisco Catalyst 4948E-F

• 1RU form factor
• (48) 10/100/1000 Mb/sec RJ45 ports

• (4) 1/10 Gb/sec SFP+ Uplink ports
• Redundant power
• Hot-swappable fans
• Back-to-front airflow a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
There bis e l l o li
one (1) Management Switch included with each Zero Data Loss Recovery Appliance
k e emThe management switch is the Cisco Catalyst 4948E-F switch and is used to connect to
X6.
a all components within the Zero Data Loss Recovery Appliance X6 on a 1 GbE management
network.
The management switch is used for network management tools. It is connected to all
components within the Recovery Appliance. This switch can be replaced by a customer-
supplied switch.

Agenda


a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Oracle Recovery Appliance Base Software
Oracle Recovery Appliance: Compute Server

• Oracle Linux 6
• Hardware drivers
• RAC Database 12c with options
• Grid Infrastructure 12c: Oracle Clusterware and ASM
• Pre-Bundled Media Manager for tape offsite
• Oracle Enterprise Manager 12c Cloud Control Agent
• Zero Data Loss Recovery Appliance Software
• QLogic FC SFP+ PCIe adapter drivers and tools (optional)
• HealthCheck (exachk)
a ble
Oracle Recovery Appliance: Storage Server fe r
an s
• Oracle Linux 6
n - t r
•
•
Hardware drivers
Oracle Exadata Storage Server Software a no
s
Oracle Recovery Appliance: Additional Components ) ha ฺ
m i d e
• PDU firmware and HTML interface
Cisco Catalyst 4948 Ethernet Switch IOSkng
ฺco t Gu
•
n d e n
• Sun Datacenter Infiniband Switch 36 a
nb s Stu
software
n i o i
u e t h
a a kb se t
l o ( cen
The Zero
b l i
eData LosslRecovery Appliance X6 base image includes Oracle Enterprise Linux as
them
kee
operating system for both Compute Servers and Storage Servers. It also includes the
a required firmware and drivers. Customers cannot modify the individual components of the
image package.
Compute Servers Integrated/Pre-Installed Software include:
• Oracle Linux 6
• Hardware drivers, management pack, and firmware drivers for various components
Restricted use of Oracle Database Enterprise Edition on the Recovery Appliance and the
following options:
• Oracle Real Application Clusters
• Oracle Advanced Compression Option
• Oracle Advanced Security Option
• Oracle Partitioning

• Oracle Grid Infrastructure with Oracle Clusterware and Oracle ASM software
• Pre-Bundled Media Manager
• Oracle Enterprise Manager Packs, with restriction for use only with the Recovery
Appliance
- Oracle Diagnostics Pack
- Oracle Database Lifecycle Management Pack for Oracle Database
• Zero Data Loss Recovery Appliance Software (RA Automation RPM)

• HealthCheck tool
• Oracle Integrated Lights Out Manager (Oracle ILOM)
• Oracle Intelligent Platform Management Interface (IPMI)
• QLogic PCIe adapter drivers and tools (optional)
Storage Servers Integrated/Pre-Installed Software:
• Oracle Linux 6 a ble
fe r
• Hardware drivers, management pack, and firmware drivers for various components
ans
• Oracle Exadata Storage Server Software (cell server) n - t r
• Oracle Integrated Lights Out Manager (Oracle ILOM) a no
• Oracle Intelligent Platform Management Interface (IPMI) h a s
m ) ideฺ
g ฺ co Versions
Check the following support note for additional information aboutusoftware and supported
t G (Doc ID 1927416.1)
k n
versions: Zero Data Loss Recovery Appliance Supported n
Also check the following support note forbinstalling u de Third-party software on Oracle
an tadditional
n S
Recovery Appliance: Zero Data Loss
u nioRecovery
h i s Appliance - Installing Third-Party Software
t (Doc ID 2014361.1)
l o @
and Modifying Internal Appliance e
Software
s
k b el to u
( a a nse
e l lo l i ce
m b
a kee

Oracle Integrated Lights Out Manager
(Oracle ILOM)
Embedded Service Processor
• Simplifies maintenance and management by allowing
full control of the server remotely

• Includes a built-in fault diagnosis engine that proactively
detects hardware problems in real time and sends notifications
• Provides a simple web interface for managing and configuring the
server
• Supports standard management interfaces, such as SNMP trap
notifications and IPMI 2.0 r a ble
n s fe
• Connects with Oracle Enterprise Manager 12c Cloud Control tra n -
n o
h a sa
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
OraclebIntegrated li Out Manager (ILOM) provides advanced service processor (SP)
Lights
k e em and software that you can use to manage and monitor your Oracle hardware.
hardware
a Oracle ILOM is pre-installed on all of Oracle Recovery Appliance Compute Servers, Storage
Servers, and Sun Datacenter Infiniband Switches. It is a vital management tool in the data
center.
Oracle ILOM enables you to experience a single, consistent, and standards-based service
processor across all Oracle Recovery Appliance servers and switches. This means you will
have:
Single, consistent system management interfaces for operators
• Support for rich and standard protocols
• Oracle Enterprise Manager 12c Cloud Control management tools and interfaces
• Integrated system management functions at no extra cost
The Oracle ILOM SP runs its own embedded operating system and has a dedicated Ethernet
port, which together provide out-of-band management capability. Oracle ILOM automatically
initializes as soon as power is applied to the server. It provides a full-featured, browser-based
web interface and has an equivalent command-line interface (CLI). An industry-standard
SNMP interface and an IPMI interface are also available.

Agenda


a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Implementation Workflow
• Separation of duties
• Prerequisites for recovery appliance administration
• Tools for recovery appliance administration

• Planning for recovery appliance
• Setup and configuration for recovery appliance
• Maintenance tasks for recovery appliance
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
A typical l l o li
eRecovery Appliance environment includes personnel with the following roles:
m b
a kee• Cloud Control administrator: The application administrator with this role administers
Oracle Enterprise Manager Cloud Control (Cloud Control).
• Recovery Appliance administrator: This administrator manages Recovery Appliance.
• Protected Database administrator: This administrator is responsible for configuring
backups to the Recovery Appliance by using the virtual private catalog account
assigned by the Recovery Appliance administrator.
You must work with the Oracle field engineers to install and set up an Oracle Recovery
Appliance.
Use the following tools to complete administrative tasks for the Recovery Appliance:
• Cloud Control: Cloud Control is a system management tool with a graphical UI that
enables you to manage and monitor the Recovery Appliance and its protected
databases. This is the preferred UI for Recovery Appliance tasks.
• SQL*Plus: SQL*Plus is a command-line tool that enables you to run DBMS_RA
program units, and query recovery catalog views. You use SQL statements and Oracle-
supplied PL/SQL packages to complete these tasks in SQL*Plus.

You must complete the following general tasks for planning:
• Task 1: Group protected databases into tiers
• Task 2: Determine the recovery requirements for each database tier
• Task 3: Determine the recovery requirements for each protected database
• Task 4: Determine access requirements for the Recovery Appliance
• Task 5: Create a backup migration plan for the Recovery Appliance
• Task 6: Review Cloud Control reporting and monitoring tools

You must complete the following general tasks for setup and configuration:
• Task 1: Create Cloud Control user accounts
• Task 2: Create a protection policy for each database tier
• Task 3: Configure access on Recovery Appliance for protected databases
• Task 4: Configure protected databases (for DBAs)
a ble
• Task 5: Migrate legacy backups to the Recovery Appliance (for DBAs)
fe r
• Task 6: Create copy-to-tape schedules to meet recovery requirements
ans
• Task 7: Configure the Recovery Appliance replication n - t r
o
an
Typically, you must perform the following tasks for maintenance:
s
• Task 1: Monitor activity on the Recovery Appliance ) ha eฺ
• Task 2: Monitor backup jobs (for DBA) ฺ c om Guid
k ng ent
• Task 3: Generate and review reports on the Recovery Appliance
n
n b a
• Task 4: Restart the Recovery Appliance S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Summary
In this lesson, you should have learned how to:

• Identify various Oracle Engineered Systems
• Explain the benefits of the Oracle Recovery Appliance

• List the Recovery Appliance hardware components
• List the Recovery Appliance software components
• Describe the Recovery Appliance implementation steps
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

3
Recovery Appliance Functions and

Features a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake
Objectives

• Describe data protection challenges in the modern enterprise
• Explain the Recovery Appliance solution

• List the different Recovery Appliance functions and features
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
In this b l l o
e you learn
lesson, li about the role that the Recovery Appliance has in your database
m infrastructure. In addition, you will get in-depth understanding of the Recovery
kee
protection
a Appliance solution covering hardware and software components functionality and features.

Agenda
• Today’s Data Protection Goals and Challenges

• Backup Appliances Are Not Designed for Database
• Zero Data Loss Recovery Appliance Functionality and

Features
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Why CIOs Care
Top Backup and Recovery Challenges

Long backup windows –

54% backups taking too long
Impact
Growing business requirements

51% for more reliable, efficient
disaster recovery
14% of all backup jobs fail
48% Long restore/

19% of all restores are unsuccessful
a ble
recovery times
fe r
an s
48%
Total cost of
n - t r
backup solutions
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den (Percent of respondents, n=1,200 )
i o nb s Stu
Source: IDG Research Services, May 2012; ESG Group - Trends in Data Protection Modernization in 2012
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
The fundamental li
problems with today’s database protection solutions stem from:
b
ak ee•m Backup and recovery methods that are based on decades-old nightly backups
paradigm, where up to a day’s worth of data can be lost on every restore
• High backup overhead on production servers and networks for processing all database
data during backups, whether it has changed or not
• Ever-increasing backup windows due to non-stop data growth
• Backup appliances that cannot scale to protect the hundreds to thousands of databases
in the data center
• Poor visibility and control of the full data protection lifecycle, from disk to tape to
replication

Why Customers Care
Data-Loss Consequences Impact

Reputational damage to the

34% company
of consumers would no longer
deal with a company if they were
23%
Not being able to recover the lost 84% informed that the company had lost
files their Personal Information
23% Access to confidential data of companies suffering from a
catastrophic data loss do not survive
10%
Personal penalties such as
fines or jail time
19% - 43% never reopen and 51% close
within two years
6% The loss of your job
a ble
small firms that experience a
fe r
3% Company Penalties 7/10 major data loss go out of
an s
business within a year
n - t r
2% None
a no
(Percent of respondents, n=200 )
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
Source: IDG Research Services, May 2012; ESG Group - Trends in Data Protection Modernization in 2012
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l o
eprovides alilist of customer’s data loss consequences and the serious impacts in
This slide l
b
em decisions and results.
ebusiness
ak

Inadequate Data Recovery = Big Business Impacts
European Cloud Infrastructure Provider 5-day

• Storage array failed outage
Global Specialty Retailer

8-day
• Disk failure and mirrored disk failure; restore from local backup failed; outage
restore by using copy at DR site also failed
a ble
fe r
a n s
U.S. State Government r 5-day
n-t outage
• SAN memory failure, problem mirrored to standby SAN
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Some b e l l
examples
o li the number of outage days due to inadequate data recovery
showing
k e em
implementations
a

Key Database Protection Goals
Business Goals
• Never lose critical business data
• No impact on business applications
IT Goals
• Ensure database-level recoverability
• Centralized service to protect all databases a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
Current Backup Solutions Fail g o GuThese Goals
toฺcAchieve
n k n ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
Currentbbackup li fail to achieve business and Information Technology department
el solutions
k e em
goals.
a

Agenda


Features
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Backup Appliances Are Not Designed for Database
Data-Loss Exposure Daily Backup

Window
Lose all data since
last backup Large performance
impact on production
Poor Database
Recoverability
Many Systems
to Manage a ble
fe r
Many files are copied, Scale by deploying an s
but the protection state more backup
n - t r
of the database is
unknown a no appliances
h a s
m ) ideฺ
g ฺ co t Gu
Treat Databases as Just a n kn todPeriodically
Files e n
Copy
n b S t u
u nio this
l l o @ use
be e to
a k
l o (a cens
The protection
b li described in "Traditional Database Backup Techniques" are not
el strategies
em to solve the challenges created by this transformation. Enterprises find themselves
edesigned
ak without a consistent backup and recovery strategy.
The following shortcomings are common to most or all of the traditional backup techniques:
• Data loss exposure: A database is only recoverable to its last valid backup, which may
have occurred hours or days ago. In addition, storage snapshots and third-party
appliances cannot validate Oracle data blocks, and so cannot detect Oracle block-level
corruptions.
• Daily backup windows: As database sizes increase, the lengths of the backup
windows also increase, creating additional load on production systems. Critical
databases cannot afford to be deprived of resources used for daily backups and related
maintenance activities.
• Poor Database Recoverability: Because most third-party backup snapshot and
Recovery Appliances lack Oracle integrated data block and database backup validation,
restore and recovery operations tend to fail. Such failures result in extended down time
and potentially larger data loss.

• Many Systems to Manage: As the number of databases increases exponentially, the
ease of manageability decreases. Backup scripts proliferate and change. New DBAs may
struggle to understand what the legacy scripts do. Questions about the status, backup
location, and recovery point objective (RPO) of a particular database become harder to
answer.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

Oracle Deduplication and Validation Challenges
Understanding the backup stream is critical in identifying redundant

data.
• The file system data format does not change in the backup process.
– Ideal for storage deduplication technologies
– Deduplication ratio claims of 10-50x
• RMAN passes a prepackaged backup set.
– The backup stream/block format is Oracle proprietary.
– RMAN backup stream is largely opaque to external backup
a b le
applications and storage. fe r
n s
n - tra
— Full backups (production impact) are needed for deduplication.
– Minimal deduplication versus file system data

a no
• s interpret
Storage technologies cannot validate Oracle blocksanor
h
transactional change data.
c m)
o Gui deฺ
ฺ
ng May t Not Work With
What Works With File Systems n k e n
Oracle b a
(RMAN)
n S t ud
Backups
u nio this
l l o @ use
be e to
a k
l o (a cens
Current
b el
disk-based li protection solutions impose large loads on production systems. This
data
k e em continuously increases as databases grow.

impact
a Some key challenges are:
• Most deduplication appliances require periodic full backups. Full backups read the entire
database that has a heavy impact on production storage, servers, and networks.
• When source-side deduplication is used to reduce network requirements, this has the
adverse effect of imposing high CPU and memory loads on production servers.
• During the recovery phase, incremental backups must be applied to the restored
datafiles before the database can be opened. This process can significantly prolong
recovery time, as the incremental can span multiple days of changes, and because the
apply operation usually runs across the network.
• File system validation tools do not understand internal Oracle Database data block and
redo log entry formats, avoiding deep levels of data validation.

Agenda


Features
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Recovery Appliance:
Unique Benefits for Business and IT
Minimal Impact
Eliminate Data Loss
Backups
Real-time redo
Production databases
transport provides
only send changes; all
instant protection of
backup and tape
ongoing transactions
processing offloaded
a ble
Database-Level Cloud-Scale Protection
fe r
Recoverability
an s
End-to-end reliability,
Easily protect all
n - t r
databases in the data
visibility, and control of
a no center by using a
databases, and not of
h a s massively scalable
disjointed files
m ) ideฺ service
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Oracle’s l l o li Recovery Appliance tightly integrates advanced data-protection
eZero Data Loss
b
k e em
technologies with the Oracle Database to address these challenges head-on. The Recovery
a Appliance redefines the database protection landscape with an innovative, never-before-seen
approach that:
• Eliminates Data Loss: As backups go, you are only protected up to your last backup –
in the event of a full restore and recovery operation, any changed blocks and archived
logs not backed up result in some amount of data loss, spanning sometimes hours and
sometimes even days. The Recovery Appliance completely changes this paradigm by
supporting the reception of real-time redo blocks from all the hundreds to thousands of
databases across the enterprise, dramatically reducing data loss exposure from
hours/days to sub-seconds.
• Has Minimal Impact Backups: With the Recovery Appliance, customers simply run the
Recovery Manager (RMAN) in an incremental forever strategy – that’s it, no more
recurring full backups, no more tape backups, and no more RMAN backup deletion and
validation commands on the production server. All expensive backup workloads are
offloaded to the appliance, where dedicated hardware and storage handle these tasks in
the most efficient manner.

• Offers Database-Level Recoverability: Each incremental block received is validated,
compressed, and then indexed as a part of a new virtual full backup, which appears as a
normal full backup in the RMAN catalog. Essentially, this means validated full backups are
now available at any point in time, without ever incurring the cost of running full backup
operations.
• Offers Cloud-Scale Protection: Ability to massively scale at a cloud level by supporting
tens to thousands of databases across a data center, essentially forming a private Data
Protection Cloud within the enterprise. There are several technology components within
the Recovery Appliance that makes this possible.
In addition to these unique benefits, the Recovery Appliance also provides:
• End-to-End Data Protection Visibility: The Recovery Appliance provides a complete,
end-to-end view into the data protection lifecycle by using Oracle Enterprise Manager
Cloud Control – from the time the backup is created by RMAN on the database to the time
it is stored on disk, on tape, and/or replicated to another Recovery Appliance in a remote
a ble
data center.
fe r
• ans
Secure Replication: Backups on a local Recovery Appliance can be easily and quickly
n - t r
replicated via secure transport to a remote Recovery Appliance for protection against
no
disasters such as site outages or regional disasters. The replication topology can be
a
tailored to match the data center’s requirement. h a s
• Enables Autonomous Tape Archival: Optionalo m
tape i d eฺsends backups directly
)integration
g ฺ
from the appliance to tape, offloading resource-intensive
u
c t Gfull-backup-to-tape operations
k n n
from production database servers. This
b axn7,also de efficiency and reduces costs by
increases
t u
enabling tape systems to be used
i o n24
s S of only during off-peak backup windows.
instead
• Scales Efficiently: Built on
n
ua massively i
th scalable architecture, the Zero Data Loss
Recovery Appliance o @
llenables users
e
us to expand capacity seamlessly to petabytes of
b e
k downstime. t o
storage, withano
( a n e
e
o Enterprise
lic Data: The ability to replicate change data in real-time to another
b ellData
• Secures
Zero Loss Recovery Appliance, along with end-to-end data validation, protects
m
a kee business critical data from site disasters or software/hardware malfunctions.

Oracle Zero Data Loss Recovery Appliance: Overview
Tape Library
Oracle Recovery Appliance Design Goals
Protected Databases Recovery Appliance
Delta Push
Tape Archival
• DBs access and send only changes • Offloads Tape Backup
• Minimal impact on production • Tapes utilized all day
• Real-time redo transport instantly
protects ongoing transactions
a ble
fe r
an s
n - t r
Protects all DBs in your Data Center
Delta Store
a no
• Stores validated, compressed DB changes on
• Petabytes of data
• Oracle 10.2-12c, any platform
disk
h a s
• Fast restores to any point-in-time using deltas
• No expensive DB backup agents
m ) ideฺ
• Built on Exadata scaling and resilience
• Scales to 1000s of DBs
co t Gu
• Enterprise Manager end-to-end control
g ฺ
a n kn den Replicate to Remote
n b S t u Recovery Appliance
n i o i s
@ u e th
( a ak nse
Oracle’s e l
Zero
lo Data Loss
l i ceRecovery Appliance tightly integrates with new Recovery Appliance-
m b
k e e capabilities in the Oracle Database and the RMAN backup tool to provide data
specific
a protection capabilities and performance that are not possible with any other data protection
solution.
The principal design goal for the Recovery Appliance is to eliminate the loss of critical
database data that is still possible by using existing data protection solutions.
The second design goal for the Recovery Appliance is to reduce backup-related processing
on production database systems to the absolute minimum, and transmitting only the changed
data. With unnecessary backup processing eliminated, production systems can now focus on
their primary goal of serving business-critical workloads.
The third design goal for the Recovery Appliance is to provide a cloud-scale database
protection service for tens to thousands of databases in a data center.

Key Architecture Components
• Based on Oracle Exadata Database

Machine X6-2 HC with embedded 2-
node RAC+ASM database

– Serves as the central RMAN
Recovery Catalog, storing all
backup metadata Protected
Databases Recovery
– Delta Store ‘backup data’ configured Catalog
in separate ASM disk group, stored
by using a compressed format
• Pre-bundled Media Manager for tape bl e
fe r a
offline s
n
a Store
trDelta
• 16Gb QLogic Fiber Cards supported n -
for tape connectivity
a no
• RMAN + Recovery Appliance h a s
Backup Module on protected m ) ideฺ
databases g ฺ co t Gu
n n
• EM Cloud Control 12c Release a4n+k ude
b t
u nion s S
Recovery Appliance plug-in
h i
e t
a a kb se t
l o ( cen
l
e Appliance
The Recovery
b li receives incremental backups and real-time redo from the protected
em
edatabases. The appliance contains the Recovery Appliance metadata database. This
ak database includes the following components:
• The RMAN recovery catalog, which is subdivided into multiple virtual recovery catalogs
• One or more storage locations. Recovery Appliance storage contains the delta store,
which includes multiple delta pools.
Media Manager software is pre-bundled with the Recovery Appliance to archive backups to a
tape library.
Multiple protected databases send backups and real-time redo to the Recovery Appliance.
Protected databases can run on different releases of Oracle Database.
Oracle Enterprise Manager Cloud Control running on a separate server in the environment
helps administrators manage all Recovery Appliances, protected databases, and tape devices
in the Recovery Appliance environment.

Recovery Appliance:
Minimal Impact
Eliminate Data Loss
Backups
Real-time redo
transport provides
backup and tape
a ble
Database Level Cloud-Scale Protection
fe r
Recoverability
an s
Easily protect all
n - t r
a no center using massively
h a s scalable service
disjointed files
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Data Loss:
Eliminates li As backups go, you are only protected up to your last backup – in the
b
k e em of a full restore and recovery operation, any changed blocks and archived logs not
event
a backed up result in some amount of data loss, spanning sometimes hours and sometimes
even days. The Recovery Appliance completely changes this paradigm by supporting the
reception of real-time redo blocks from all the hundreds to thousands of databases across the
enterprise, dramatically reducing data loss exposure from hours/days to sub-seconds.

Data Guard-Like Protection for All Databases
• Traditional Backup Techniques: Generic Bit-Copy Backup Appliance

– Backup taken once a day
– Data loss exposure: all transactions since last backup

• Recovery Appliance Backup Technique
– Uses Data Guard continuous real-time redo transport from
in-memory buffers to immediately protect ongoing transactions
Traditional
a ble
fe r
Full Backups
an s
+
n - t r
Protected
Incrementals
a no
Databases h a s
m ) ideฺ
ฺ c o Gu Recovery
n k ng+ ent Appliance
n b S t ud
aIncrementals
Redo
u nio this
l l o @ use
be e to
a k
l o (a cens
el backupliapproaches, if the online redo log is lost, then media recovery loses all
In traditional
b
m after the most recent available archived redo log file or incremental backup. An RPO
kee
changes
a of a day or more that might result from a traditional approach may be unacceptable.
The Recovery Appliance solves the RPO problem through a continuous transfer of redo
changes to the appliance from a protected database. This operation is known as real-time
redo transport.

Data Loss Protection from Site Disasters
• Replication to Remote Appliance protects data from disasters or site failures

• Automated restore from Local Appliance or directly from Remote Appliance
One Way Bi-Directional
a ble
fe r
Hub and
an s
Spoke
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu Tape Library
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
To protect l l o li or site outage, one Recovery Appliance can replicate backups to a
e against server
b
em Recovery Appliance.
edifferent
ak When the upstream Recovery Appliance sends the incremental backup to the downstream
Recovery Appliance, it creates a virtual full backup as normal. The downstream Recovery
Appliance creates backup records in its recovery catalog. When the upstream Recovery
Appliance requests the records, the downstream Recovery Appliance propagates the records
back.
If the local Recovery Appliance cannot satisfy virtual full backup requests, then it
automatically forwards them to the downstream Recovery Appliance, which sends virtual full
backups to the protected database. DBAs use RMAN as they usually do, without needing to
understand where or how the backup sets are stored.

Data Loss Protection from Users or Errors
• Optional low-cost Tape Archival provides unalterable Recovery

Appliance
protection from:
– Intentional attacks by hackers, employees, and

viruses
– Accidental data deletion due to using tape as
long-term archive
– Malfunction of primary data protection system
• Autonomous Tape Archival is built-in; just connect tape 16Gb FC
drives
a b le
– Offloads tape backup activity from production fe r
servers n s
n - t ra
– No expensive media manager backup agents on
production systems a no
h
– Tape drives run all day without slowing production a s
m ) ideฺ Supports Most
systems
g ฺ co t Gu Major Tape
Libraries
– Integrated Media Manager included,
k n or use
e n your
own tape software ban tud
on is S
n i
u e th
@
llo to us
b e Copyright © 2016, Oracle and/or its affiliates. All rights reserved.
( a ak nse
A robuste l lo strategy
backup l i ce protects data against intentional attacks, unintentional user errors
m b
kee
(such as file deletions), and software or hardware malfunctions. Tape libraries provide
a effective protection against these possibilities.
The fundamental difference between the two approaches is that the Recovery Appliance
backs up to tape, and not to protected databases. The Recovery Appliance comes with
preinstalled Media Manager software and supports optional Fibre Channel cards. Thus,
installation of a Media Manager is not necessary on the protected database hosts.
When the Recovery Appliance executes a copy-to-tape job for a virtual full backup, it
constructs the physical backup sets, and copies them to tape, and then writes the metadata to
the recovery catalog. If desired, the Recovery Appliance can also copy successive
incremental backups and archived redo log file backups to tape. Though the backup on the
Recovery Appliance is virtual, the backup on tape is a non-virtual, full physical backup. The
Recovery Appliance automatically handles requests to restore backups from tape, with no
need for administrator intervention.

The advantages of the Recovery Appliance tape solution are as follows:
• The Recovery Appliance performs all tape copy operations automatically, with no
performance load on the protected database host.
• Tape backups are optimized. The Recovery Appliance intelligently gathers the necessary
blocks to create a non-virtual, full backup for tape.
• The Media Manager software is preinstalled, eliminating the need for costly third-party
media managers.
• Tape drives and tape libraries function more efficiently because the Recovery Appliance is
a single large centralized system with complete control over them. In other tape solutions,
hundreds or thousands of databases can contend for tape resources in an uncoordinated
manner.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Data Loss Protection from Corruptions
• The Recovery Appliance understands and validates database

formats end to end.
Production Tape Archival

Appliance
• Data validated when
copied to and restored
from tape
a ble
fe r
an s
Remote Replica
n - t r
a no
h a s
• Data validated on
• Data validated on receive
m ) ireceive,
d e ฺ restore, and
• Data periodically revalidated
g ฺ co t Gu periodically
• Data validated on restore
a n kn den on-disk
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
A basicbprinciple li
of backup and recovery is to ensure that backups can be restored
m
kee
successfully. To ensure that there are no physical corruptions within the backed-up data
a blocks, backups require regular validation.
The Recovery Appliance provides end-to-end block validation, which occurs in the following
stages of the workflow:
• Recovery Appliance validation: The Recovery Appliance automatically validates the
backup stream during the backup ingest phase, before writing the backups to disk. No
manual RESTORE VALIDATE step is required. Also, a background task running on the
Recovery Appliance periodically validates the integrity of the virtual full backups in the
delta pools
• Oracle ASM: Oracle ASM stores the backup and redo data for the Recovery Appliance.
Oracle ASM mirrored copies provide redundancy.

• Oracle Exadata Storage HARD checks and disk scrubbing: Exadata Storage Server
software integrates comprehensive Oracle Hardware Assisted Resilient Data (HARD)
specifications, providing a unique level of validation for Oracle block data structures.
Exadata HARD checks support server parameter files, control files, online redo log files,
and datafiles. The checks work during Oracle ASM rebalancing or resynchronization.
Oracle Exadata Storage Server Software detects corruptions introduced into the I/O path
between database and storage. The software stops corrupted data from being written to
disk when a HARD check fails. Furthermore, Oracle Exadata Storage Server Software
automatically inspects and repairs hard disks periodically when hard disks are idle. If the
software detects bad sectors on a hard disk, the software automatically sends a request to
Oracle ASM to repair the bad sectors by reading the data from another mirror copy. By
default, the hard disk scrub runs every two weeks.
• Tape library: The Recovery Appliance validates blocks when it copies them to tape, and
also when it restores them from tape.
• Downstream Recovery Appliance in a replication configuration: If you configure a ble
fe r
s
replication, the downstream Recovery Appliance validates data during the backup ingest
an
and restore phases.
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

Recovery Appliance:
Minimal Impact
Eliminate Data Loss
Backups
Real-time redo
transport provides
backup and tape
a ble
fe r
Recoverability
an s
Easily protect all
n - t r
disjointed files
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Minimal l l o li
eImpact Backups: With the Recovery Appliance, customers simply run the RMAN in
b
k e em
an incremental forever strategy – that’s it, no more recurring full backups, no more tape
a backups, and no more RMAN backup deletion and validation commands on the production
server. All expensive backup workloads are offloaded to the appliance, where dedicated
hardware and storage handle these tasks in the most efficient manner.

Space-Efficient “Virtual” Full Backups
• After a one-time full backup, incrementals are used to create virtual full
database backups on a daily basis.
– Pointer-based representation of physical full backup as of incremental backup time

– Virtual backups typically 10x space efficient
– Enables long backup history to be kept with the smallest possible space
consumption
– “Time Machine” for database
Day N Virtual Full
Day 1 Virtual Full
a ble
fe r
an s
Protected
Day 0
Full
Day 1
Incr
Day N
Incr
n - t r
Databases n o Store
Delta
h a sa
m ) ideฺ
g ฺ co t Gu
a n kn den
No More Full Backups: b
o n Incremental
S tu Forever Architecture
u n i h i s
e t
a a kb se t
l o ( cen
l
Practically,
b li
e all backup-related processing is offloaded to the Recovery Appliance. This
k e em time-consuming compression, backup deletion, validation, and maintenance

includes
a operations. This frees production system resources, even outside the backup window, which
increases the performance of the production systems.
The virtual full backups contain individual blocks from multiple incremental backups. For
example, if you take a level 0 backup on Monday with SCN 10000, and if you take an
incremental level 1 backup on Tuesday with SCN 11000, then the Recovery Appliance
metadata database shows a virtual level 0 backup current to SCN 11000.
Essentially, virtual full backups are space-efficient, pointer-based representations of physical
full backups as of the point-in-time of an incremental backup. When a restore operation is
required, the delta store re-creates a physical full backup from the appropriate incremental
backup SCN.

Incremental Forever Backup Strategy
• This is the backup strategy strongly recommended to back up

protected databases.
– Level 0 incremental full backup (only one)

– Successive level 1 cumulative incremental
– Archived redo log backup
Note: No regular full backups are required.
• Advantages: ble
– Eliminates traditional backup windows and improves the s fe r a
protected database performance - t r an
– Eliminates the disadvantage of Block Change a non (bitmap
Tracking
a s so we are only
failure) because the Incremental is cumulative,
h
one bitmap away from the full backup m ) ideฺ
o uฺc t G
n g
k den
a n
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e main goals
One ofbthe li of the Recovery Appliance is to reduce backup-related processing on
em database systems to the absolute minimum by transmitting only the changed data.
eproduction
ak With unnecessary backup processing eliminated, production systems can now focus on their
primary goal of serving business critical workloads.
The Recovery Appliance implements an incremental forever backup architecture to minimize
impact on production systems.
This architecture is based on two innovative technologies: Delta Push and Delta Store.

Delta Push
Tape
Full/Incremental
Backup Sets
Remote File
Validate
Server Process
Redo Blocks
Archived Log
Backup Sets
Protected Redo Blocks Redo
Databases Staging
Area
Archived Log Backups
Delta Store
a ble
Validate
fe
Replica r
Backup Sets
Data Blocks
Index Blocks
(Create Virtual Full)
an s
Appliance
Backup Staging
Compress + Write to Disk
n - t r
RMAN
Incrementals via
Recovery
(FLASH)
a no
Appliance
Backup HTTP Servlet
h a s Incremental Backup Sets
Module
m ) ideฺ
g ฺ co t Gu Archived Log Backup Sets
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
With Deltal l o li
e Push, protected databases only send incremental backups containing unique
b
em to the Recovery Appliance. There is no need for recurring full backups. Delta Push is
echanges
ak also known as “incremental forever” because, after a one-time full backup, only incremental
backups are run on production systems. Effectively, Delta Push is a highly optimized form of
source-side deduplication. Changed blocks on production databases are very efficiently
identified by using RMAN block change tracking, which eliminates the need to read
unchanged data.
Special integration between protected databases and the Recovery Appliance eliminates
committed undo, unused, and dropped tablespace blocks from the backup stream,
significantly reducing overhead and space consumption.
Because Delta Push sends only changed data and not full backups, network traffic is greatly
reduced compared to other solutions. This enables low-cost 10 Gig Ethernet to be used for
backups. Expensive dedicated Fibre Channel or Fibre Channel over Ethernet backup
networks are not needed. Also, minimizing network traffic allows the Recovery Appliance to
be located further away from the protected databases and even, in some cases, across a
WAN in a remote data center.

When the protected database performs an online redo log switch, the Recovery Appliance
converts and assembles the redo changes into compressed archived redo log file backups.
The Recovery Appliance catalog automatically tracks these archived redo log backups in its
recovery catalog. The RMAN can restore and apply these archived redo log backups as
usual.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Real-Time Redo Transport
• Redo data is transferred from memory buffers on the database server; very
low impact.
• If the redo connection is lost, the appliance "buttons up" the received redo
data and creates a “partial archived log backup”.
– Preserves recovery until the last change received by the appliance
(sub-second RPO)
• When the connection is reinstated, a gap detection process on the
appliance automatically fetches missing archived logs from protected
databases.
Protected Databases Storage
a ble
Location
fe r
an s
- t r
X m
h a
Partial
a
) LogidBackup
co t GuArchived
e ฺ
no
sArchived
n
g ฺ
a n kn den Log Backup
nb s Stu
Fetch Missing
Archived Logs
n i o i
u e t h
a a kb se t
l o ( cen
Redo data
b l i
e contains lrecords of all changes made to a database and is therefore critical in
k e em
minimizing data loss if data failure occurs. By using the real-time redo transport feature of
a Recovery Appliance, you substantially reduce the window of potential data loss that exists
between successive archived redo log backups. A typical RPO is zero to sub-second when
you enable real-time redo transport.
With real-time redo transport enabled, a protected database generates redo changes in
memory, and then immediately transfers them to the Recovery Appliance, which validates
them and writes them to a staging area.
When the protected database performs an online redo log switch, the Recovery Appliance
converts and assembles the redo changes into compressed archived redo log file backups.
The Recovery Appliance catalog automatically tracks these archived redo log backups in its
recovery catalog. The RMAN can restore and apply these archived redo log backups as
usual.

The Recovery Appliance does not apply the redo that it receives to the backups sent by the
protected databases. Thus, to continue providing updated virtual level 0 backups, the Recovery
Appliance must incorporate new incremental backups into the delta store. The appliance
provides a virtual level 0 backup corresponding to each level 1 incremental backup sent by the
protected database. In a recovery scenario, you restore the appropriate level 0 backup, and
then use redo log files to roll it forward.
Redo log entries are transferred from buffer to the Remote File Server process in the Recovery
Appliance RAC database. Redo log files are assembled in the Recovery Appliance in the Redo
Staging Area (also known as the Landing Pad) located in the DELTA Store ASM Disk Group.
After that, the assembled redo log files per protected database are moved as Archived Redo
Log files into the DELTA Store ASM Disk Group too.
If a protected database loses connection to the Recovery Appliance, an INCOMPLETE_ARCH
task is performed to make the partial log file available as an archivelog backup. A gap detection
code runs to transfer missing logs from the protected databases to the Recovery Appliance.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Delta Store: Incremental Backups
• The backup strategy to the Recovery Appliance starts with an initial

full (incremental level 0) backup.
• After the level 0 backup, only incremental backups are needed,

consisting of just the changed database blocks relative to the
previous day’s incremental backup.
Day 0 A0 B0 C0 D0 E0
Full Backup
a ble
Day 1 C1 D1 fe r
Incremental Backup
an s
n - t r
Day 2 a no
D2 s
Incremental Backup
) h a ฺ
ฺ c om Guide
Day 3
n k nB3g eC3nt
Incremental Backup
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
The delta
b l is theli totality of all protected database backup data in the Recovery Appliance
estore
em location. All datafile and archived redo log backups reside in the delta store.
estorage
ak By default, the RMAN makes full backups. A full backup of a datafile includes every allocated
block in the file being backed up. A full backup of a datafile can be an image copy, in which
case every data block is backed up. It can also be stored in a backup set, in which case
datafile blocks not in use may be skipped.
An incremental level 0 backup is a full backup that happens to be the parent of incremental
backups whose level is greater than 0. Incremental backups at level 0 can be either backup
sets or image copies, but incremental backups at level 1 can only be backup sets.
The Recovery Appliance only supports incremental backups at level 0 and 1 to realize the
benefit of the optimization features provided by Virtual Full Backups. The Recovery Appliance
still supports full or partial backups (no incremental), but such backups are treated as simple
backup files as they appear to traditional file or tape storage systems and are not treated as
Virtual Full Backups.

Both incremental level 0 and 1 backups per protected database are stored in the BACKUP
STAGING AREA (we use flash drives from Storage Nodes) for validation. After that data blocks
are indexed, compressed (advanced compression option or ACO, if enabled), and transferred to
the DELTA STORE ASM Disk Group.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Delta Store: Virtual Full Backup Creation
• A virtual full backup is created in the Recovery Appliance catalog for each
received RMAN incremental backup.
• It operates at a datafile level and appears as a normal level 0 backup in

RMAN LIST/REPORT and catalog queries.
Full Backup
Day 1 Virtual Full Backup

Day 1 C1 D1
a ble
Incremental Backup
fe r
an s
Day 2 Virtual Full Backup
n - t r
Day 2
a
D2 no
Incremental Backup
h a s
m ) ideฺ
Day 3 Virtual
g ฺ coFull Backup
t G u
Day 3
a n kB3n C3den
Incremental Backup
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li the incoming changed data blocks, and then compresses, indexes,
e Store validates
The Delta
b
emstores them. These changed blocks are the foundation of virtual full database backups,
eand
ak which are space-efficient pointer-based representations of physical full backups as of the
point-in-time of an incremental backup.
Oracle Databases must be running with the block change tracking option that causes Oracle
Database to track datafile blocks affected by each database update. The tracking information
is stored in a block change tracking file. When block change tracking is enabled, the RMAN
uses the record of changed blocks from the change tracking file to improve incremental
backup performance by only reading blocks known to have changed, instead of reading whole
datafiles.

Delta Store: Efficient Indexing and Storage
Management
• The index task permits the Recovery Appliance to know the physical
location of every data block sent by the protected databases. Goal:
Recovery
Window =
Day 0 A0 B0 C0 D0 E0 3 Days
Full Backup
Day 1 - Virtual Full Backup
Day 1 C1 D1 A0 B0 C1 D1 E0
Incremental Backup
a b le
Day 2 A2 D2 fe r
ans
Incremental Backup A2 B0 C1 D2 E0
n - tr
no
Day 3 B3 C3
h a s a B3 C3 D2 E0
Incremental Backup
m ) ideฺA2
g ฺ co t Gu Day 4 - Virtual Full Backup

Day 4 aD4n
kn den
nb s Stu
A4 C4 C4
Incremental Backup A4 B3 D4 E0
n i o i
u e t h
a a kb se t
l o ( cen
Backup b e is theliautomated stage in which a Recovery Appliance scans a backup that
l
ingest
k e emsent by a protected database. The Recovery Appliance decomposes the backup into
was
a smaller sets of blocks, writes the blocks into the appropriate storage location, and indexes the
backups. Indexing includes inserting rows into the Recovery Appliance metadata database to
describe the physical location of every block.
This effective indexing stage helps to the Recovery Appliance to optimize the storage and
improve the recovery operations.
Space is released for blocks not needed for the recovery window goal. In the previous
example, on Day 4, assuming a three-day recovery window, Day 1 Virtual Full Backup is no
more needed and could be purged upon space pressure.

Delta Store: Virtual Full Backup Optimization
• The optimize task is a background process that runs on the most recent
virtual full backup to “optimize” its read I/O access for future restore
operations.
B0 E0 C1 A2 D2
Full Backup
B3 C3 A4 C4 D4
Day 1 C1 D1
Incremental Backup free free free free free
a b le
Day 2 fe r
Incremental Backup
A2 D2
an s
Optimization
n - t r
Day 3 A4
aC4no D4 E0
B3
Incremental Backup
B3 C3
h a s
m ) C3ideD2
A2 ฺ B0 C1
Day 4
Incremental Backup
A4 C4 D4
g ฺ co t Gu
a n kn den free free free free free
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Virtualb
full l l o
e blocks areli reordered into contiguous sets for fewer and larger read I/O requests,
k e em is more efficient than many smaller read I/O requests. This stage is for periodic
which
a optimization for future restore operations requests.
As old blocks are deleted and new incremental backups arrive for updated datafiles, the
blocks in a backup can become less contiguous. This state can degrade the performance of
restore operations. The Recovery Appliance automatically reorganizes the blocks to maintain
contiguity during ordinary maintenance and validation.
Optimization includes updating rows into the Recovery Appliance metadata database to
describe the new optimized physical location of every block.

Delta Store: Virtual Full Backup Purging
• Purging is a mechanism for reclaiming space.

Day 0 A0 B0 C0 D0 E0 A0 D0
B0 C0 E0
Full Backup
C1 D1 A2 D2 B3
Day 1 C1 D1
Incremental Backup
C3 A4 C4 D4 free
Day 2 A2 D2
Incremental Backup
Purging
a b le
fe r
Day 3
B0 E0 C1 A2
a n s D2
Incremental Backup
B3 C3
-t r
n onC4 D4
B3 C3 A4
Day 4 A4 C4 D4 h a sa
Incremental Backup
m ) ideฺ free free free free free
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Appliance
The Recovery li prefers to have space already available for new allocations. To do
b
em
eso:
ak • The Recovery Appliance executes as normal PURGE tasks in preexisting schedulers.
• This task uses histogram-based purging that defines a freespace_goal, based
upon recent usage patterns.
• This task cannot be used to sacrifice a recovery window goal for any database.
The Recovery Appliance uses a PURGE_IMMEDIATE path when no space is left for
allocations. To do so:
• This task executes as a special purpose scheduler.
• This task will sacrifice recovery window goals if needed.
• Requesters have to wait for purge to find space.

When the purging task is executed, databases are ordered based on the following criteria:
• TIME: If databases are above the recovery window goal, they are purged in the order of
the percentage their recovery window exceeds their recovery window goal.
• SPACE: If all databases are below their recovery window goal, they are purged in the
order of the percentage that their current space exceeds their reserved space.
When a database is targeted for purge, an SCN is chosen to target the purge in this way:
• Backups older than the SCN are deleted

• PURGE_DF tasks are created to compact the affected Delta Pools
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Delta Store: How It Works
• Delta Store is configured as ASM normal Redundancy, with

each Storage Cell being a Failure Group.
– If you lose a Storage Cell, you still have a copy of the data in
the Recovery Appliance.
• Delta Store hosts database blocks in Container files (2TB in
Length).
• Deleting a Database does not release space in the ASM le
group; instead, blocks are released in the Container Files. erab
n s f
-t r a
n on
h a sa
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
The base l l o
econfigurationli includes three storage servers that are internally connected using
b
em
ehigh-speed InfiniBand. It provides a highly available configuration with 94 TB of usable
ak capacity for incoming backups. The base rack can be upgraded incrementally by adding
additional storage servers into the rack, up to a maximum of 18 storage servers in a full rack.
Each storage server adds 32TB of usable capacity. The total usable capacity of a full rack is
580TB, with an effective capacity of up to 5.8 Petabytes of Virtual Full Backups.
The Recovery Appliance can protect databases whose total size is approximately the same
as the available capacity of the appliance for a typical recovery window of 10 days. For
example, a single full-rack configuration with 18 storage servers that has a 580TB usable
capacity can protect approximately 580TB of source database for a 10-day recovery window,
storing 10 580TB virtual full backups plus all the redo data generated for that 10-day period.
Accurate sizing of the Recovery Appliance depends on several factors related to protected
databases, including the initial database size and growth rate, storage consumed by temp and
undo, free space, database change rate, redo generation rate, desired recovery window, and
compressibility of the database.

If additional capacity is required, a second base rack can be connected to the first rack via high-
speed InfiniBand. The second rack includes its own pair of compute servers, which add
connectivity and processing power to the configuration. As with the first rack, storage capacity
can be easily expanded by incrementally adding storage servers. Up to 18 fully configured
racks can be connected together into a single appliance, providing more than 10 Petabytes of
usable capacity; this means more than 100 Petabytes of Virtual Full Backups.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Delta Store: Receiving Backup Pieces
Data Flow
128MB 128MB HTTP Servlet
Protected
Database
128MB
available in No Purge
storage
location?
immediate
a ble
fe r
Yes
an
Perform PURGEs
n - t r
operation
Flash (4MB AU) a no

h a s
Yes m ฺ Store
) ideDelta
No Is all of
backup piece
g ฺ co t Gu
INDEX
_BACKUP
received ?
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e
Data Flow l l o li
b
k e em Databases send backup pieces in chunks of 128MB in size to the Recovery
Protected
a Appliance HTTP Servlet (XMLDB). If the Recovery Appliance has enough free space to
allocate them, it continues receiving chunks until it receives the total number of chunks in the
backup piece. If there isn’t enough free space left in the Delta Store, the Recovery Appliance
immediately executes a purge operation to release and free up space to make room for more
chunks. After all the backup piece chunks are received, the Recovery Appliance executes the
INDEX BACKUP task that indexes a backup piece into one or more Delta Pools (separate
Delta Pool for each datafile).

Delta Store: Container Files and Delta Pools
• Container File: The storage within a Delta Store is implemented as a

collection of files in the DELTA ASM disk group.
– Each container file is at most 2TB in length.

– Each container file is broken up into 4MB allocation units.
• Delta Pool: A set of datafile blocks from which a virtual full backup is
constructed.
– One Delta Pool is created for each datafile (across all incarnations).
– Blocks in the Delta Pool are compressed individually.
– Blocks are added to the Delta Pool from the Container Files by the
a ble
INDEX_BACKUP task.
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l o
e Files li
Containerl
b
k e emRecovery Appliance uses Flash Cache (WriteBack mode) to store backups that it
The
a receives inside Container files. It improves the write-intensive operations because writing to
Flash Cache is much faster than writing to hard disks. After that, Storage Cell Software writes
them back onto the hard disk. In addition to improving write operations, Flash Cache improves
read operations to enable the Recovery Appliance to create Delta Pools.
Allocation units in Container files are used for the following three purposes:
• Collections of blocks that will be stored in a Delta Pool (chunks)
• Segments of backup pieces put in a storage location (may also include partial logs)
• Metadata that link segments of a backup piece together

Delta Pools
The delta store is the collection of delta pools. As the Recovery Appliance receives backups
from protected databases and stores them inside Container files, it indexes them and stores
them in delta pools. A delta pool is the set of datafile blocks from which the Recovery Appliance
constructs virtual full backups. The Recovery Appliance automatically manages delta pools, so
that it can create a virtual full backup that corresponds to any incremental backup ever
received.
Other Delta Pool features are given below:

• Each separate datafile whose backups are sent to the Recovery Appliance has its own
separate delta pool.
• Database tables are used to index the blocks according to their block number, SCN, and
the backup that generated them. There is one row in table per version of block stored
results in a very large table.
• ble
Each virtual backup that is generated is indicated by internal keys called vb_keys (virtual
a
backup keys), which are created for each datafile in each backupset. fe r
a n s
r
-t the Delta
• Deleting a virtual backup piece will NOT result in blocks being deleted from
n
Pool. They are only orphaned by the operation. n o
• Orphaned blocks are removed from the Delta Pool by the a s a task.
) h PURGE ฺ
• Blocks may be moved around in the Delta Pool by
ฺ c o m u ideDEDUPLICATION, and
the PURGE,
OPTIMIZATION tasks.
k n g nt G
• Blocks are read from the Delta Pool by
b anRESTORE,
t u deVALIDATE, BACKUP_SBT and PURGE,
n S
nio thistasks.
DEDUPLICATION, and OPTIMIZATION
u
l l o @ use
a k be e to
l o (a cens
b el li
m
a kee

Delta Store: Delta Pool Plans
• Whenever a set of blocks needs to be read from a Delta Pool,

a Plan is created that identifies the blocks that are needed and
the order in which they should be read.

• Plans for the five most recent virtual level 0 backups of each
datafile are maintained.
• Plans are constructed by the PLAN task in the background by
many tasks such as INDEXATION, OPTIMIZATION, PURGE
and so on. Also, RMAN operations such as RESTORE, r a ble
n s fe
BACKUP_SBT, and COPY_SBT expect plans to be pre-
n - tra
created, but create them on demand, if needed. no
h a sa
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li describing what data needs to be read from disk and in what order it
A Planbisea list of pointers
mbe read. Many Recovery Appliance internal tasks create plans to help complete their
kee
must
a work.

Internal Tasks
• Oracle Recovery Appliance schedulers

– Backup and restore tasks
– Validate tasks
– Purge tasks
– Miscellaneous tasks
• Priority: Try to run tasks with the lowest number (highest priority)
first.
• States
a ble
– EXECUTABLE fe r
a n s
– RUNNING
n -t r
o
– COMPLETED
s an
– RETRYING
) ha eฺ
– FAILED
ฺ c om Guid
– CANCEL | CANCELLING
n k ng ent
– CANCELLED nba tud
u nio this S
l l o @ use
be e to
a k
l o (a cens
Internal
b tasks li
el are executed as jobs in the Recovery Appliance database. The Recovery
k e em utilizes the Oracle Scheduler framework.

Appliance
a Some internal tasks are divided into categories, as given below:
• Backup and Recovery: Tasks such as INDEX_BACKUP and RESTORE are related to
internal ops working with Delta Pools, Archived Logs, Polling directories, tape, or
downstream Replica Recovery Appliances, in addition to working with Plans and some
internal verifications.
• Validate: Tasks such as VALIDATE are scheduled to validate backups stored for
databases.
• Purge: Tasks related to purging, freeing space, deduplication, and optimization such as
PURGE, PURGE_DUP, and OPTIMIZE.
• Miscellaneous: These tasks are outside the other well-defined categories. Some
examples are DB_STATS_REFRESH, CHECK_FILES, and REBUILD_INDEX. These
tasks are for internal maintenance operations.

Schedulers try to run tasks with the lowest number (highest priority) first. Tasks with highest
numbers are “busywork” tasks.
Using the task states you should be able to monitor the execution of these different tasks and
move on to the next steps.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Recovery Appliance Metadata Database: CATALOG
• Manages the backup metadata of all protected databases registered with

the Recovery Appliance
• Contains the recovery appliance catalog

• Every protected database that stores backups on the Recovery Appliance
must use the Recovery Appliance catalog.
– This means that each database must have its own unique DBID.
– Enterprise Manager will want a unique name as well.
• Protected databases can use the Recovery Appliance catalog without using
the Recovery Appliance as a backup repository.
r a ble
• Contains multiple Recovery Appliance user accounts n s fe
• The schema on the Recovery Appliance metadata database is n - tra by the
owned
RASYS user. a no
• Resides in its own ASM Disk Group called CATALOGhas(High Redundancy)
) ideฺ
m
co t Gu
g ฺ
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l o
e Applianceli Metadata Database
Recoveryl
b
k e emkey component of the Recovery Appliance is the Recovery Appliance metadata
The
a database. This database manages metadata for all backups, and also contains the RMAN
recovery catalog. The Recovery Appliance metadata database is preconfigured, pre-tuned,
and managed by the Recovery Appliance.
Recovery Appliance Schema
The Recovery Appliance schema contains metadata used internally by the Recovery
Appliance to manage backups on behalf of its protected databases. RASYS is the Recovery
Appliance administrative user who owns the Recovery Appliance schema. The Recovery
Appliance schema contains the Recovery Appliance catalog and the Delta Store Metadata.

Recovery Appliance Catalog
This is the RMAN Recovery Catalog. Updates to the recovery catalog reflect the results of
Recovery Appliance indexing and space management collection. These updates do not occur in
the control files of the protected databases. For this reason, protected databases that store
backups in the Recovery Appliance must use the Recovery Appliance catalog.
The RMAN connects to the Recovery Appliance catalog by using the same Recovery Appliance
account that is employed for backup and recovery operations. Each Recovery Appliance user
account is also a virtual private catalog account.
Recovery Appliance Delta Store Metadata
This metadata stores configuration data such as user definitions, protection policy definitions,
and client database definitions. The metadata database also stores the contents of the Delta
Store.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

RMAN Recovery Catalog and Virtual Private Catalogs
• RMAN Recovery Catalog

– Only the RMAN Recovery Catalog in the Recovery Appliance must be used.
– Existing Metadata in other RMAN Recovery Catalogs outside the Recovery

Appliance can be imported into the Recovery Appliance (migration).
• RMAN utility must connect to the Protected Database and to the Recovery
Appliance.
– The connection to a Protected Database is typically as the SYS user.
– The connection to the Recovery Appliance is as a virtual private catalog user.
• The Recovery Appliance implements Virtual Private Catalogs (VPC) to
r a ble
support multitenancy.
n s fe
• The process of assigning a VPC account to a Protected Database
n - trisa
o
performed at the time Protected Database is enrolled in thenRecovery
Appliance. s a
h a
) the same ฺ VPC user
• Multiple protected databases can be enrolled with
m
o Gu i d e
supporting multitenancy in this way. gฺc
k n ent
b a n ud
n S t
u nio this
l l o @ use
be e to
a k
l o (a cens
el RecoveryliAppliance for data protection of your protected databases, metadata for
To usebthe
em protected databases must be stored in the Recovery Appliance catalog and not in the
ethese
ak RMAN recovery catalog. Existing backup metadata that is currently stored in the RMAN
recovery catalog can be imported into the Recovery Appliance catalog.
To perform protected database backup or recovery operations by using a Recovery
Appliance, you must connect to the protected database and to the Recovery Appliance
catalog.
The RMAN Recovery Catalog includes functionality to create virtual private RMAN catalogs
(VPC) for groups of databases and users. In Oracle Database 12c Release 1 (12.1.0.2), the
RMAN recovery catalog is created and managed by using Oracle Virtual Private Database
(VPD), providing better performance and scalability when a large number of virtual private
catalogs are created.

Virtual Private Catalog users are Recovery Appliance users in the Recovery Appliance and they
are authorized to connect to, and request services from, the Recovery Appliance. Every
Recovery Appliance user account is an Oracle Database user account on the Recovery
Appliance metadata database, and the owner of a virtual private catalog. When RMAN backs
up a protected database, it connects to the recovery catalog with the Recovery Appliance user
account credentials.
Protected databases that authenticate with a VPC account can perform backup and restore
operations in Recovery Appliance.

The default storage location (DELTA ASM disk group) in the Recovery Appliance also contains
a fast recovery area for catalog backups.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Data Encryption
• Recovery Appliance Encryption

– The Recovery Appliance does not itself encrypt and decrypt
data.
– Options (implemented outside the Recovery Appliance)
— Transparent Data Encryption (TDE, protected database
tablespaces)
— Oracle Net Security (network transfer)
— Redo Encryption (real-time redo transport)
a ble
Tape Encryption (drive-based hardware) fe r
—
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Appliance
The Recovery li does not provide server-side encryption, which means that the
b
em does not itself encrypt and decrypt data.
eappliance
ak Options:
• TDE: Oracle recommends that you enable TDE on tablespaces in the database, and
then take incremental backups as usual. TDE requires the Advanced Security Option
(ASO).
• Oracle Net Security: It encrypts the transfer of backups and redo to the Recovery
Appliance.
• Redo Encryption: It encrypts redo both at rest on the Recovery Appliance and during
the network transfer to the appliance.
• Tape Encryption: The Recovery Appliance supports tape drive-based hardware
encryption. In this case, the tape drive encrypts the data, and the software.

Protected Databases
• A client database whose backups are managed by a Recovery

Appliance
• License: Standard or Enterprise Edition

• Storage or volume manager: filesystem, ASM, ACFS, ZFS, and so
on
• Database Options: Oracle Real Application Clusters, Oracle Data
Guard, Oracle Streams, and/or Oracle GoldenGate
• Module: The Recovery Appliance Backup Module to be installed
r a ble
• Benefits of using the Recovery Appliance:
n s fe
– Minimizes the impact of backups on
n - tra
production servers
a no
– Eliminates data loss h a s
m ) ideฺ
– Integrates with other HA technologies
g ฺ co t Gu
– Reduces restore and recovery
a n kn timed e n
ion
b
– Optimizes storage requirements
n Stu
is
u
@ use t h
l l o
be e to
a k
l o (a cens
el
A clientbdatabase li backups are managed by a Recovery Appliance is called a protected
whose
m
kee
database.
a A protected database can be a Standard or Enterprise Edition database on any storage or
volume manager (such as ASM, filesystem, ACFS, ZFS, and so on) and can use any of the
following database options: Oracle Real Application Clusters, Oracle Data Guard, Oracle
Streams or/and Oracle GoldenGate.
Recovery Appliance Backup Module: This backup module is the shared library that
transfers backup data to and from the Oracle Recovery Appliance. It must be installed in the
host of the protected database.

Recovery Appliance Storage
The Recovery Appliance uses the following storage:

• Storage location = Delta Store
– DELTA ASM Disk Group (Normal Redundancy)

– Can be shared among multiple protected databases
– Benefits:
— More efficient disk usage
— Backup deduplication and compression
— Shared disk backup pool distributed
• Backup polling location a ble
fe r
– Optional shared storage (NFS using 10gb Eth Network)
an s
– Outside the Recovery Appliance (upload backups without ZDLRA interaction)
n - t r
– no
Backup polling policy: file system path + search frequency
a
• Redo staging area h a s
– Real-time redo transport destination m ) ideฺ
g
– DELTA ASM disk group (normal redundancy)ฺ co t Gu
– Compressed archived logs a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Storage
l o
eLocation li
l
b
m ASM Disk Group serves as the destination for protected database backups. A
eAeDELTA
a k Recovery Appliance storage location can be shared among multiple protected databases.
The Recovery Appliance administrator decides which clients will use the storage location.
Benefits:
• Efficient disk usage: The Recovery Appliance uses common storage to absorb spikes
from all protected databases, thereby reducing the total amount of over-allocated
storage. Oracle recommends that protected databases continue to maintain fast
recovery areas for storage of local online and archived redo log files, control file auto-
backups, and flashback logs.
• Database-optimized backup deduplication and compression
• The shared disk backup pool is distributed based on a database protection policy, which
defines the disk recovery window goal for each database protected by the policy.

• By default, the delta pool is stored in normal redundancy Oracle ASM disk groups, which
means that the Recovery Appliance maintains two copies of all on-disk backups.
Database backups can survive the loss of any one disk or storage server. The Recovery
Appliance metadata database, which tracks the files and blocks, is stored in a high
redundancy Oracle ASM disk group.
A more complex implementation may require multiple storage locations, for reasons such as
expansion and space allotment.
Backup Polling Location

This is an optional file system directory on shared storage, outside the Recovery Appliance, that
is a destination for backup pieces and archived redo log files from a protected database. The
Recovery Appliance polls the directory at specified intervals, retrieves any found backups, and
then processes and stores them.
A backup polling policy defines a file system directory where a protected database places
backups without interacting directly with the Recovery Appliance. The backup polling directory
a ble
is an NFS mount point, and is not in a Recovery Appliance storage server. fe r
ans
n - t r
The polling policy defines the file system path to the storage and how often it will be searched
sent to the Recovery Appliance using the polling method. a no

for new backups. Polling policies are optional and do not need to be created if backups are not
h a s
The Recovery Appliance checks the polling directory for newly
m ) ideฺbackups. When backups
created
exist, the Recovery Appliance copies the backups from
g ฺ cothe tpolling
G u directory to its internal
n n en
Recovery Appliance storage location, and thenkprocesses them. After enough time has passed
a the
for the Recovery Appliance to copy the backups,
b t d
uprotected database deletes the backups
n S
from the polling directory.
u nio this
Redo Staging Area
l l o @ use
For Recovery Appliance
a k beinstallations
e to that enable real-time redo transport, this is the destination
for redo streams
l o e ns databases transmit to the Recovery Appliance. The staging area
(athat cprotected
resides on
b el RecoveryliAppliance disks (the DELTA ASM disk group). The Recovery Appliance
collects
m data into archived redo log files, which it then converts to compressed archived redo
e e
ak backups that it writes to a Recovery Appliance storage location.
log

Recovery Appliance Storage Space Management
Recovery Appliance settings that affect storage space management include:

• Recovery window goal (days)
– The recovery window goal specifies the days, counting backward from the current time,
within which a point-in-time recovery is possible.
– It effectively extends the number of days, if free space is available.
– Sufficient Space: Backups created before a recovery window goal may be available (but
not guaranteed by default).
– Insufficient Space: The Recovery Appliance may purge backups attempting to respect the
recovery window requirement first.
• Reserved space (GB)
– Disk space (in GB) is guaranteed to each protected database to meet its recovery window
a ble
goal.
fe r
– Sufficient Space: Reserved space is ignored and the recovery window goal settings apply.
an s
– Insufficient Space: The recovery window goal is ignored and reserved space settings
apply. The Recovery Appliance purges backups for the database whose backups exceed n - t r
the Reserved Space by the highest percentage.
a no
• Guaranteed copy (NO/YES) h a s
– Older backups need to be copied to tape or a replicam ) iAppliance.
Recovery
d e ฺ
– NO (Default): Old backups are purged when more
g ฺ cospacet isGneeded
u even if the backups are
not copied or replicated.
a n kn old dbackup
e n before it has been copied or
nb s Stu
– YES: The Recovery Appliance never purges
replicated.
n i o i
u e t h
a a kb se t
l o ( cen
e duty oflithe Recovery Appliance administrator is to plan for the appropriate
l
An important
b
em of disk space for a specified retention window and database size. The Recovery
eamount
ak Appliance avoids storage islands and over-allocation typical to host or LUN-oriented
provisioning. By using the Recovery Appliance, space is dynamically reallocated between
databases to meet recovery window goals, such as recovering to any time in the last 35 days.
Recovery Window Goal
The recovery window goal specifies the interval (typically specified in days), counting
backward from the current time, within which point-in-time recovery must be possible. The
Recovery Appliance attempts to retain sufficient backups to meet the recovery window goal
defined for each database.
Reserved Space
The reserved space defines the amount of disk space guaranteed to each protected database
to meet its recovery window goal.

Guaranteed Copy
A key question in storage management is whether it is more important to ensure that older
backups are copied or replicated than it is to accept new backups or redo.
• NO (default): The Recovery Appliance can purge backups before they have been copied
to tape or replicated when it is necessary to make space for newer backups. In this case,
the protected database may have more or less than the reserved space.
• YES: When a guaranteed copy is enabled, the Recovery Appliance never purges a
backup before it has been copied to tape or replicated. After the Recovery Appliance has
consumed the reserved space with backups that have not been copied to tape or
replicated, it does not accept new backups or redo. This setting only changes the
behavior of storage management when the tape system or the replicated Recovery
Appliance is unavailable for an extended period.
The Recovery Appliance uses recovery window goals and reserved space settings to allocate
storage dynamically to meet business requirements. If the Recovery Appliance has purged as
a ble
e r
much backup data as possible while still meeting the recovery window goal for each database,
f
ans
and if more space is needed, then the Recovery Appliance evaluates the reserved space
n - t r
setting of each database. Recovery Appliance purges backups for the database whose backups
o
exceed the reserved space by the highest percentage.
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Recovery Appliance:
Minimal Impact
Eliminate Data Loss
Backups
Real-time redo
transport provides
backup and tape
a ble
fe r
Recoverability
an s
Easily protect all
n - t r
disjointed files
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
Database-Level li
Recoverability: Each incremental block received is validated, compressed,
b
k e emthen indexed as a part of a new virtual full backup, which appears as a normal full backup
and
a the RMAN catalog. Essentially, this means validated full backups are now available at any
in
point in time, without ever incurring the cost of running full backup operations.

End-to-End Data Protection Visibility
2
1
Well-Defined RMAN Recovery
Oracle Blocks Backup Set Appliance
End-to-end Oracle
Block Validation
DW BI CRM ERP
Application-oriented
Protection Policy
a ble
3 fe r
an s
n - t r
a no
h a s
m
Integrated ) ideฺ
EM Cloud Control:
ฺ co t Gu
Management
g
a n kn den
Policy-Based Management: Application-Oriented
n b S t uData Protection
u nio this
l l o @ use
be e to
a k
l o (a cens
The Recovery
b li understands internal Oracle database block formats, which enables
el Appliance
m levels of data validation. All backup data and redo blocks are automatically validated as
kee
deep
a they are received by the Recovery Appliance, as they are copied to tape, and as they are
replicated. In addition, backup blocks are also periodically validated on disk. This ensures that
recovery operations will always restore valid data – another unique differentiator that is only
possible because of the Recovery Appliance’s deep database integration. If a corruption is
discovered during validation, the Recovery Appliance’s underlying storage software
automatically reads the good block from a mirrored copy and immediately repairs the
corrupted block.

Fast Recovery with Oracle Recovery Appliance
Traditional
• Minimum amount of data transferred Restore on
- Only restore virtual full backup Friday
- No time required to transfer incrementals

Apply
• No Oracle Database merge during Redo Log
Faster Recovery
recovery Merge
- Virtual full backups are ready to use
- Only apply redo logs Restore
Thursday
• Quicker site-failure recovery Recovery Restore
Appliance Wednesday
- Direct restore from replicas or tape Restore
Restore
Tuesday
a ble
Apply Restore fe r
Redo Log
an
Monday s
n - t r
Restore
a no Restore
Virtual Full Full
a s
Backup
h eฺ Backup
)
ฺ c om Guid
Delta Push and Virtual Full Backupsa n kngRapid
Enable d e n t
Recovery to Any Point In Time
n b t u
u nio this S
l l o @ use
be e to
a k
l o (a cens
The Recovery
b li uses virtual full backups to provide rapid recovery to any point in
el Appliance
emregardless of the amount of data being recovered. The on-disk recovery strategy of
etime,
ak Recovery Appliance has the advantage that RMAN can recover virtual full backups to any
point in time without applying incremental backups.
When a database is protected by the Recovery Appliance, RMAN must only restore a single
level 0 backup for the day of the RPO, and then recover up to the last second by using redo
log files sent via the real-time redo transport feature. For example, if the recovery window is 7
days, and if the RPO is 5 days ago, then RMAN can restore a single virtual full (level 0)
backup that is current to 5 days ago, and then recover it by using redo, and not level 1
incremental backups.

Fast Recovery with Oracle Recovery Appliance
Tape
Workflow
Data Blocks
RAC Node-1 RAC Node-2
(Sessions & (Sessions &

Tasks) Tasks) Redo Blocks
Protected
Databases
“RESTORE DATABASE”
Full Backups
“RECOVER DATABASE” Delta Store
Archived Log Backups

Virtual Full Backups
Full Backups
a b le
fe r
HTTP Servlet
a n s
Re-assemble and
Validate Physical Full
r
n-t Replica
Archived Logs
a no Backup
ZDLRA
s
ha eฺ
)
Full Backups om id
Enterprise Manager
ฺ c G u
k ng ent
Prepared
Network
n
for
n b a
S t ud
Transmission
u nio this
l l o @ use
be e to
a k
l o (a cens
el Appliance
The Recovery
b li is able to reassemble virtual full backups located in the Delta Store
emphysical full backups and validate them. After that, these physical full backups are sent
einto
ak thought the network by using the HTTP Servlet. If no available virtual full backups are in, the
local Recovery Appliance automatically sends a request to a Recovery Appliance replica to
re-assemble and validate them into a physical full backup on behalf of it and transfer it back to
be sent directly through the network by using the HTTP Servlet. In addition, for archival
backups, it is able to automatically request the integrated Media Manager or third-party media
servers the physical full backups archived in the tape libraries to be sent directly through the
network by using the HTTP Servlet too.

Recovery Appliance:
Minimal Impact
Eliminate Data Loss
Backups
Real-time redo
transport provides
backup and tape
a ble
fe r
Recoverability
an s
Easily protect all
n - t r
disjointed files
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
Database-Level li
Recoverability: Each incremental block received is validated, compressed,
b
k e emthen indexed as a part of a new virtual full backup, which appears as a normal full backup
and
a the RMAN catalog. Essentially, this means validated full backups are now available at any
in
point in time, without ever incurring the cost of running full backup operations.

Policy-Based Cloud Scale Protection
• Recovery Appliance Protection Policies

– Standardized recovery window, tape retention, replication
policies
Gold Policy – Customer Critical

Disk: 35 days
Tape
Tape: 90 days
a ble
Silver Policy – Internal Critical
fe r
Disk: 10 days
an s
Tape: 45 days
Replica
n - t r
a no
h a s
Bronze Policy - Test/Dev
m ) ideฺ Replica Recovery
Disk: 3 days
g ฺ co t Gu Appliance also
Tape: 30 days
n k n en Policy-Based
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
The Recovery
b li introduces the concept of a protection policy, which defines recovery
el Appliance
em that are enforced on a per-database basis on the appliance and on tape. Using
egoals
ak protection policies, databases can be easily grouped by recovery service tier. The Recovery
Appliance includes predefined “Platinum”, “Gold”, “Silver”, and “Bronze” policies, which can
be customized to support various business service-level agreements. For example, database
backups under the Gold policy target a 35-day recovery window on a local Recovery
Appliance and 90 days on tape, while backups managed under the Silver policy target a 10-
day recovery window on a local Recovery Appliance and 30 days on tape. Tiered protection
policies are also independently applied on the remote replicated Recovery Appliance.
The Recovery Appliance uses protection policies to simplify the management of protected
database backups. A protection policy is a collection of attributes that defines recovery goals
and storage space requirements for one or more protected databases. A Recovery Appliance
contains multiple protection policies that define varied recovery goals. Each protected
database is assigned exactly one protection policy that determines how the Recovery
Appliance stores and maintains backup data for that protected database.

The default care Platinum, Silver, Gold, and Bronze, and each of them has different recovery
window goals. You can create your own protection policies to accommodate databases with
different backup and recovery requirements.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Dynamic Policy-Based Space Allocation
• Space dynamically reallocated between databases to meet

recovery window goals
– Example: Recover to any time in the last 35 days

• Avoids storage islands and over-allocation typical of host or
LUN-oriented provisioning
a ble
Recovery
fe r
Sales DB
Window Goals
an s
n - t r
a no
h a s
m ) ideฺ
Web DB
g ฺ co t Gu
a n kn den Sales Web
n b t u
u nio this S
l l o @ use
be e to
a k
l o (a cens
el
Using protection
b li as the basis, the Recovery Appliance fully manages all backup
policies
k e em space according to each database’s recovery window goals. For example, the
storage
a “Finance Database”, a member of the Gold policy, can be recovered within the past 35 days,
while the “Products Database”, a member of the Silver policy, can be recovered within the
past 10 days. If free space is available in the Delta Store, backups older than the recovery
window goal can be retained, effectively extending the recovery window. Upon space
pressure, the Recovery Appliance purges backups and automatically re-provisions space
between databases to meet the recovery window goals for every protected database. The
appliance may also purge backups proactively, in advance of any space pressure, based on
historical space usage – again, in order to meet recovery windows goals for all databases.
Space is purged in a database intelligent fashion with an understanding of the dependencies
between datafiles, redo logs, and control files.
This recovery window-oriented space management approach eliminates the need to manage
space at an opaque storage-volume level as is typical with generic backup appliances. With
this innovative approach, data protection is aligned with each application’s business criticality,
and manual rebalancing of space is eliminated.

Modern Cloud-Scale Database Protection
• Scale-out compute servers for data processing

• Scale-out storage servers for persistence and deduplication
• Scalable InfiniBand internal fabric

Protected
Databases
No Bottlenecks, No
Single Point of Failure Fibre Channel
Connectivity connectivity
scales with scales with
a ble
capacity capacity
fe r
an s
n - t r
no
10 GigE or
InfiniBand
s a
h a
) ideฺ
Oracle Database m
co t Gu
10.2, 11g, 12c
g ฺ
a n kn den
Single System Scalesnto
o b ProtectS tuan Entire Data Center
u n i h i s
e t
a a kb se t
l o ( cen
l
e Appliance
The Recovery
b li is based on the Oracle Exadata architecture and therefore inherits
k e em
its proven scalability, redundancy, and performance. As additional databases within the
a enterprise are protected by the Recovery Appliance, compute servers and storage servers
can be easily added to the appliance, providing a simple, no-downtime, scale-out data
protection cloud that seamlessly supports business growth.

Recovery Appliance Replication
• When a Protection Policy is added to the Replication Server on Upstream Appliance, all future
incremental and archived log backups for its protected databases are queued for replication.
• Level 0 backup is first replicated.

• Each level 1 incremental received by downstream generates a new virtual full backup.
• New virtual full records on downstream are periodically synchronized to upstream catalog.
• The number of concurrent replication streams can be adjusted, based on network
consumption/environment.
HTTP
a ble
fe r
Full (Level 0) Backup
an s
n - t r
no
Storage
Location
s a
Incremental and
Archived Log Backups h a
) ideฺ
Virtual Full
Backups
m
co t Gu
Upstream Downstream g
n en ฺ Archived Log
Backups
Appliance nkAppliance
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
In Recovery
b li replication, one Recovery Appliance (the upstream Recovery
el Appliance
m
kee
Appliance) forwards backups to another Recovery Appliance (the downstream Recovery
a Appliance). After initial configuration, replication is fully automatic. Each Recovery Appliance
in a replication topology manages its own protection and polling policies.
Because a downstream Recovery Appliance processes backups independently from the
upstream Recovery Appliance, a downstream Recovery Appliance can have completely
different policies for every database whose backups it is storing.

Topologies
• One Way: Backup data flows from local databases to an upstream Recovery Appliance,
which forwards it to a remote downstream Recovery Appliance.
• Bidirectional: Backups flow from local databases to a local Recovery Appliance, which
then forwards them to a remote Recovery Appliance. Conversely, backups flow from
remote databases to a remote Recovery Appliance, which then forwards them to the local
Recovery Appliance. In this case, each Recovery Appliance plays both the upstream and
downstream roles in the replication topology. Every Recovery Appliance serves as the
primary backup location for one set of protected databases, and the secondary backup
location for the other set. In this way, every Recovery Appliance is actively utilized while
also providing disaster recovery services for the other Recovery Appliance.
• Hub and Spoke: Backups flow from one set of databases to a local Recovery Appliance,
and from a different set of databases to a different local Recovery Appliance. The local
Recovery Appliances then forward these backups to a single remote Recovery Appliance,
a ble
which archives the backups to tape.
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Media Manager Integration
• Pre-bundled Media Manager is the default Recovery Appliance tape archival solution.
• Backups are stored on tape as complete backup sets, and not as virtual backups.
• Restore operations do not need interaction with the Recovery Appliance.

• Protection policies apply to tape backups.
• Integration with third-party media managers is possible.
CTDB Day 1 Full
a ble
fe r
Full Backup
an s
CTDB Day 1 Virtual Full
Day 0 Day 1
n - t r
Full Incr
a no
h a s
Incremental &
Archived Log Backupsom
) ideฺ
g ฺ c t Gu
Recovery Appliance
a n kn den Tape Library
b t u
n i on is S
@ u e th
( a ak nse
e l
The Recovery
lo Appliance
l i ce includes a built-in media manager. The Recovery Appliance offloads
m b
ak ee
tape backup operations from protected databases to the Recovery Appliance. Thus, protected
database hosts do not need the RMAN-integrated media management software module.
Instead, a single copy of the pre-bundled media manager module is installed on the Recovery
Appliance. The Recovery Appliance automatically manages the copy of backups to tape for
all protected databases.
Protected databases send backups to the Recovery Appliance, which stores them on disk in
the specified storage locations. To reclaim disk space and to create transportable tape
backups, business requirements may necessitate archival to tape.
Backups are stored on tape as complete backup sets, and not as virtual backups, so tape
backups are usable by RMAN without mediation by the Recovery Appliance.

Enterprise Manager Cloud Control
• The Recovery Appliance administrator centrally monitors and manages all

database protection activity across all tiers.
• The database administrator monitors the protection status of the database

from disk, to tape, and to replica.
• Offloaded replicas and tape backups appear in the recovery catalog.
Recovery
Appliance
a ble
fe r
Tape
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
Protected
a n kn den Remote
Databases
i o nb s Stu
Enterprise Manager Appliance
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Cloud b l l o
e lists every
Control li Recovery Appliance in the enterprise. The Recovery Appliance
k e em page is a command center that centralizes management of the Recovery Appliance
home
a environment. From this page, you can manage Recovery Appliance storage and performance,
and view recent activity and issues that may need attention. The storage locations page
expands on the information provided in the storage locations section on the Recovery
Appliance Home page.
It is easy to provision protected databases for the Recovery Appliance.
• Recovery Appliance Administrator
- Chooses new database in the Enterprise Manager list
- Assigns protection policy
- Sets new database credentials
• Database Administrator
- Selects Recovery Appliance target in Enterprise Manager
- Enables real-time redo transport

• The primary monitoring tool for Recovery Appliance administrators is the Oracle
Enterprise Manager Cloud Control (Cloud Control) incident and event notification
framework. The primary interface is the Recovery Appliance home page, which
prominently displays warnings, alerts, and errors. The monitoring framework integrated
with Cloud Control is an effective way of managing issues and tracking them until
resolution. In addition to these monitoring capabilities for Recovery Appliance, Cloud
Control also includes powerful monitoring framework for protected databases.
• Recovery Appliance replicas and Media Manager software have Enterprise Manager
Cloud Control plugins to enable the management and monitoring of these components by
using the web or CLI interface, and utilities of Cloud Control.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

Reporting
• Powerful reporting framework through Oracle BI Publisher part of

Oracle Enterprise Manager Cloud Control
• Pre-created reports
– Capacity Planning summary and details
– Recovery Window analysis Tape
– Protected Databases details
– Top 10 ranks
• Manual and automatic scheduling
r a ble
• Custom reports for querying metadata catalog views for extended
n s fe
a
reporting n-tr
no
s a
h a
) ideฺ
m
co t Gu
g ฺ
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Through l l o li the Recovery Appliance provides pre-created BI Publisher reports
ethe BI Publisher,
b
k e emenable you to meet the following goals:
that
a • Ensure that the Recovery Appliance has sufficient storage space for its needs: By
using the capacity reports, you can plan for additional storage, reduce the number of
new protected databases added to the Recovery Appliance, or adjust protection policies
so that the aggregate recovery window space decreases.
• Ensure that the network is not overloaded: The capacity reports also indicate
whether the Recovery Appliance has maximized network capacity. In some cases, you
can reduce network load by redistributing network traffic more evenly throughout the
day. If network traffic is not distributed, and if network peaks are close to maximizing
network bandwidth, then you may need to adjust the backup window times of some
• Provide a good view of system performance and activity for service requests.
• Obtain a brief or highly detailed status report for any protected database, which can
sometimes be useful for troubleshooting databases that are not meeting recovery
window goals

Creating custom reports helps you to answer questions like this:
• What tasks have been waiting the longest?
select task_id, task_type, interrupt_count, systimestamp-creation_time
wait_time from ra_task where creation_time = (select min(creation_time) from
ra_task where archived=’N') and archived='N';
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Summary

• Describe data protection challenges in modern data centers
• List the different features of the Recovery Appliance

• Explain the Recovery Appliance functionality
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

4
Recovery Appliance Installation r a ble

s fe
- t r an
o n
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake
Objectives

• Describe the Oracle Recovery Appliance installation process
• Use Oracle Exadata Deployment Assistant (OEDA) to install

• Explain the different options to extend and expand the
Recovery Appliance
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Agenda
• Installation Process
• Extending and Expanding the Recovery Appliance
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Oracle Exadata Deployment Assistant (OEDA)
OEDA is used to configure the Oracle Recovery Appliance.

• Work with your network and database administrators to
evaluate the current network settings.

• OEDA:
– Guides you through a series of pages
– Generates a set of configuration files
– Installs the Recovery Appliance software
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li the Oracle Exadata Database Machine, the Recovery Appliance, or
e can configure
The OEDA
b
emOracle SuperCluster. The OEDA deployment tool is used to perform the initial Oracle
ethe
ak Recovery Appliance configuration. It loads network settings, creates user accounts, installs
Oracle Recovery Appliance software, and secures the system based on information in the
configuration files created by the OEDA configuration tool.
You can choose from multiple platforms to run the OEDA tool: Microsoft Windows, Linux, Mac
OS X, and Solaris (SPARC and x86-64).

The assistant generates a file to be used when setting up the system.
The host names, IP addresses, public addresses, SCAN addresses and virtual IP (VIP)
addresses must be registered in the DNS before the initial configuration and installation.
All addresses registered in the DNS must be configured for both forward resolution and
reverse resolution. Reverse resolution must be forward confirmed (forward-confirmed reverse
DNS) such that both the forward and reverse DNS entries match each other.
See the following support note for additional information: Zero Data Loss Recovery Appliance
Supported Versions (Doc ID 1927416.1): Patch Release History for the ZDLRA section
(OneCommand)
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

OEDA
OEDA configuration guidelines:

• Do not run the tool on Storage Servers.
• Choose a platform to run the OEDA tool for configuration.

– Microsoft Windows
– Linux
– Mac OS X
– Solaris (SPAC and x86-64)
• Configuration script = config.sh le
• Deployment script = install.sh r a b
s fe
- t r an
no n
s a
h a
) ideฺ
m
co t Gu
g ฺ
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Consider l l o li guidelines:
ethe following
b
k e e•m Do not run the OEDA deployment tool on Exadata Storage Servers.
a • When used for deployment (installation), the deployment assistant is run on any of the
following systems that can access the Exadata database servers:
- Oracle Exadata Database Machine database server
- Oracle SuperCluster database domain
- Oracle SuperCluster database zone
- Microsoft Windows machine
- Oracle Linux machine
- Oracle Solaris SPARC machine
- Apple OS X machine
• The configuration utility is the config.sh script. This tool creates some files and
directories that must not be edited or changed.

The deployment utility, install.sh, requires the configuration files created by the configuration
tool script, config.sh, and the compressed software files from the Oracle Technology Network
for the software release being deployed.
Note: Ensure that the version of the OEDA used to generate the XML file and to install the
ZDLRA is the same as the version listed.
a b le
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

OEDA Configuration: Welcome Screen
• You must select the Zero Data Loss Recovery Appliance option here.
• Do not import legacy configurations that are not ZDLRA
configurations.
• Do use import any time you have a new version of configurator and
an existing ZDLRA configuration.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
The OEDA l l o
e configures li the engineered system based on the configuration information entered
b
ak ebyem
the customer. The OEDA is provided as a compressed file. To use the OEDA, extract the
file on the client server. After the file has been extracted, change to the directory that has the
extracted folder, and run one of the following commands:
• On Linux, Apple, and UNIX: config.sh
• On Microsoft Windows: config.cmd

OEDA Configuration: Customer Details Screen
• The name prefix is used to generate a host name for all rack
components: compute nodes, storage nodes, switches, and
power distribution units (PDU).
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Customer l l o li
e Details information:
b
k e e•m Customer Name (Required): The customer name can contain alphanumeric characters
a and spaces
• Application: The application used on the Oracle Exadata Database Machine or another
protected Oracle Database system
• Network Domain Name (Required): The name of the domain in which Recovery
Appliance will operate
• Name Prefix (Required): The prefix used to generate host names for all rack
components. The prefix can contain up to 20 alphanumeric characters. White space is
invalid. For example, a value of ra01 results in a compute server name of ra01db01
and a storage server name of ra01cel01.
• Region: The geographic area where Recovery Appliance will be installed. Select a
region from the list.
• Time Zone: The time zone for the installation site. Select a time zone from the list. The
displayed time zones are within the selected region.
• DNS: Up to three IP addresses for the DNS servers
• NTP: Up to three IP addresses for the Network Time Protocol (NTP) servers

OEDA Configuration: Hardware Selection Screen
• ZDLRA supports flexible configuration for X5-2 and X6-2

models, while the X4-2 model has predefined templates.
• Choose the appropriate customer configuration and click

“Add.”
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e selection
The hardware li page identifies the number of racks that you want to deploy at the
b
em time, and the number of storage servers within each rack. The Recovery Appliance
esame
ak supports a flexible configuration. You can have between 3 and 14 storage servers in each
rack. Multiple racks must be cabled together.
Oracle Recovery Appliance X5-2 and X6-2 support Elastic Rack configurations. Elastic Rack
configurations give you flexibility at the time you need to configure the number of Storage
Nodes and racks. You will learn more about this later. Oracle Recovery Appliance X4-2 has
predefined configurations, starting with three Storage Nodes and expanding up to 14 Storage
Nodes per Rack.
For example, to deploy two full Oracle Recovery Appliance X4-2 racks (14 storage servers)
and one minimally configured Oracle Recovery Appliance X4-2 rack (3 storage servers) at the
same time:
1. Expand the X4 ZDLRA folder.
2. Add X6-2 ZDLRA 14 Cell HC 4TB twice.
3. Add X4-2 ZDLRA 3 Cell HC 4TB once.

OEDA Configuration: Rack Details Screen
• Specify the number of storage nodes.

• Include a spine switch (optional).
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Appliance
The Recovery li comes with two compute nodes and the number of storage nodes of
b
emchoice.
eyour
ak Click the check box if you want to include a spine switch as a part of the rack configuration
(for multipack deployments). Do this for each rack that will have a spine switch.

OEDA Configuration: Customer Network Requirements
• Ingest Network
• Accepts backups locally, similar to the client network in
Exadata
Network Description Ports Required IP Requirement
Ingest Network This required network 4 x 10 GbE connections Enable Jumbo Frames.
connects the protected (UTP or Fiber)
Oracle Database servers to The preferred scenario is
the Recovery Appliance for Active via LACP.
backup. A private Ethernet
network must be designed
a ble
to support the transfer of fe r
large volumes of data by
an s
using techniques such as
n - t r
jumbo packets.
a no
You can use this network if
h a s
you are connecting to third-
party media servers for m ) ideฺ
tape.
g ฺ co t Gu
There is VLAN port tagging nk
n en
n b
support on this network. a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
el network
This required
b li connects the protected Oracle Database servers to the Recovery
em for backup within the same data center. Also known as a backup network, this high-
eAppliance
ak speed, private Ethernet network must be designed to support the transfer of large volumes of
data by using techniques such as jumbo packets. The Recovery Appliance connects to this
network by using two 10 GB connections to each of the two compute servers in the rack. You
can configure the two connections as active/passive (redundant) or active/active. The
compute servers support channel bonding to provide higher bandwidth and availability.
The single client access name (SCAN) supports failover between the two compute servers in
the Recovery Appliance. In an installation with multiple Recovery Appliance racks configured
as a cluster, VIP addresses support failover among the racks. The protected database
systems can resolve the host names to dynamically assigned addresses.
Third-party tape hardware and software also use the ingest network.
Refer to the following note for additional information on VLAN port tagging support: Enabling
8021.Q VLAN Tagging in Zero Data Loss Recovery Appliance Over Ingest Networks (Doc ID
2047411.1)

• Replication Network
• Not used in a local configuration
Network Description Ports Required IP Address

Requirement
Replication This optional 4 x 10 GbE
Network replication network connections (UTP
makes connecting or Fiber) a ble
the local Recovery fe r
Appliance (the an s
upstream appliance) n - t r
with a remote a no
Recovery Appliance
h a s
(the downstream
m ) ideฺ
appliance) possible.
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e replication
The optional li network connects the local Recovery Appliance (the upstream
b
em with a remote Recovery Appliance (the downstream appliance). Oracle
eappliance)
ak recommends a broadband, encrypted network, instead of an insecure public network,
wherever possible.
The Recovery Appliance supports the following configurations between the upstream and
downstream appliances:
• One-way: Data flows from the upstream appliance to the downstream appliance.
• Bidirectional: Data flows in both directions between the upstream appliance and the
downstream appliance.
• Hub and spoke: Data flows from multiple upstream appliances to one downstream
appliance.

• InfiniBand Network
• Non-routable network automatically configured
Network Description Ports Required IP Address

Requirement
InfiniBand Network This network connects Internal to the Recovery You will require a subnet
the compute servers Appliance in OEDA for the IB
and storage servers by network.
using the InfiniBand IB partitions are not
a ble
switches. Oracle supported today. If you have to connect
fe r
Database uses this Because of this, to a second system
an s
network for Oracle RAC
cluster interconnect
Exadata or (Exadata or
SuperCluster running SuperCluster), you have n - t r
traffic and for accessing no
multiple IB partitions to use the same subnet
a
data on the storage
a s
cannot be connected. as that system, but
h
servers.
m ) ideฺ different IP addresses.
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e connects
This network li the compute servers and storage servers using the InfiniBand
b
em Oracle Database uses this network for Oracle RAC cluster interconnect traffic and
eswitches.
ak for accessing data on the storage servers. This non-routable network is fully contained in the
Recovery Appliance.
This network does not connect to your data center networks. It is automatically configured
during installation.

• Fiber Channel SAN Network

• Depends on if you are using Oracle or third-party tape
components
IP Address
Network Description Ports Required
Requirement
Fiber Channel Arbitrated If you are using the pre- You require two ports
Loop (FCAL) Network bundled media per server (four in a
manager, then you can rack).
a ble
back up the Recovery
fe r
Appliance to the storage You must use a SAN.
an s
area network (SAN) in
n - t r
your data center for
backups to tape. a no
The ingest network is
used to communicate
a s
with Automated
h
) ideฺ
Cartridge System
m
g ฺ co t Gu
Library Software
a n kn den (ACSLS, Oracle

StorageTek), if you are
i o nb s Stu partitioning libraries.

u n t h i
l lo@ o us e
e
kb se t
a a
( cen
If you are l l o li
eusing the pre-bundled Media Manager, then you can back up the Recovery
b
m to the SAN in your data center for backups to tape. The network connections
kee
Appliance
a depend on whether you have an Oracle tape solution or use third-party hardware.
• Pre-bundled Media Manager: When you use the Oracle-recommended tape solution, a
fiber channel adapter is installed in each compute server to provide a connection to the
fiber channel SAN. Tape backups are isolated on this network and, thus, do not interfere
with the performance of the other networks.
• Third-Party Tape Systems: When you use a third-party tape system, the backups to
tape use the 10 Gb ingest network. This is the same network that the local protected
databases use to back up to the Recovery Appliance.

• Management Network
• Connects through the Ethernet switch to the compute servers,
the storage servers, and the InfiniBand switches, and

connects directly to the PDUs
IP Address
Network Description Ports Required
Requirement
Management Network It connects through the One for each Compute
Ethernet switch to the Node, one for each
a ble
compute servers, the Storage Node, one for
fe r
storage servers, and the each InfiniBand Switch,
an s
InfiniBand switches. It
connects directly to the
and one for each PDU
n - t r
PDUs.
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l
The management
o li connects through the Ethernet switch to the compute servers, the
network
b
em servers, and the InfiniBand switches. It connects directly to the PDUs.
estorage
ak This (required) 1 gigabit, Ethernet (GbE) network connects to your existing management
network. You use it to do administrative work on all components of the Recovery Appliance.
The management network connects to the servers, Oracle ILOM, and InfiniBand switches
through the Cisco Ethernet switch in the rack.
There are three uplinks to the management network:
• One from the Ethernet switch
• One from each of the two PDUs. Network connectivity to the PDUs is required only if the
electric current is monitored remotely.
Each server has two network interfaces for management. One provides management access
to the operating system through the NET0 Ethernet interface, and the other provides access
to Oracle ILOM through the ILOM Ethernet interface. The Recovery Appliance is delivered
with the ILOM and NET0 interfaces connected to the Ethernet switch on the rack.
Cabling or configuration changes to these interfaces is not permitted. Do not use the
management network interfaces for client or application network traffic. Instead, you can use
NET1, NET2, NET3, or the two interfaces on the PCI slot.

OEDA Configuration: Customer Network Screen
• The ingest network accepts backups locally, similar to the client

network in Exadata.
• The replication network is optionally used for remote backup copies.

• The LACP is optional, and requires network/switch support at
customer site, if enabled.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Appliance
The Recovery li uses up to four networks. Each network must be on a separate
b
em from the others:
esubnet
ak • Subnet 1 (required): The administration network provides access to the servers and
switches in the Recovery Appliance, either directly or through Oracle ILOM. This
network typically uses a copper cable.
• Subnet 2 (required): The ingest network connects the Oracle Exadata Database
Machine to a local Recovery Appliance. This network can be bonded, and it uses either
a copper cable (1 Gb or 10 Gb) or an optical fiber cable (10 Gb). The Link Aggregation
Control Protocol (LACP) is optional. The data center must provide network and switch
support.
• Subnet 3 (required): The private InfiniBand network connects components within the
rack and, optionally, multiple racks together. This network uses InfiniBand cables.
• Subnet 4 (optional): The replication network connects a Recovery Appliance to an
optional, remote Recovery Appliance. This network can be bonded, and it uses either a
copper cable (1 Gb or 10 Gb) or an optical fiber cable (10 Gb). A replication network is
also called a backup, DR, or independent client network.

Other options in this screen:
• Subnet Mask: The subnet mask to use for the network
• Bonded or Non-Bonded: Identifies whether the network is bonded. The Admin and
Private InfiniBand networks are always non-bonded.
• Gateway: The IP address of the gateway for the network.
• Network Format: Identifies the appropriate Ethernet card for the cable type
- 1/10 Gb Copper Base-T for copper cables (typically for laptops and personal
computers)
- 10 GbE Optical for fiber optic cables (typically for data centers)
• Enabled LACP: The option for the ingest network if LACP is enabled in the data center
• Available Network: The option to configure the optional replication network
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

OEDA Configuration: Admin Network Screen
• The administration network connects through the Ethernet

switch to the compute servers, the storage servers, and the
InfiniBand switches, and connects directly to the PDUs.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l o
e serverli and storage server has two network interfaces for the administration
l
Each compute
b
k e em
network:
a • An Ethernet interface provides management access to the operating system.
• An ILOM Ethernet interface provides access to Oracle Integrated Lights Out Manager
(Oracle ILOM).
Other options in this screen include:
• Starting IP Address for Pool: The first IP address in a pool of consecutive addresses
assigned to the administration network port of the compute servers, storage servers,
and InfiniBand switches. Addresses in this pool are also assigned to the ILOM port of
the database and storage servers.
• Pool Size: This value is calculated from the hardware selection. The pool should consist
of consecutive IP addresses. If consecutive IP addresses are not available, then you
can modify individual IP addresses on the review page.

• Ending IP Address for Pool: This value is calculated from the starting IP address and
the pool size.
• Is the default gateway for database servers: Select this option if the gateway IP
address is the default gateway for the compute servers. Oracle recommends using the
ingest network gateway instead, because it has a bonded interface.
• Defines the host name for the database servers: Select this option if the
administration network names define the host names for the compute servers.
• (*) Sample host names: It displays sample names on the administration network, using
the prefix entered on the Customer Details page. To modify the name format, click
Modify.
(*) You can change the default administration host names by using the Administration
Network Format Masks dialog box. Names are typically changed to meet corporate naming
standards or for compatibility with existing systems.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

OEDA Configuration: Ingest Network Screen
• The ingest network provides the connection between Oracle

Database systems and the Recovery Appliance for bulk data
transfers. It is the backup network.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
The ingestl l o li
e network provides the connection between Oracle Database systems and Recovery
b
em for bulk data transfers. It is the backup network.
eAppliance
ak This screen shows the following options:
assigned to the ingest network port of the compute servers.
can modify individual IP addresses later in the configuration process.
the pool size.
• Is the Default Gateway for Compute Servers: Select this option if the gateway IP
address is the default gateway for the compute servers. Oracle recommends this
configuration.
• Defines the Host Name for the Compute Servers: Select this option if the ingest
network names define the host names for the compute servers. (*)
• Sample Ingest Names: Displays sample names on the ingest network, using the prefix
entered on the Customer Details page. To modify the format, click Modify.
(*) You can change the default ingest network host names on the Ingest Network Format
Masks dialog box.

OEDA Configuration: InfiniBand Network Screen
• An InfiniBand Network connects the compute servers and storage

servers through the InfiniBand gateway switches in the rack.
• The InfiniBand network can also connect multiple Recovery

Appliance racks.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e InfiniBand
The private li network connects the compute servers and storage servers through the
b
em gateway switches in the rack. The InfiniBand network can also connect multiple
eInfiniBand
ak Recovery Appliance racks.
This screen options:
assigned to the InfiniBand network port of the compute servers, storage servers, and
InfiniBand switches.
can modify individual IP addresses on the review page.
the pool size.
• Enable Active Bonding on Compute Node Network: Select this option to support
failover between the two compute servers on the InfiniBand network.
• Sample InfiniBand Names: Displays sample compute server and storage server
names on the InfiniBand network, using the prefix entered on the Customer Details
page. To modify the name format, click Modify.
You can change the format of the InfiniBand network host names by using the Private (IB)
Network Format Masks dialog box.

OEDA Configuration: Replication Network Screen
• The replication network is an optional network.

• It can be used to provide dedicated bandwidth for ZDLRA <-> ZDLRA
replication.
• When using replication, a separate network is highly recommended.
Without it, you would use up ingest network bandwidth.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e replication
The optional li network connects the Recovery Appliance to another (downstream)
b
em Appliance. It provides dedicated bandwidth for this task. Complete this page if you
eRecovery
ak have a replication network.
If a replication network is unavailable, then you can use the ingest network to connect to a
downstream appliance. However, this configuration creates competition for bandwidth on the
ingest network.
This screen shows the following options:
assigned to the replication network port of the storage servers. The pool size identifies
the required number of IP addresses. If consecutive IP addresses are not available, then
you can modify individual IP addresses on the review page.
• Pool Size: This value is calculated from the hardware selection.
the pool size.
• Sample Replication Name: Displays a sample compute server name on the replication
network, using the prefix entered on the Customer Details page. To modify the format,
click Modify.
You can change the replication host name format by using the Replication Network Masks
dialog box.

OEDA Configuration: Compute Node OS Screen
The defaults here are required.

a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Appliance
The Recovery li runs the Linux operating system. There are no choices to be made
b
ak eonem
this page.

OEDA Configuration: Review and Edit Screen
• Ensure that data is correct and/or update it.

• If you change the ZDLRA physical configuration, you should
regenerate data.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Use this l l o li the network names and IP addresses assigned to each device. If you
epage to review
b
em a pool of IP addresses that are not all available for use, then replace the duplicate
eidentified
ak addresses on this page. You can edit the host names and IP addresses just by clicking the
field. All host names and IP addresses must be unique.
• Regenerate Data: Click this button to see the current list of generated names, after
modifying the names or format masks.
• Lookup IP: It checks that the IP address matches the device name, and corrects it if
they do not match.

OEDA Configuration: Cluster Info Screen
• Only one cluster is allowed.

• Bring over all the nodes that you want in the active
configuration.
• Ensure that the cluster name is unique here.
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
You can
b i
l one lcluster
ecreate on the Recovery Appliance.
em
eOptions:
ak • Cluster Name (required): A unique name for the cluster. The name must start with an
alphanumeric character. It can contain additional letters, numbers, periods (.), and
hyphens (-). The default name is cluster-clu1.
• Available: It lists all compute servers and storage servers included in this installation.
• Selected: It lists all compute servers and storage servers that are assigned to the
cluster. Click All to assign all servers to the cluster.

OEDA Configuration: Cluster Details Screen
• Some fields in this screen are locked out.

– These are set specifically for the appliance.
• Greyed out fields show their values, but they cannot be

changed.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li have values that you provided on earlier configuration pages. You
e on this page
Many fields
b
k e emreview these values and revise them if necessary. The greyed out fields are read only,
can
a and are shown only to provide additional information about the installation.
• Cluster: The values at the top of the page are derived from the rack configuration
settings on the Customer Details screen.
• Users and Groups: This section is read only. It displays the users and groups defined
for Oracle Database on the Recovery Appliance. You do not access this database
directly, because it exclusively supports the backup and recovery of protected
databases.
• Software Locations: This section identifies the location of the Oracle Database
software on the Recovery Appliance. Most fields are read only.
• Disk Group Details: This section defines the disk groups for Oracle Database on the
Recovery Appliance. Most fields are read only. Oracle recommends HIGH redundancy
for DELTA ASM disk groups only for mission-critical applications.
• Initial Database: It includes the Cluster Database name and character set.

• Ingest Network: Review the values for the ingest network, and correct them as
necessary. The values are primarily derived from the Ingest Network screen.
• Replication Network: Review the values for the replication network, and correct them
as necessary. The values are primarily derived from the Replication Network screen.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

OEDA Configuration: Ingest Cluster Networking Screen
• SCAN, Ingest network, VIP network, and replication network

a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Use this l l o li the network names and IP addresses on the SCAN, ingest network,
epage to review
b
emnetwork, and replication network. If you identified a pool of IP addresses that are not all
eVIP
ak available for use, then replace the duplicate addresses on this page. You can edit the host
names and IP addresses just by clicking the field. All host names and IP addresses must be
unique.
• Regenerate Data: Click this button to update the list of generated names, if you
modified any names or format masks.
• Lookup IP: Checks that the IP address matches the device name, and corrects it if they
do not match.

OEDA Configuration: Replication SCAN Screen
• Optional Replication Network VIP/SCAN configuration

• This screen will exist if the replication network is selected.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li if you previously configured a replication network. The Recovery
e appears only
This page
b
em defines a SCAN for the rack, and a VIP for the compute servers. All names and IP
eAppliance
ak addresses must be unique.
• Regenerate Data: Click this button to update the list of generated names, if you
modified any names or format masks.
• Lookup IP: Checks that the IP address matches the device name, and corrects it if they
do not match.

OEDA Configuration: Cell Alerting Screen
The Recovery Appliance does not require cell alerting but you
must configure one of the following alerting options:
• Cell alerting
• OCM
• ASR
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li when a problem is detected in a storage server. The alert can be
e are triggered
Cell alerts
b
em using the Simple Mail Transfer Protocol (SMTP), Simple Network Management
edelivered
ak Protocol (SNMP), or both.
The Recovery Appliance does not require cell alerting, but you must configure one of these
alerting options:
• Auto Service Request (ASR)
• Cell alerting
• Oracle Configuration Manager (OCM)

OEDA Configuration: Platinum Support Screen
• Enter the Customer Platinum configuration information here.

• This is currently an optional screen.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
OraclebPlatinum li provide enhanced support at no additional cost for qualified
Services
k e em
engineered systems. This is a connected service and requires the installation of Oracle
a Advanced Support Gateway software within the system.
The Oracle Advanced Support Gateway is configured with the user information that is
required by the Platinum Service monitoring agent.
• Customer Details: These include customer name, customer support identifier, and My
Oracle Support email address that corresponds to the CSI.
• Platinum Gateway Details: The Recovery Appliance can use an existing Oracle
Advanced Support Gateway in the network. Otherwise, you can create a gateway on a
server or a supported virtual machine, which is connected to the same management
network or ingest network as the Recovery Appliance.
• Platinum Agent Configuration Details: Oracle Platinum Services uses an agent to
monitor the health of the Recovery Appliance. Use the default configuration settings for
the agent, unless you have a specific reason to change them. For example, the default
port number might be assigned already to another device.

OEDA Configuration: Auto Service Request Screen
• ASR is not required if Platinum is enabled

a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Request
Auto Service li (ASR) automatically opens service requests when specific hardware
b
em occur.
efaults
ak The Recovery Appliance does not require an ASR, but you must configure one of the
following alerting options:
• ASR
• Cell alerting
• Oracle Configuration Manager
Options in this screen:
• Enable Auto Service Request: To use ASR with the Recovery Appliance and then
enter the configuration details
• ASR Manager Host Name: The host name of the server where the ASR Manager is
installed
Note: Oracle recommends a stand-alone server that has connectivity to the Recovery
Appliance.
• ASR Technical Contact Name: The name of the technical contact person for the
Recovery Appliance

• Technical Contact Email: The email address of the technical contact person
• My Oracle Support Account Name: The email address of the My Oracle Support
account
• HTTP Proxy used in upload to ASR: Select if you use an HTTP proxy to upload to the
Oracle ASR site, and then enter the proxy details
• HTTP Proxy Host: The host name of the proxy server
• Proxy Port: The port number for the HTTP proxy server. The default is 80.
• HTTP Proxy requires authentication: Select if the HTTP proxy server requires
authentication
• HTTP Proxy User: The username for the proxy server. If the connection requires a
password, then you must provide it during installation.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

OEDA Configuration: Configuration Manager Screen
• Customer information for Oracle Configuration Manager

a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li Manager collects and uploads configuration information to an
e Configuration
The Oracle
b
em repository for access when needed by Oracle Support.
eOracle
ak The Recovery Appliance does not require the Oracle Configuration Manager, but you must
configure one of the following alerting options:
• ASR
• Cell alerting

OEDA Configuration: Tape Library Screen
• ZDLRA tape configuration by using OSB

• Enable Admin VIP: This enables OSB as a CRS Service.
• Most libraries are Default SCSI. This is an industry standard using the FC-
SCSI interface for robotic control.
• You can add one or more tape libraries here, but usually just one is added.
• All tape libraries will be hosted inside the same domain.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
You can l l o li tape library for the Recovery Appliance only if you use the pre-bundled
econfigure the
b
em Manager for tape backups. You can manually configure other media managers as
eMedia
ak clients only.
Options:
• Use Tape: Select this option if you plan to use the pre-bundled Media Manager for tape
backups. Then you can enter the configuration details.
• Number of Tape Libraries: It denotes the number of tape libraries available to the
Recovery Appliance. All tape libraries must be hosted in the same domain. A separate
subpage is created for each library, so that you can enter the configuration details.
• Enable Admin VIP: Select this option to create a VIP on the ingest network and enable
the Media Manager as a Cluster Ready Services (CRS) service. Then you can enter the
VIP details.
• VIP Name: This is the alphanumeric name of the VIP.

• VIP IP: It is the IP address of the VIP.
• Library Connectivity: Select the appropriate software for your tape system:
• ACSLS ACS API: The ACSLS API has these fields:
- ACSLS Server: The name of the server running ACSLS
- ACS ID: The ACS identification number
• ACSLS FC-SCSI: The ACSLS FC-SCSI has these fields:
- ACSLS Server: The name of the server running ACSLS

- Logical Library ID: The logical library identification number
- SCSI: The Simple SAN tape library has no configuration fields by default.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

OEDA Configuration: Comments Screen
• Installer Comments
• There might be default questions here to help the person
installing the software.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Enter any
l o
eadditionalliinformation about the deployment in the text field, particularly anything of
l
b
k e emto the person performing the installation. The information appears at the bottom of
help
a InstallationTemplate.html, which is one of the generated files.

OEDA Configuration: Generate Files Screen
• To create the configuration files, click Next.

• In the pop-up window, select a destination directory and click
Save.
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
Selectb eldirectory lini which you will save the configuration files.
the
k e em
a

OEDA Configuration: Finish Screen
• The Summary page lists saved configuration files.

• Click Finish to end config.sh.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l o
e page listsli the names of the generated files. Click Finish to close the utility.
l
A summary
m b
a kee

A sample list of configuration files:
host:ExadataConfigurations user$ ls -l
total 1376
-rw-r--r-- 1 user staff 204778 Jul 26 11:21 Test_Customer-demo-
InstallationTemplate.html
-rwxr--r-- 1 user staff 574 Jul 26 11:21 Test_Customer-demo-
checkip.sh
platinum.csv
preconf_rack_0.csv
-rw-r--r-- 1 user staff 218159 Jul 26 11:21 Test_Customer-
demo.xml
a ble
-rw-r--r-- 1 user staff 218116 Jul 26 11:20 Test_Customer- fe r
ans
demo.xml.26-Jul-16_112101
n - t r
-rw-r--r-- 1 user staff 27853 Jul 26 11:21 Test_Customer-
a no
demo.zip
h a s
-rw-r--r-- 1 user staff 8761 Jul 26 11:21
m d ฺ
) databasemachine.xml
e
ฺ co t Gu i
host:ExadataConfigurations user $ g
kn den
a n
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o li
b
k e em
a

OEDA Configuration: Installation Template
• This summary page shows information provided by the

customer as it was given during the configuration assistant
session, and the resulting defaults.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Template
The Installation li report includes the following sections with detailed information:
b
ak ee•m Management Network Configuration
• Ingest Network Configuration
• Private InfiniBand Network Configuration
• Replication Network Configuration
• Clusters Information
• Rack Machines Overview
• Platinum Configuration
• Tape Library Connectivity
• Alerts
• Appendix A: Network Names and IPs
• Appendix B: Required Files and Patches

OEDA Installation: Steps Overview
1. Get the current software

2. Run ra_preinstall.pl
3. Run OEDA Installer: install.sh

4. Run ZDLRA Installer: ra_install.pl
5. Run EM Discovery
After this, the installation is complete and the system is ready to
hand off to the customer. a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
After an
l o
eOracle ACSliengineer completes these steps, the installation is complete and the
l
b
em is ready to hand off to the customer.
esystem
ak

OEDA Installation: Get the Current Software - Step 1
• Oracle Database 12c (12.1.0.2)

• Oracle Grid Infrastructure 12c (12.1.0.2)
• Oracle Database Patch for Engineered Systems and DB In-

Memory 12.1.0.2.13 (OCT2015)
• Patch 22304421 - EXADATA: ORA-00600
[FLASHCACHE::ISSUEIO INV PINPAGE(R) RESULT] WITH
21976960
r a ble
• OPatch version 12.2.0.1.5 for Oracle software releases sfe
(installer) 12.1.0.x and 12.2.0.x (JUL 2016) - t r an
n on
• OneCommand Apr 2016 v16.112 - Patch 23125053
h a sa
• Exachk Patch 18622611 m ) ideฺ
ฺ co t Gu
• ZDLRA RA Automation RPM (12.1.1.1.7-22998011
g PATCH)
k n n
n udeManagement Tools
• QLogic QConvergeConsole nbaGUI/CLI
St
u nio this
l l o @ use
be e to
a k
l o (a cens
As instructed
b i
el by thelOEDA Installation Template, you must download the following files from
emOracle Support (MOS) into the OEDA's WorkDir directory in the first Compute Node:
eMy
ak • Database 12c software
- linuxamd64_12102_database_1of2.zip
- linuxamd64_12102_database_2of2.zip
• Grid Infrastructure 12c software
- linuxamd64_12102_grid_1of2.zip
- linuxamd64_12102_grid_2of2.zip
• Database Patch for Engineered Systems and DB In-Memory 12.1.0.2.13 (OCT2015).

Note: At the time of creating this seminar this patch was originally replaced by patch
22243551 (JAN2016), but the most recent replacement for this patch is 22899531
(APR2016). Do NOT apply any other Oracle Server software patches unless directed by
Oracle Support.
• p21744410_121020_Linux-x86-64.zip
EXADATA: ORA-00600 [FLASHCACHE::ISSUEIO INV PINPAGE(R) RESULT] WITH
21976960
• p22304421_1210213DBEngSysandDBIM_Linux-x86-64.zip
OPatch patch of version 12.2.0.1.5 for Oracle software releases (installer) 12.1.0.x and
12.2.0.x (JUL 2016)
• p6880880_121010_Linux-x86-64.zip
Exachk utility. Refer to the following support note to get the latest version: Oracle Exadata
Database Machine exachk or HealthCheck (Doc ID 1070954.1) and Zero Data Loss Recovery
Appliance Supported Versions (Doc ID 1927416.1) a ble
fe r
• exachk.zip
a n s
r
-t in addition
ZDLRA RA Automation RPM (includes ra_preinstall.pl and ra_install.pl perlnscripts
o
to other required files)
s an
• ra_automation-12.1.1.1.7-22712784.x86_64.rpm
) ha eฺ
QLogic QConvergeConsole GUI/CLI Management Tools
ฺ c om G u id Library integration is
(if the Tape
n k ng ent
enabled) Go to http://driverdownloads.qlogic.com/QLogicDriverDownloads_UI for download.
n b a
• QConvergeConsoleCLI-1.1.03-22.x86_64.rpm
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
b el li
k e em
a

OEDA Installation: Pre-Install Check - Step 2
• The ra_preinstall.pl script performs initial checks and

prepares the Recovery Appliance for new installations and
upgrades.
• The script is included in the Recovery Appliance patch .zip
file.
• It should be run from there prior to new installs and upgrades.
[root@zdlraadm01 WorkDir]# /usr/bin/perl ra_preinstall.pl \
a ble
--config_xml=/u01/ra_install/linux-64/ExadataConfigurations/ZDLRA1AutoInstall-zdlra01.xml \
fe r
--oeda_dir=/u01/ra_install/linux-64/ --oracle_uid=4000 --dba_gid=4000 --oinstall_gid=4001
Updating OEDA for ra_automation-12.1.1.1.7-23494283.x86_64
an s
ra_automation-12.1.1.1.7-23494283.x86_64.rpm and /u01/ra_install/linux-
n - t r
no
64//WorkDir/ra_automation-12.1.1.1.7-23494283.x86_64.rpm match. Skipping copy.
[root@zdlraadm01 WorkDir]#
s a
h a
) ideฺ
m
co t Gu
g ฺ
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o li performs initial checks and prepares the Recovery Appliance for new
b
ra_preinstall.pl
k e em
installations and upgrades.
a The command shown in this slide shows an example of executing the ra_preinstall.pl
script.

Reference:
RA_PREINSTALL(1) User Contributed Perl Documentation
RA_PREINSTALL(1)
NAME
ra_preinstall.pl
DESCRIPTION
Script to patch the config xml with the new patch bundles and
update es.properties with the current rpm name.
ARGUMENTS
The script takes the following arguments
--config_xml (REQUIRED)
--config_xml=CONFIG.XML_PATH The path where the config.sh xml
a ble
file is located.
fe r
--oeda_dir (REQUIRED)
a n s
--oeda_dir=OEDA_DIRECTORY_PATH The path of the oOEDA n r
-t directory.
--oracle_uid (OPTIONAL) s an
--oracle_uid=NEW_USER_ID The new oracle ) hausereฺid. Value must be
>= 1001 ฺ c om Guid
--oinstall_gid (OPTIONAL) n k ng ent
n b a Thetudnew oinstall group id. Value
nio to
--oinstall_gid=NEW_GROUP_ID
i s S
must be >= 1001 and not uequal
@ e t h 1011 or 1012.
e
--dba_gid (OPTIONAL)
b llo to us
a ak nse
--dba_gid=NEW_GROUP_ID
( The new dba group id. Value must be >=
1002 andllonot equal e
lic to 1011 or 1012.
b e
e m--help (OPTIONAL)
a ke Provides the scripts help message.
EXAMPLES
./ra_preinstall.pl --config_xml=/scratch/ZDLRA7_Auto_Install-
scao09.xml --oeda_dir=/scratch/linux-x64 --oracle_uid=5555 --
oinstall_gid=5555 --dba_gid=5555 --help
COPYRIGHT AND LICENSE

Copyright (c) 2013, 2016, Oracle and/or its affiliates. All
rights reserved.
perl v5.18.2 2016-07-07
RA_PREINSTALL(1)

OEDA Installation: Installation Script - Step 3
• The install.sh script is used to run the OEDA in installation

mode.
• The script is run by the root user.

• It can run or undo all the steps, a specific step, or a range of steps.
[root@scaz15adm03 linux-x64]# ./install.sh -cf ZDLRA7_Auto_Install-scao09.xml -l
1. Validate Configuration File
2. Setup Required Files
3. Create Users
4. Setup Cell Connectivity
5. Verify InfiniBand
6. Calibrate Cells
a ble
7. Create Cell Disks
fe r
8. Create Grid Disks
an s
9. Configure Alerting
10. Install Cluster Software
n - t r
no
11. Initialize Cluster Software
12. Install Database Software
13. Relink Database with RDS
s a
14. Create ASM Diskgroups
15. Create Databases h a
) ideฺ
16. Apply Security Fixes m
co t Gu
17. Install Exachk
g ฺ
kn den
18. Setup ASR Alerting
19. Create Installation Summary
a n
nb s Stu
20. Resecure Machine
n i o i
u e t h
a a kb se t
l o ( cen
e l
The install.sh
b li uses the files generated by OEDA to configure the networks, Oracle
script
em and other software components. The script is run by the root user.
eDatabase,
ak The command shown in the slide shows an example for listing all the steps for a specific
installation.

Reference:
[root@slcm03adm03 linux-x64]# ./install.sh --help
install.sh -cf <config.xml> -l [options]
install.sh -cf <config.xml> -s <step #> | -r <num-num>
install.sh
ARGUMENTS:
-l List all the steps that exist
-cf config file name [Full path please]
-s <step #> Run the step # at a time
-r <num-num> Run the steps one after the other as long as no
errors
are encountered
r a ble
-u <num-num> | <step#> Undo a range of steps or a particularfestep
a n s
-h Usage
n -t r
o
Version: 15.153.07:00
s an
) ha eฺ
[root@slcm03adm03 linux-x64]#
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
b el li
k e em
a

OEDA Installation: RA Installation Script - Step 4
• The ra_install.pl script is used to perform additional

configuration steps.
• The script is run by the root user.

• The script can install or uninstall a specific step.
[root@scaz15adm03 linux-x64]# ./ra_install.pl --help
ra_install.pl - Recovery Appliance Installer
You must be logged in as root to run this command.
All steps should be ran successfully for install.
Usage: ./ra_install.pl --help | --step=STEP_NUMBER {--install|--uninstall} [--stdout]
a ble
Options:
fe r
--help: displays this help message
--step=number: Which step to run
an s
--stdout: Print all messages to stdout rather then the log file
n - t r
no
--install: Installs the step [Default]
--uninstall: Uninstalls the step
Step Numbers:
s a
Step 1 - Validation and Configuration Prep
Step 2 - OS Setup h a
) ideฺ
Step 3 - Oracle User Setup
m
co t Gu
Step 4 - DBFS Setup
g ฺ
kn den
Step 5 - Tape Backup configuration
n
Step 6 - ZDLRA DB Backup Setup
a
nb s Stu
Step 7 - Enable ZDLRA Services
n i o i
u e t h
a a kb se t
l o ( cen
e l
The ra_install.pl
b li script performs additional configuration steps for the Recovery
em infrastructure. You must run this script while logged in to a Recovery Appliance
eAppliance
ak compute server, and not from another server on the same network.
The command shown in the slide shows an example for listing all options and steps.

Reference:
[root@slcm03adm03 linux-x64]# ./ra_install.pl --help
ra_install.pl - Recovery Appliance Installer
You must be logged in as root to run this command.
All steps should be ran successfully for install.
Usage: ./ra_install.pl --help | --step=STEP_NUMBER {--install|--

uninstall} [--stdout]
Options:
--help: displays this help message
--step=number: Which step to run
--stdout: Print all messages to stdout rather then the log
file
a ble
--install: Installs the step [Default]
fe r
an s
--uninstall: Uninstalls the step
n - t r
Step Numbers:
a no
Step 1 - Validation and Configuration
h a s Prep
Step 2 - OS Setup m ) ideฺ
ฺ o Gu
cSetup
Step 3 - Oracle User
k g
n ent
Step 4 - DBFS b n
aSetuptud
- io
n s S configuration
Step 5 n
u i
Tape hBackup
t
l l
Step o @ 6 -u s e
ZDLRA DB Backup Setup
e
kb Step o
t7 - Enable ZDLRA Services
a a n
( celinux-x64]#s e
l l o
[root@slcm03adm03
e l i
b
k e em
a

OEDA Installation: EM Discovery - Step 5
• Discovering Targets:
– Cluster: Oracle Cluster and High Availability Services
– Cluster Database: RAC Database, Listeners, and ASM Instances, and

ASM Disk Groups
– Recovery Appliance: Compute Nodes, Storage Nodes, IB Switches,
management switch, and PDUs
Cluster Recovery
Database Appliance
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li Manager to administer the Recovery Appliance, you must deploy the
e the Enterprise
To enable
b
em
emanagement agents to each compute server, then discover the targets for the Recovery
ak Appliance. The targets include the cluster, database, listeners, ASM, and the appliance itself.
Refer to the following support note for additional information: Prerequisites for Using the
Oracle Zero Data Loss Recovery Appliance Plug-in (12.1.0.1) (Doc ID 1929507.1)
You will learn more later about configuring the Recovery Appliance with Oracle Enterprise
Manager Cloud Control.

Network Test Throughput
The network_throughput_test.sh script uses Qperf from

the client machine back to the Recovery Appliance.
• It requires SSH.
• The execution node may be one of the nodes being tested.
[root@scaz15adm03 linux-x64]# ./network_throughput_test.sh \

-s sending_nodes -r receiving_nodes -i bondeth0
Using 'scam09db01 scam09db02 scam09db03 scam09db04 scam09db05 scam09db06 scam09db07 scam09db08'
hosts for sending nodes
a ble
Using 'scas10adm01 scas10adm02 scas10adm03 scas10adm04' hosts for receiving nodes
fe r
Validating Trusted SSH to sending nodes scam09db01 scam09db02 scam09db03 scam09db04 scam09db05
an s
scam09db06 scam09db07 scam09db08 ... OK
Validating Trusted SSH to receiving nodes scas10adm01 scas10adm02 scas10adm03 scas10adm04 ... OK
n - t r
Total Network Bandwidth 8,372,597,121 bytes/sec
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l o
e helps youlito identify the theoretical network throughput between a Protected
This scriptl
b
em system and a Zero Data Loss Recovery Appliance, or between two Zero Data Loss
eDatabase
ak Recovery Appliances in a replication configuration.
Refer to the following support note for additional information: Zero Data Loss Recovery
Appliance Network Test Throughput script (Doc ID 2022086.1)
Reference:
# ./network_throughput_test.sh -h
Usage: ./network_throughput_test.sh -s [sending_nodes_file] -r
[receiving_nodes_file] -i [interface]

Agenda
• Installation Process
• Extending and Expanding the Recovery Appliance
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Expanding the Recovery Appliance
• Base Rack
– 2x Compute Servers with high-speed connectivity
— Eight 10gb Ethernet ports per rack

— Twelve 40Gb InfiniBand ports per rack
— Optional four 16Gb Fibre Channel ports for tape connectivity
– 3x Storage Servers
— Each storage server has 12 high-capacity disks
— Usable capacity for incoming backups in base rack: 94TB
• Choose the number of additional storage servers at the time of purchase or later to
increase capacity.
– 32TB usable capacity per storage server
a ble
• Full Rack: 2x Compute Servers, 18x Storage Servers, 580TB usable capacity fe r
an s
Extra Storage
n - t r
a no
Recovery
h a sAppliance
m ) ideฺ Base Rack
g ฺ co t Gu
a n kn den Fully
nb s Stu
Redundant
n i o i
u e t h
a a kb se t
l o ( cen
l
e Appliance
The Recovery
b li racks have a flexible configuration that allows a minimum of three
em servers. You can extend a rack up to a total of 18 storage servers for Recovery
estorage
ak Appliance X6.
The base configuration includes two compute servers and three storage servers internally
connected by using high-speed InfiniBand. It provides a highly available configuration with 94
TB of usable capacity for incoming backups. The base rack can be upgraded incrementally by
adding additional storage servers into the rack, up to a maximum of 18 storage servers in a
full rack. Each storage server adds 32 TB of usable capacity. The total usable capacity of a
full rack is 580 TB, with an effective capacity of up to 5.8 PB of virtual full backups.

The Recovery Appliance can protect databases whose total size is approximately the same
as the available capacity of the appliance for a typical recovery window of 10 days.
For example, a single full-rack configuration with two compute servers and 18 storage servers
that has a 580 TB usable capacity can protect approximately 580 TB of source database for a
10-day recovery window, storing ten 580 TB virtual full backups plus all the redo data
generated for that 10-day period.
Accurate sizing of the Recovery Appliance depends on several factors related to protected
databases including the initial database size and growth rate, storage consumed by temp and
undo, free space, database change rate, redo generation rate, desired recovery window, and
compressibility of the database.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Extending the Recovery Appliance
• Multi Rack: Add InfiniBand connected racks to scale

• Can expand with new-generation hardware
• Scale to 18 Racks
– Up to 216TB/hour Delta Ingest and Restore
– More than10 PB of usable capacity
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
You can l l o li
eadd racks together to increase system capacity and performance. The Recovery
b
em has a flexible configuration. A rack has two compute servers and a minimum of
eAppliance
ak three storage servers. A Recovery Appliance X6 or X5 full rack has 18 storage servers and a
Recovery Appliance X4 full rack has 14 storage servers. The maximum configuration is 18 full
racks.
If additional capacity is required, a second base rack can be connected via high-speed
InfiniBand to the first rack. The second rack includes its own pair of compute servers which
add connectivity and processing power to the configuration. As with the first rack, storage
capacity can be easily expanded by incrementally adding storage servers. Up to 18 fully
configured racks can be connected together into a single appliance, providing more than 10
PB of usable capacity, i.e. more than 100 Petabytes of Virtual Full Backups.
The power and flexibility of the Recovery Appliance scale-out architecture is revealed when
there is a need to support additional databases, or when business data grows. Storage,
compute, and network capacity is incrementally added in a balanced fashion that maintains
high performance. This architecture is far superior to traditional backup appliances which are
usually limited to two controllers and therefore cannot scale storage, networking, and compute
in a balanced, bottleneck-free fashion.

Summary

• Identify various steps to install the Oracle Recovery Appliance
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

5
Zero Data Loss Recovery Appliance

Configuration a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake
Objectives

• Describe the Recovery Appliance configuration tasks
• Create Protection Policies for a group of databases

• Explain how Recovery Appliance uses metadata
• Perform configuration tasks in the Recovery Appliance
Storage components
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Agenda
• Configuration: Overview
• Recovery Appliance Administrative Interfaces
• Oracle Enterprise Manager Discovery Process for Recovery

Appliance
• Recovery Appliance Administration Roles
• Protection Policies
• Recovery Appliance Metadata
r a ble
• Recovery Appliance Storage n s fe
a - tr
no n
h a sa
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li
m be
e
ake

Recovery Appliance Configuration Tasks
General tasks:
• Planning
• Separation of Duties: User Accounts

• Protection Policies for Protected Databases
• Network Access to Recovery Appliance for Protected
Databases
• Recovery Appliance Metadata Database
r a ble
• Virtual Private Catalog n s fe
n - tra
• Scheduler no
• Storage Nodes (Cell) s a
h a
) ideฺ
• ASM m
ฺcon
u
g nt G
k
an tude
n b
u nio this S
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Agenda

Appliance
r a ble
a - tr
no n
h a sa
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li
m be
e
ake

Recovery Appliance Administrative Interfaces
• Enterprise management tools include:

– Oracle Enterprise Manager Cloud Control
– Automatic Service Request

– Oracle Configuration Manager
• Secure Shell (SSH) allows remote access. Nonetheless, local
access might be necessary during initial configuration.
• ILOM enables remote management and administration of
hardware. r a ble
n s fe
• The dcli utility runs commands on multiple nodes in parallel
tra
threads. n -
a no
• SQL*Plus works as a command-line tool or h a s
SQL Developer as a GUI tool that enables
m ) ideฺ
you to run DBMS_RA program n g ฺcoand
units, t G u
query
k n
recovery catalog views. ban tude
n i on is S
@ u e th
( a ak nse
e l
Administration l ce
lo Interfaces
i
m b
e• Cloud Control: This is a system management tool with a graphical user interface that
ake enables you to manage and monitor the Recovery Appliance and its protected
databases. This is the preferred UI for Recovery Appliance tasks.
• Secure Shell (SSH) allows remote access, but local access might be necessary during
initial power-on or troubleshooting.
• ILOM provides a remote management console using multiple protocols, such as
HTTP/HTTPS, IPMI, SNMP, and so on.
• With local access, you have access to the node operating system as well as the ILOM.
• You use SQL statements and Oracle-supplied PL/SQL packages to complete these
tasks in SQL*Plus or SQL Developer.
• The dcli utility facilitates centralized management across an server realm by
automating the execution of OS and utilities commands on a set of servers. The utility
then returns the output to the centralized management location where the dcli utility
was run.

Agenda

Appliance
r a ble
e
• Recovery Appliance Storage nsf tr a
n -
no
h a sa
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li
m be
e
ake

Recovery Appliance Discovery
Prerequisites for Discovering a Recovery Appliance Target:

1. Discover Recovery Appliance Host (Compute Server)
– Will include EM agent deployment and Recovery Appliance plug-in

installation
2. Discover Recovery Appliance Cluster and Cluster Database
– Will include Cluster software (Clusterware)
– Will include Cluster Database software (Database, Listeners, and ASM)
a ble
fe r
ZDLRA EM Agent
an s
and OMS plug-ins
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
b
Recovery
l
eAppliance li Discovery Prerequisites
k e em Server
Compute
a If agents are already installed on the Recovery Appliance compute servers and uploading to
the Oracle Management Server, proceed to the Recovery Appliance Discovery Prerequisites
section below. Otherwise, management agents must be pushed to each compute server by
using the following procedure:
1. From the Setup menu, select Add Target, select Add Targets Manually, choose Add
Host Targets, and then click Add Host.
Enterprise Manager displays the Add Host Targets: Host and Platform page.
2. On the Host and Platform page, for the Platform choice, select Same for All Hosts. Use
the Add button to add all Recovery Appliance compute servers to the list (specify the
fully qualified host name of each server). Select the proper platform for the first server in
the list. Click Next.
Enterprise Manager displays the Add Host Targets: Installation Details page.

3. On the Installation Details page, provide the Agent installation directory (which must
already exist on each compute server), Named Credential for the user that will own the
Agent installation, and other requested properties. Click Next.
Enterprise Manager displays the Add Host Targets: Review page.
4. On the Review page, click Deploy Agent to deploy Agents to all Recovery Appliance
compute servers.
Cluster and Cluster Database

The cluster and cluster database that reside on the Recovery Appliance system must be
discovered prior to discovering the Recovery Appliance target.
Steps
1. From the Setup menu, choose Add Target, select Add Targets Manually.
Enterprise Manager displays the Add Targets Manually page.
2. On the Add Targets Manually page, select the Add Targets Using Guided Process (Also
a ble
Adds Related Targets) option.
fe r
ans
3. From the Target Types drop-down list, choose Oracle Cluster and High Availability
Service, then click Add Using Guided Process… n - t r
no
Enterprise Manager displays the Cluster Discovery: Specify Host page.
a
4. On the Specify Host page, specify one of the host compute h a s
servers on the Recovery
Appliance, then click Next. m ) ideฺ
g ฺ
Enterprise Manager displays the Cluster Discovery: co Results
t G u page.
n n en
kinformation
b a
5. Review the automatically detected cluster
t u d on the Results page, verify that all
compute servers on the Recovery n
io Appliance
s S cluster are included in the list of cluster
u n h i
hosts at the bottom of the page, thent click Next.
o @ the
Enterprise Managerlldisplays u s e
Cluster Discovery: Review page.
6. On the Reviewa k be click
page, s e o The cluster target will be created.
tSave.
o ( a en
7. Return
b e l lic Targets Manually page (as per step 1 above). On the Add Targets
l to the Add
m Manually page, select the Add Targets Using Guided Process (Also Adds Related
e
ke Targets) option.
a 8. From the Target Types drop-down list, choose Oracle Database, Listener, and High
Availability Service, then click Add Using Guided Process…
Enterprise Manager displays the Discovering Targets: Search Criteria page.
9. On the Search Criteria page, specify one of the hosts in the Recovery Appliance cluster,
select the On all hosts in cluster option, then click Next.
Enterprise Manager displays the Discovering Targets: Results page.
10. On the Results page, the Recovery Appliance cluster database should be listed, along
with the ASM instances and listeners on all compute servers. Review and revise the
target information as needed, click Configure for each target to provide appropriate
database monitoring credentials, then click Next.
Enterprise Manager displays the Discovering Targets: Review page.
11. On the Review page, click Finish. The cluster database, ASM, and listener targets will
be created.

Recovery Appliance Discovery
Manually Discovering a Recovery Appliance Target:

• Previous steps
– Discover Cluster and Cluster Database targets

• Discover Recovery Appliance itself
– Will include software and hardware
Guided Discovery
a ble
Workflow
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
Recoveryb l
eAppliance li Manual Discovery
k e em
Steps
a After discovering the cluster and cluster database, the Recovery Appliance target itself can be
discovered. As a part of discovering the top-level Recovery Appliance target that will include
all Enterprise Manager Recovery Appliance software and hardware management
functionality, a separate Recovery Appliance hardware target will also be discovered.
Follow these steps to manually discover a Recovery Appliance target:
1. From the Setup menu, choose Add Target, then select Add Targets Manually.
Enterprise Manager displays the Add Targets Manually page.
2. On the Add Targets Manually page, select the Add Targets Using Guided Process (Also
Adds Related Targets) option.
3. From the Target Types drop-down list, choose Recovery Appliance, then click Add
Using Guided Process. Enterprise Manager displays the Recovery Appliance Hardware
Discovery page.

4. Select the Discover new Recovery Appliance hardware components as the targets option.
Click Discover Targets.
5. Follow the detailed instructions in all subsequent pages of the Recovery Appliance
Hardware Discovery wizard, supplying all requested credentials for hardware
components. On the Review page, review all hardware component details, then click
Submit. The Database Machine target representing the Recovery Appliance hardware will
be created, along with targets for all hardware components. Enterprise Manager displays
the Target Creation Summary page, showing a summary of all hardware targets created,
including the Database Machine target representing the Recovery Appliance hardware
and all member targets for all hardware components.
6. On the Target Creation Summary page, click Continue With Recovery Appliance
Discovery. Enterprise Manager displays the Recovery Appliance Discovery: Properties
page.
7. On the Recovery Appliance Discovery: Properties page, select the Target Name for the
a ble
Recovery Appliance. The Recovery Appliance Hardware target name is already filled-in
fe r
ans
with the target name discovered above.(If the Recovery Appliance hardware discovery
- t r
was completed without completing the full Recovery Appliance discovery process, that
n
no
may leave one or more Recovery Appliance hardware targets in a state where they are
a
h a s
not associated with a Recovery Appliance target. In this case, when Recovery Appliance
) ideฺ
discovery is re-initiated from the Add Targets Manually page, the discovery process will
m
g ฺ co t Gu
not automatically go in to the Recovery Appliance hardware discovery wizard. Instead, the
a n kn den
process will go directly to the Properties page in order to allow selection of an
i o nb s Stu
unassociated Recovery Appliance hardware target. In this case, the hardware target
name will not be pre-filled. Click the Select Target icon to launch the Search and Select
u n t h i
Targets popup, which shows a list of Recovery Appliance hardware targets that have not
l lo@ o us e
e
yet been associated with a Recovery Appliance target. Select the correct Recovery
kb se t
a
Appliance hardware target.)
a
8. l
In the l o
Recovery
l i en
( cAppliance Administrator Credentials section, specify the database user
m be
credentials for the Recovery Appliance recovery catalog owner. This user is also the
a kee Recovery Appliance administrator. You can use Named or New credentials. Click More
Details to see detailed information about the credentials.
9. In the Recovery Appliance Monitoring Credentials section, specify the database user
credentials that will be used to monitor the Recovery Appliance. If necessary, that user
will be granted the role required to access monitoring information in the Recovery
Appliance database. Optionally, you can select the option to Use base catalog user
credentials as monitoring credentials which disables the username and password fields.
10. Click Next to move to the Media Manager Domain discovery page.
11. If Media Manager is installed on the Recovery Appliance, specify the installation home
directory and monitoring credentials for the Media Manager domain. Otherwise, check the
option to skip Media Manager discovery.
12. Click Next to move to the Review page.
13. On the Review page, you can review the target discovery selections. The page lists the
Target Name, Target Type, Recovery Appliance Hardware, Database System, and
Cluster Database.
14. Click Submit to complete the discovery process. The Recovery Appliance target is
created.

Agenda

Appliance
r a ble
a - tr
no n
h a sa
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li
m be
e
ake

Recovery Appliance Administration Roles
Separation of Duties
• Cloud Control Administrator duties:
– Discover Recovery Appliance and

Targets
– Manage Protected Databases
– Manage Recovery Appliances
• Recovery Appliance Administrator duties:
– Create user accounts
– Work with Protection Policies
– Manage storage space a ble
fe r
– Configure Recovery Appliance
an s
Replication
n - t r
– Configure Copy-to-Tape Integration
a no
– Monitor Recovery Appliance and
h a s
generate reports
m ) ideฺ
• Protected Database Administrator duties:
g ฺ co t Gu
a n kn den
– Configure backup and recovery options
i o nb s Stu
– Perform backup and recovery operations
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
A typical l l o li
eRecovery Appliance environment includes personnel with the following roles:
b
•m Cloud Control administrator: The application administrator with this role administers
a kee Oracle Enterprise Manager Cloud Control (Cloud Control).
- Discovering targets, including the Recovery Appliance
- Managing one or more protected databases
- Managing one or more Recovery Appliances
• Recovery Appliance administrator: This administrator manages the Recovery
Appliance. Typical duties include:
- Creating the protection policies
- Assigning protected databases to protection policies
- Managing space on Recovery Appliance
- Configuring tape and replication operations
- Creating the Recovery Appliance user accounts that own virtual private catalogs
- Monitoring Recovery Appliance, and generating reports
• Protected Database administrator: This administrator is responsible for configuring
backups to the Recovery Appliance using the virtual private catalog account assigned
by the Recovery Appliance administrator.

Cloud Control User Accounts
User accounts:
• Cloud Control super-user: SYSMAN
– Avoid using this account.

• Cloud Control administrator
– Multiple accounts may exist.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
ZDLRA_ADMIN:
g ฺ co t Gu
sample Cloud
Control administrator
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l o
• be Controllisuper-user (SYSMAN): This application account exists by default. Its
l
Cloud
m purpose is to administer Cloud Control itself, and is not directly related to managing a
e
ake Recovery Appliance or protected databases.
• Cloud Control administrator: This is a Cloud Control user account that has been
granted the roles and privileges needed to discover Recovery Appliances and Targets
and to manage a specific protected database or a specific Recovery Appliance. Multiple
Cloud Control administrative accounts may exist, depending on your business
requirements.

Managing a Recovery Appliance Target
Creating a Cloud Control administrator

• General Information
– Username
– Password
– Password Profile
– Email
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li
e administrator
m b
Cloud Control (Recovery Appliance Target)
e following steps describe how to create an Enterprise Manager administrator (for example,
ake
The
ZDLRA_DB_ADMIN) that will have the necessary roles, target privileges, resource privileges,
and credentials to discover and manage one or more Recovery Appliances for use with Cloud
Control:
1. Log in to Enterprise Manager as an Enterprise Manager administrator (for example,
SYSMAN) that has the privileges to create other Enterprise Manager administrators.
2. From the Setup menu, choose Security then select Administrators. Click Create, which
displays the Create Administrator wizard.
3. On the Properties page, supply the name (for example, ZDLRA_EM_ADMIN),
password, password profile for security best practices and email for alert notification.
In this example we use the MGMT_ADMIN_USER_PROFILE password profile. Create
or choose a password profile that best adapts to your business requirements.

Managing a Recovery Appliance Target

• Roles
– Ensure EM_USER role is selected

• Target Privileges
– Discovery and manage target privileges
• Resource Privileges
– Job System, Named Credentials
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
b
Cloud Control li
el administrator (Recovery Appliance Target)
e m
ake
Steps
4. On the Roles page, ensure EM_USER is in the Selected Roles.
5. On the Target Privileges page, complete these steps:
a) In the Privileges applicable to all targets section, if the administrator needs the
ability to discover a Recovery Appliance, enable the Add Any Target privilege.
b) In the Target Privileges section, add the following privileges for the specified
targets, depending on whether the administrator needs the ability to discover
and/or manage the Recovery Appliance.
Discover:
- View privilege for the Agent targets that are monitoring the Recovery
Appliance compute nodes.

Discover and/or manage:
- View privilege for the Recovery Appliance cluster database target.
- View privilege for the database system target corresponding to the Recovery
Appliance cluster database.
- View privilege for the Oracle Exadata Database Machine target.
Manage:
- Full privilege for the Recovery Appliance target. (This privilege will be
automatically configured for a user that discovers the Recovery Appliance
target.)
- View privilege for the targets corresponding to each client database that will be
protected by the Recovery Appliance.
6. On the Resource Privileges page, complete these steps:
a) For the Job System resource type, click the Manage Privilege Grants icon. In the
a ble
Resource Type Privileges table, check Create and continue.
fe r
ans
b) The administrator will either need access to existing named credentials or the ability
n - t r
to create named credentials. The credentials involved depend on whether the
a no
administrator needs the ability to discover and/or manage the Recovery Appliance:
h a s
- To give the administrator the ability to create named credentials, click the
m ) ideฺ
g ฺ co t Gu
Manage Privilege Grants icon on the Resource Privileges page under the
Named Credential resource type. In the Resource Type Privileges table, check
n kn den
Create new Named Credential and continue.
a
i o nb s Stu
- To grant the administrator access to existing credentials, from the Resource
u n t h i
Privileges page, for the Named Credential resource type, click the Manage
l lo@ o us e
Privilege Grants icon. In the Resource Privileges table, click Add, then search
e
kb se t
and select for the required named credentials and continue.
a a
(discover n manage a Recovery Appliance target, the administrator will need
o
c) llTo l i c eand/or
m be to have access or be able to create the following credentials:
a kee - Credentials for the Recovery Appliance Administrator (for example, RASYS).
The named credentials (as viewed by choosing Security from the Setup menu
and then selecting Named Credentials) must be of type Recovery Appliance
Administrator Credentials.
- Credentials for the host of the Recovery Appliance. The named credentials
must be of type Host Credentials.
- If different than the Recovery Appliance Administrator, credentials for the
Recovery Appliance Monitoring user (for example, DBSNMP). The named
credentials must be of type Recovery Appliance Administrator Credentials.
- Credentials for the Media Manager Domain administrator (for example,
ADMIN). The named credentials must be of type Media Manager Domain
Credentials.

d) To manage a Recovery Appliance target, the administrator must have access or be
able to create the following credentials:
- Credentials for the Recovery Appliance user(s) (for example, RMAN) that will
own the virtual private catalog(s) that will be used by the protected databases.
The named credentials must be of type Database Credentials.
7. Click Review.
8. Click Finish.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

Managing Protected Databases with Recovery Appliance

• General Information
– Username
– Password
– Password Profile
– Email
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li
e administrator
m b
Cloud Control (Protected Databases Target)
e following steps describe how to create an Enterprise Manager administrator (for example,
ake
The
ZDLRA_DB_ADMIN) that will have the necessary roles, target privileges, resource privileges,
and credentials to manage one or more protected databases for use with Recovery
Appliance:
1. Log into Enterprise Manager as an Enterprise Manager administrator (for example,
SYSMAN) that has the privileges to create other Enterprise Manager administrators.
2. From the Setup menu, choose Security then select Administrators. Click Create, which
displays the Create Administrator wizard.
3. On the Properties page, supply the name (for example, ZDLRA_DB_ADMIN),
password, password profile for security best practices and email for alert notification.
In this example we use the MGMT_ADMIN_USER_PROFILE password profile. Create
or choose a password profile that best adapts to your business requirements.

Managing Protected Databases with Recovery Appliance

• Roles
– Ensure that the EM_USER role is selected.

• Target Privileges
– View the Recovery Appliance and full DB/host target privileges.
• Resource Privileges
– Job System, Named Credentials
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
Cloudb Control li
el administrator (Protected Databases Target)
k e e4.m On the Roles page, ensure EM_USER is in the Selected Roles.

a 5. On the Target Privileges page in the Target Privileges section, add the following
privileges for the specified targets:
a) View privilege for the Recovery Appliance target.
b) Full privilege for the targets corresponding to each database that will be managed
by the administrator and protected by the Recovery Appliance.
c) Full privilege for the targets corresponding to the hosts of each database that will
be managed by the administrator and protected by the Recovery Appliance.
6. On the Resource Privileges page, for the Job System resource type, click the Manage
Privilege Grants icon. In the Resource Type Privileges table, check Create and
continue. The administrator will either need access to existing named credentials or the
ability to create named credentials.

7. To give the administrator the ability to create named credentials, from the Resource
Privileges page, for the Named Credential resource type, click the Manage Privilege
Grants icon. In the Resource Type Privileges table, check Create New Named Credential
and continue.
8. To grant the administrator access to existing credentials, from the Resource Privileges
page, for the Named Credential resource type, click the Manage Privilege Grants icon. In
the Resource Privileges table, click Add, then search for and select the required named
credentials and continue. Access to the following credentials are required:

- Database credentials for the targets corresponding to each database that will be
managed by the administrator and protected by the Recovery Appliance. The
named credentials must be of type Database Credentials.
- Host credentials for the targets corresponding to the hosts of each database that will
be managed by the administrator and protected by the Recovery Appliance. The
named credentials must be of type Host Credentials.
r a ble
- Credentials for the Recovery Appliance user(s) (for example, RMAN) that fwill
s e own
tra n
the virtual private catalog(s) that will be used by the protected databases. The
named credentials must be of type Database Credentials. n -
9. Click Review. a no
h a s
10. Click Finish.
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o li
m b
a kee

Agenda

Appliance
r a ble
a - tr
no n
h a sa
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li
m be
e
ake

Protection Policies
Defined Classes of Service
Tasks
1. Plan for needed policies by grouping protected databases
based on recovery requirements.

2. Configure a protection policy for each group.
3. Maintain protection policies by modifying them as needed.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Appliance
The Recovery li provides a Data Protection as a Service infrastructure allowing you
b
m an effective data protection private cloud environment including disk backup
ak etoecreate
(Recovery Appliance), copy-to-tape and/or replication.
Basic tasks involved in managing protection policies:
• During the planning phase, group the databases into tiers, and decide the recovery
requirements for each tier.
• During the configuration phase, create one protection policy for each database tier.
• During the ongoing maintenance phase, modify protection policies as needed.

Protection Policies
Defined Classes of Service
Predefined Protection Policies

Service Tier Recovery Window Additional Settings

Database backups, real-time redo
45 days on disk, 90 days on
Platinum tape
transport, replication, and tape
backups. All settings are mandatory.
Gold
35 days on disk, 90 days on Database backups, real-time redo
transport, replication, and tape
a ble
tape backups (if tape is available).
fe r
Database backups, real-time redo
an s
Silver
35 days on disk, 90 days on
tape available). n -
transport, and tape backups (if tape is
t r
a no
Bronze
h a s
35 days on disk, 90 days on Database backups and tape backups
(if tape is available). There is no real-
tape
m ) ideฺ time redo transport.
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Appliance
The Recovery li provides a Data Protection as a Service infrastructure, allowing you
b
m an effective data protection private cloud environment, including disk backup
ak etoecreate
(Recovery Appliance), copy-to-tape, and/or replication.
Protection policies define a class of service of recovery requirements, allowing you to
standardize data protection by database tier. For example, you may want to create a “Gold”
protection policy for mission-critical databases, “Silver” for business-critical and “Bronze” for
Test/Dev databases.
Protection policies are the foundation for Data Protection As a Service. Each database under
Recovery Appliance management must be associated with a protection policy and can then
be managed as a group for defining replication and/or copy to tape schedules.
A protection policy is a named collection of properties that you can assign to multiple
protected databases. Using a single policy for multiple databases reduces Recovery
Appliance administration time, and enables you to change the properties of multiple protected
databases with one operation. To accommodate databases with differing backup and
recovery requirements, create as many protection policies as required.

For every database associated with it, a protection policy specifies:
• The recovery window goal for disk backups
• The recovery window for tape backups
• Whether the Recovery Appliance must replicate backups or copy them to tape before
deleting them
• Which Recovery Appliance storage location is used for backups
• An optional backup polling policy
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Protection Policies
Creating Protection Policies

• You must be logged in as a Cloud Control Recovery
Appliance Administrator with RASYS Named Credentials.

• The protection policy must not be associated with any
protected database.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
ea protection
To create li policy using Oracle Enterprise Manager Cloud Control:
b
ak ee1.m Access the Protection Policies page.
2. Click Create. The Create Protection Policy page appears.
3. Enter values as follows:
- In the Name field, enter the name of the new protection policy. For example, enter
bronze_dev.
- In the Description field, enter a description for the new policy. For example enter:
Policy with disk recovery window of 3 days and no tape
backup.
- In the Recovery Window field of the Disk Recovery Window Goal section, specify
a recovery window goal that the Recovery Appliance should attempt to meet for
point-in-time recovery using disk backups, and then select the units. For example,
enter 3 and then select days.
- In the Threshold field of the Unprotected Data Window Threshold section, enter
the maximum tolerable interval for data loss. For example, enter 5 and then select
minutes.

• In the Recovery Window field of the Media Manager Recovery Window Policy section,
optionally specify a larger window within which point-in-time recovery from a media
manager will be maintained. For example, if no tape backup is desired, then leave this
field blank.
• In the Maximum Retention field of the Maximum Disk Backup Retention section, enter
the maximum time that the Recovery Appliance must retain disk backups. For example,
leave this field blank, which means that the Recovery Appliance does not purge
backups unless you explicitly purge them or space pressures exist within a storage
location.
• Optionally, in the Backup Polling Location section, define a backup polling policy. For
example, to specify that backup polling is disabled, leave the fields blank.
- In the Location field, specify a directory accessible by the Recovery Appliance.
- In the Frequency field, specify a time interval, and then select the time units.
- To specify that the Recovery Appliance must delete the backups from the polling
a ble
location after copying them, select Delete Backups After Copy. fe r
ans
•
n - t r
In the Backup Copy Policy section, specify whether the Recovery Appliance must
a no
replicate backups or copy backups to tape before deleting them. For example, select
Always accept new backups even if it requires purging existing backups not yet copied
to tape or replicated. h a s
m ) ideฺ
4. Click OK.
g ฺ co t Gu
The Protection Policies page appears, with
a n knthe newly
d e n created protection policy listed.
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li
m be
e
ake

Protection Policies
SQL Developer
Creating Protection Policies
• You must log in to the metadata database as RASYS.
• The protection policy must not be associated with any protected

database.
• Creating a Backup Polling Policy is optional.
Create a
PROCEDURE create_protection_policy ( Protection Policy.
protection_policy_name IN VARCHAR2,
description IN VARCHAR2 DEFAULT NULL,
storage_location_name IN VARCHAR2,
a ble
polling_policy_name IN VARCHAR2 DEFAULT NULL,
recovery_window_goal IN DSINTERVAL_UNCONSTRAINED,
fe r
max_retention_window IN DSINTERVAL_UNCONSTRAINED DEFAULT NULL,
an s
recovery_window_sbt IN DSINTERVAL_UNCONSTRAINED DEFAULT NULL,
unprotected_window IN DSINTERVAL_UNCONSTRAINED DEFAULT NULL,
n - t r
guaranteed_copy IN VARCHAR2 DEFAULT 'NO',
allow_backup_deletion IN VARCHAR2 DEFAULT 'YES');
a no
h a s
PROCEDURE create_polling_policy(
m ) ideฺ
polling_policy_name IN VARCHAR2,
polling_location IN VARCHAR2,
g ฺ co t Gu
delete_input IN BOOLEAN DEFAULT FALSE);
a kn den
polling_frequency IN DSINTERVAL_UNCONSTRAINED DEFAULT NULL,
n Query a
b t u Protection Policy
n i on is S
@ u e th
( a ak nse
l loprotection
l i e
cpolicy,
b e
To create a execute the DBMS_RA.CREATE_PROTECTION_POLICY
em
eprocedure:
ak 1. Start SQL*Plus or SQL Developer, and then log in to the metadata database as RASYS.
2. (Optional) Run the DBMS_RA.CREATE_POLLING_POLICY procedure.
Parameters:
• polling_policy_name: The user-assigned name of the polling policy.
• polling_location: The directory that the Recovery Appliance periodically examines
for new backups. Do not specify the same directory name in multiple polling policies.
• polling_frequency: The frequency with which the Recovery Appliance examines
the specified directory for new backups. System load may cause backup polling to occur
less frequently. Specify the window as any valid INTERVAL DAY TO SECOND
expression, such as INTERVAL '2' DAY (2 days), INTERVAL '4' HOUR (4 hours),
and so on.

• delete_input: The setting that controls deletion behavior. If TRUE, then the Recovery
Appliance deletes files in the specified directory after copying them to a storage location. If
FALSE, then the Recovery Appliance does not delete files that it discovers in the polling
location.
3. Run the DBMS_RA.CREATE_PROTECTION_POLICY procedure.
Parameters:
• protection_policy_name: The user-assigned name of the protection policy.

• description: An optional description of the usage for the policy.
• storage_location_name: The name of the storage location. The Recovery Appliance
uses this location for actively received incoming backups, and for newly created backup
files for all databases sharing this protection policy.
• polling_policy_name: The name of the backup polling policy. The polling policy
specifies the rules for how the Recovery Appliance polls for backups of protected
databases that use this protection policy. If null, then no backup polling occurs for a ble
databases that use this protection policy. fe r
ans
• recovery_window_goal: The recovery window goal for databases that use this
n - t r
o
an
protection policy. For each protected database, the Recovery Appliance attempts to
s
ensure that the oldest backup on disk can support a point-in-time recovery to any time
ha eฺ
)
within the specified interval, counting backward from the current time. Specify the goal as
ฺ c om Guid
any valid INTERVAL DAY TO SECOND expression, such as INTERVAL '2' DAY (2
n k ng ent
days), INTERVAL '4' HOUR (4 hours), and so on.
b a ud
• max_retention_window: The maximum length of time that the Recovery Appliance
n S t
nio this
must retain backups for databases that use this protection policy. Recovery Appliance
u
l l o @ use
only holds backups longer than the specified period when they are required to preserve
a k be e to
the recovery window goal for a database. If null, then the Recovery Appliance does not
purge backups unless caused by explicit user actions or space pressures within a storage
o
location.
l (a cens
bel li
• recovery_window_sbt: The recovery window for SBT backups of databases that use
e m
ake
this protection policy. For each protected database, the Recovery Appliance keeps
backups long enough on tape to guarantee that a recovery is possible to any time within
the specified interval, counting backward from the current time. If this parameter is not
null, then you must also create an SBT job for this protection policy, and then schedule it
using a scheduling facility such as Oracle Scheduler. See CREATE_SBT_JOB_TEMPLATE.
Specify the window as any valid INTERVAL DAY TO SECOND expression, such as
INTERVAL '2' DAY (2 days), INTERVAL '4' HOUR (4 hours), and so on.

• unprotected_window: The maximum amount of data loss that is tolerable for
databases using this protection policy. When a protected database exceeds the specified
amount of data loss, the Recovery Appliance posts a warning to RA_INCIDENT_LOG. The
most recent time to which each protected database is recoverable is shown in the
HIGH_TIME column of RA_RESTORE_RANGE. Specify the window as any valid INTERVAL
DAY TO SECOND expression, such as INTERVAL '2' DAY (2 days), INTERVAL '4'
HOUR (4 hours), and so on.
• guaranteed_copy: The setting of the guaranteed copy feature. Specifying NO means

that the Recovery Appliance always accepts new backups, even if it must delete old
backups when space is low. This option prioritizes the ability to successfully process the
backup currently being received over the ability to restore older backups. Specifying YES
ensures that the Recovery Appliance replicates backup data or copies it to tape before
removing it from Recovery Appliance storage. This option prioritizes the ability to restore
older backups over the ability to successfully process the backup currently being received.
If set to YES, then for each protected database the Recovery Appliance can only hold up
a ble
to disk_reserve_space bytes of backup data that is not yet copied or replicated. Iffe r
ans
hardware or network errors prevent timely copying or replication, then future attempts to
create new backups will fail when the Recovery Appliance reaches the n - t r
disk_reserve_space limit. a no
h a s
allow_backup_deletion: The setting that controls whether RMAN backups for
•
m ) ideฺ
databases that use this protection policy can be deleted. Specifying NO means that the
g ฺ co t Gu
Recovery Appliance does not allow deletion of these backups. The default is YES.
4. a
Optionally, query the recovery catalog n kn dcreation
to confirm e n of the policy.
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li
m be
e
ake

Protection Policies
Modifying Protection Policies

• You must be logged in as Cloud Control Recovery Appliance
Administrator with RASYS Named Credentials.

• The protection policy must exist.
• The modification takes effect immediately.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li policy using Oracle Enterprise Manager Cloud Control:
e a protection
To update
b
2. In the Protection Policies table, select the protection policy that you want to edit. For
example, select the BRONZE_DEV row.
3. Click Edit. The Edit Protection Policy page appears.
4. Change the desired values, and then click OK. For example, in the Recovery Window
field of the Disk Recovery Window Goal section, enter 6.
The Protection Policies page appears, with the newly updated protection policy listed.

Protection Policies
SQL Developer
Modifying Protection Policies
• The protection policy must exist.

• Creating a new or modifying an existing Backup Polling Policy is optional.
• It takes effect immediately.
PROCEDURE update_protection_policy (
protection_policy_name IN VARCHAR2,
description IN VARCHAR2 DEFAULT NULL,
storage_location_name IN VARCHAR2 DEFAULT NULL,
polling_policy_name IN VARCHAR2 DEFAULT dbms_ra_int.varchar2null('p1'),
recovery_window_goal IN DSINTERVAL_UNCONSTRAINED DEFAULT NULL,
a ble
max_retention_window IN DSINTERVAL_UNCONSTRAINED DEFAULT dbms_ra_int.intervalnull('p3'),
fe r
recovery_window_sbt IN DSINTERVAL_UNCONSTRAINED DEFAULT dbms_ra_int.intervalnull('p2'),
an s
unprotected_window IN DSINTERVAL_UNCONSTRAINED DEFAULT dbms_ra_int.intervalnull('p4'),
guaranteed_copy IN VARCHAR2 DEFAULT NULL,
n - t r
no Update a
allow_backup_deletion IN VARCHAR2 DEFAULT NULL);
s a Protection Policy.
PROCEDURE update_polling_policy (
polling_policy_name IN VARCHAR2, h a
) ideฺ
polling_location IN VARCHAR2 DEFAULT NULL,
m
co t Gu
ฺ
polling_frequency IN DSINTERVAL_UNCONSTRAINED DEFAULT NULL,
g
kn den
delete_input IN BOOLEAN DEFAULT NULL);
a n
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li policy, execute the DBMS_RA.UPDATE_PROTECTION_POLICY
e a protection
To update
b
em
eprocedure. Parameters that are null retain their existing values. For example, if
ak guaranteed_copy is currently NO for a protection policy, and if you specify null for this
parameter in DBMS_RA.UPDATE_PROTECTION_POLICY, then the value remains NO.
1. Start SQL*Plus or SQL Developer, and then log in to the metadata database as RASYS.
2. Run the DBMS_RA.UPDATE_PROTECTION_POLICY procedure.
3. Optionally, query the recovery catalog to confirm the update of the policy.

Protection Policies
Removing Protection Policies

• You must be logged in as Cloud Control RA Administrator with
RASYS Named Credentials.

protected database.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
ea protection
To delete li policy using Oracle Enterprise Manager Cloud Control:
b
2. In the Protection Policies table, select the protection policy that you want to delete. For
example, select the BRONZE_DEV row.
3. Click Delete. A confirmation window appears.
4. Click Yes.
The Protection Policies page appears, with the deleted protection policy no longer listed.
(Optional) Delete the Backup Polling Policy if it is also no longer needed.

Protection Policies
SQL Developer
Removing Protection Policies

protected database.
Delete a
Protection Policy.
PROCEDURE delete_protection_policy (
protection_policy_name IN VARCHAR2);
PROCEDURE delete_polling_policy (
a ble
polling_policy_name IN VARCHAR2);
fe r
an s
n - t r
a no
h a s
m ) ideฺ
ฺ co t Gu
Query Protection
g
n not exist.
Policykdoes
n e n
a d
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
ea protection
To delete li policy, execute the DBMS_RA.DELETE_PROTECTION_POLICY
b
em
eprocedure:
ak 1. Start SQL*Plus or SQL Developer, and then log in to the metadata database as RASYS.
2. Confirm that the protection policy that you intend to delete is not currently associated
with any protected databases.
3. Delete the policy.
4. Optionally, confirm the deletion.
(Optional) Delete the Backup Polling Policy if it is also no longer needed.

Remote Access
Creating Virtual Private Catalog Accounts
SQL Developer
Enabling remote access to the Recovery Appliance for Protected
Databases
• You must log in to the metadata database as SYSTEM.

• This VPC account will be used to enroll Protected Databases
(described in the next lesson).
• Do not use the RASYS account.
a ble
fe r
an s
VPC user n - t r
creation
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l o
e connectlito the Recovery Appliance catalog when backing up to a Recovery
RMANbmustl
k e em
Appliance. In this step, you create a virtual private catalog user for a specific protected
a database or set of protected databases.
1. With SQL*Plus or SQL Developer, connect to the Recovery Appliance database as
SYSTEM.
2. Create a database user account with CREATE SESSION privileges to own the virtual
private catalog.

Agenda

Appliance
r a ble
a - tr
no n
h a sa
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li
m be
e
ake

Recovery Appliance Metadata
Database System
Metadata Catalog
• RAC Database (similar to Exadata)
• SCAN Listeners for enabling remote access to Protected Databases

• Clusterware services for protection, HA, and management
• XMLDB HTTP Servlet for backup/recover Protected Databases
• Data Guard-like RFS for Real-Time Redo Transport
• Database Scheduler for Internal Tasks
• ASM Storage (two disk groups)
– CATALOG: database files, RMAN catalog, and Recovery Appliance metadata
a ble
– DELTA: Container files, Virtual Full Backups, Archived Log files, miscellaneous fe r
files an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
The key
l o
ecomponentlofi the Recovery Appliance is the Recovery Appliance metadata
l
b
em This database manages metadata for all backups, and also contains the RMAN
edatabase.
ak recovery catalog. The Recovery Appliance metadata database is preconfigured, pre-tuned,
and managed by the Recovery Appliance.
• RFS = Remote File Server
• SCAN = Simple Client Access Network
• XMLDB = XML Database

Clusterware and RAC Database
• Clusterware
• Two Nodes RAC Database
– Benefits
— High Availability (node failure)
— Scalability (1000s Protected Databases)
— Load Balance (distribute load between RAC instances)
— IB network for cluster interconnection and ASM storage traffic
Topology
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
OraclebClusterware i Release 1 is the integrated foundation for Oracle Real Application
l12c
k e em (RAC) and the High Availability (HA) and resource management framework for all
Clusters
a applications on any major platform.
Additional information about Oracle Clusterware benefits:
http://www.oracle.com/technetwork/database/database-
technologies/clusterware/overview/index-090666.html

Sample of Clusterware resources:
[oracle@host1 ~]$
[oracle@host1 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ob_dbfs
ONLINE ONLINE host1 STABLE
ora.CATALOG.dg
le
ora.DELTA.dg
r a b
s fe
- t r an
o n
an
ora.LISTENER.lsnr
STABLEha
s
ONLINE ONLINE host2
m ) ideฺ
ora.LISTENER_REPL.lsnr
g ฺ coSTABLE
t G u
ONLINE ONLINE host1
host2 an
kn deSTABLE
n
nb s Stu
ONLINE ONLINE
ora.asm i o
n host1thi
ONLINE ONLINE u
l o @
l to u s e Started,STABLE
ora.net1.network k
b e
ONLINE ONLINE host2 Started,STABLE
( a a nse
e l lo ONLINE i
ONLINE
l ceONLINE host1 STABLE
m b ONLINE host2 STABLE
kee
ora.net2.network
a ONLINE ONLINE host1 STABLE
ora.ons
osbmedia
rep_dbfs

--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_REPL_SCAN1_NET2.lsnr
1 ONLINE ONLINE host2 STABLE
ora.LISTENER_SCAN1.lsnr
a ble
fe r
ora.MGMTLSNR
ans
1 ONLINE ONLINE host1
t
111.111.111.111 … ,STABLE
n - r
ora.cvu
o
s an
ora.mgmtdb
) ha eฺ
1 ONLINE ONLINE host1
c o m uid
Open,STABLE
ora.oc4j ฺ
ng eSTABLE tG
host1 nk
n
1 ONLINE ONLINE
n b a
S t ud
ora.scan1.vip
1
io
ONLINE ONLINE un host1thi
s STABLE
ora.scan1_net2.vip @ e
llo to ushost2
b
k se e
1
ora.scan2.vip( a a
ONLINE ONLINE
n
STABLE
1e l l o l i c e
m b ONLINE ONLINE host1 STABLE
kee 1
ora.scan2_net2.vip
a ora.scan3.vip

ora.scan3_net2.vip
ora.host1.vip
ora.host1_2.vip
ora.host2.vip
ora.host2_2.vip

ora.zdlra1.db
1 ONLINE ONLINE host1 Open,STABLE
2 ONLINE ONLINE host2 Open,STABLE
osbadmin
slcm03osbadm-avip1
slcm03osbadm-vip1
--------------------------------------------------------------------------------
[oracle@host1 ~]$
Oracle Real Application Clusters (RAC) is an option to the award-winning Oracle Database
Enterprise Edition. Oracle RAC is a cluster database with a shared cache architecture that
overcomes the limitations of traditional shared-nothing and shared-disk approaches to provide a ble
fe r
highly scalable and available database solutions for all your business applications.
ans
Additional information about Oracle RAC Database benefits: n - t r
o
an
http://www.oracle.com/technetwork/database/options/clustering/overview/index.html
s
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Architecture ZDLRA RAC DB
Node Control CLI

(dbmcli/dcli) DB Instance - ZDLRA1 DB Instance - ZDLRA2 Enterprise
Manager
SSH DB Instance DB Instance

DBRM DSKM DBRM DSKM
ASM DSKM ASM DSKM

Single
LIBCELL MS RS LIBCELL MS RS
ASM cluster
iDB Protocol
over InfiniBand InfiniBand Storage Network
a b le
fe r
a n s
Oracle Linux Oracle Linux Oracle Linux -t
n r
Cell
n o
Control CLI CELLSRV MS CELLSRV MS a MS
CELLSRV
s
(cellcli/dcli)
IORM IORM ) a
h IORM ฺ RS
SSH
RS RS
m i d e
g ฺ co t Gu
a n kn den
Exadata Cell
o nbExadata S tu
Cell Exadata Cell
u n i h i s
e t
a a kb se t
l o ( cen
l
e Appliance
The Recovery
b li software architecture includes components on the database servers
emon the Exadata cells. The overall architecture is illustrated in the slide. The following
eand
ak components reside on each database server:
• ZDLRA database servers use Oracle Linux x86_64 as the operating system.
• ZDLRA database servers can run Oracle Database 12c Release 1 with a pre-installed
and pre-configured RASYS schema that contains the Recovery Appliance Metadata.
These servers also include version 12.1 of Oracle Grid Infrastructure to provide
clusterware infrastructure to the resources.
• Automatic Storage Management (ASM) is required and provides a DELTA PUSH,
DELTA STORE and CATALOG for Oracle Recovery Appliance Database.

Architecture
ZDLRA RAC DB
Node Control DB Instance - ZDLRA1 DB Instance - ZDLRA2 Enterprise

CLI SSH
Manager
(dbmcli/dcli) DB Instance DB Instance

DBRM DSKM DBRM DSKM
ASM DSKM ASM DSKM

Single
ASM cluster
LIBCELL MS RS LIBCELL MS RS
iDB Protocol over

InfiniBand InfiniBand Storage Network
a b le
fe r
an s
Oracle Linux Oracle Linux Oracle Linux
n - t r
Cell Control
CLI
CELLSRV MS CELLSRV
a no MS
MS CELLSRV
(cellcli/dcli)
h a s IORM
IORM
SSH
RS IORM RS
m ) ideฺ RS
g ฺ co t Gu
a n knCell den
Exadata Cell
n
Exadata
b S t u Exadata Cell
u nio this
l l o @ use
be e to
a k
l o (a cens
The software
b li that reside in each Exadata cell include the following:
el components
ak ee•m Oracle Linux x86_64 provides the Exadata Storage Server operating system.
• Cell Server (CELLSRV) is the primary Exadata Storage Server software component and
provides the majority of Exadata storage services. CELLSRV is a multithreaded server.
It serves simple block requests, such as database buffer cache reads, and Smart Scan
requests, such as table scans with projections and filters. CELLSRV also implements
IORM, which works in conjunction with DBRM to meter out I/O bandwidth to the various
databases and consumer groups that are issuing I/Os. Finally, it collects numerous
statistics relating to its operations. Oracle Database and ASM processes use
LIBCELL to communicate with CELLSRV, and LIBCELL converts I/O requests into
messages that are sent to CELLSRV by using the iDB protocol.

• The Management Server (MS) provides Exadata cell management and configuration
functions. It works in cooperation with the Exadata cell command-line interface (CellCLI).
Each cell is individually managed with CellCLI. CellCLI can be used only from within a cell
to manage that cell. However, you can run the same CellCLI command remotely on
multiple cells with the dcli utility. In addition, MS is responsible for sending alerts, and
collects some statistics in addition to those collected by CELLSRV.
• The Restart Server (RS) is used to start up or shut down the CELLSRV and MS services,
and monitors these services to automatically restart them if required.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

Compute Nodes - DBMCLI
• Each Database Compute Server is administered individually.

• Most administration is performed using DBMCLI.
– DBMCLI can be executed only on the database node being

administered.
– DBMCLI works in conjunction with the Management Server to
perform administration tasks.
Example
a ble
fe r
[root@host1 ~]# dbmcli
DBMCLI: Release - Production on Thu Jul 28 03:36:22 MDT 2016
an s
Copyright (c) 2007, 2014, Oracle. All rights reserved.
n - t r
DBMCLI> list dbserver
a no
host1 online
h a s
DBMCLI>
m ) ideฺ
co t Gu
CellCLI> exit
quitting
g ฺ
[celladmin@host1 ~]$
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l o
e utility isli the command-line administration tool for configuring database servers,
l
The DBMCLI
b
emmanaging objects in the server environment. DBMCLI runs on each server to enable you
eand
ak to configure an individual database server. You use DBMCLI to start and stop the server, to
manage server configuration information, and to enable or disable servers. The command-line
utility is already installed when Oracle Recovery Appliance is shipped.
Management Server on database servers implements a web service for database server
management commands, and runs background monitoring threads. The management service
provides the following:
• Comprehensive hardware and software monitoring including monitoring of hard disks,
CPUs, and InfiniBand ports.
• Enhanced alerting capabilities.
• Important system metric collection and monitoring.
• A command-line interface called DBMCLI to configure, monitor, and manage the
database servers. DBMCLI is pre-installed on each database server. DBMCLI
configures Auto Service Request, capacity-on-demand, Infrastructure as a Service, and
database server email alerts.
DBMCLI is a part of the Management Server that runs on database servers.

SCAN Listeners
SCAN = Simple Client Access Network

• Benefits
– Simple to connect Protected Databases: single name - Cluster SCAN

Name
– Load Balancer and Failover functionality
– Better scalability
– DNS integration
DB Listeners (back-end)
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
SCAN Listeners (front-end) nkn e n
a d
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
When b ae l l o li
Protected Database submits a request, the SCAN listener listening on a SCAN IP
k e em and the SCAN port is contacted on a client's behalf. Because all services on the
address
a cluster are registered with the SCAN listener, the SCAN listener replies with the address of
the local listener on the least-loaded node (Each scan listener keeps updated cluster load
statistics) where the service is currently being offered. Finally, the client establishes
connection to the service through the listener on the node where service is offered. All of
these actions take place transparently to the client without any explicit configuration required
in the client.
For additional details: Grid Infrastructure Single Client Access Name (SCAN) Explained (Doc
ID 887522.1)

ASM Disk Groups
ASM = Automatic Storage Manager
• Benefits
– Simplifies and automates storage management

– Increases storage utilization and agility
– Delivers predictable performance, availability, and scalability
• Disk Groups
– CATALOG: Database files, Recovery Appliance metadata, and RMAN
Recovery Catalog
– DELTA: Container files, Virtual Full Backups, Archived Log files, and
miscellaneous files
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Automatic l l o li
e Storage Management (ASM), a feature of the Oracle Database, provides Oracle
b
k e em with simplified storage management that is consistent across all server and
customers
a storage platforms. With ASM, storage is managed as a small number of storage pools called
ASM Disk Groups. Database related files are assigned to ASM Disk Groups and ASM
manages the layout and data organization ensuring optimal performance and protection from
storage hardware failure. ASM Disk Groups provide a convenient and easy means for
consolidating storage and simplifying the administrative tasks previously required of DBAs
and System Administrators.
For additional details: http://www.oracle.com/technetwork/database/database-
technologies/cloud-storage/index.html

Storage Usage
RASYS Schema per Protected DB
• Views
– Benefits
— Simplifies access to
Recovery Appliance
Metadata: Recovery
Appliance Catalog and
RMAN Recovery Catalog
— Easy integration with
Reporting Tools, such as a b le

e r
OEM BI Publisher sf
Create Protection Policy.
a n
• DBMS_RA PL/SQL Package n - tr
– Benefits a no
Simplifies and automates h a s
—
Recovery Appliance m ) ideฺ

administration g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o li
m b
e
ake

Database Scheduler
Internal Tasks
• Benefits
– Simplifies and automates job execution

– Increases monitoring and manageability
– Delivers performance, availability, and scalability for job subsystem
• Task Categories
– BACKUP/RESTORE/VALIDATE
– PURGE
r a ble
– MISCELLANEOUS sfe n
n - tra
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
el
OraclebScheduler li Scheduler) enables database administrators and application
(the
k e em
developers to control when and where various tasks take place in the database environment.
a The Scheduler provides complex enterprise scheduling functionality, which you can use to:
• Schedule job execution based on time or events
• Schedule job processing in a way that models your business requirements
• Manage and monitor jobs
• Execute and manage jobs in a clustered environment

Database Scheduler
• Task Type: BACKUP/RESTORE/VALIDATE

Name Description
INDEX_BACKUP Indexes a backup piece into one or more Delta Pools (separate Delta Pool for each datafile)
BACKUP_ARCH Copies a raw archivelog received by RFS into an archivelog backup piece stored in a storage location
INCOMPLETE_ARCH Copies an incomplete archivelog read by RFS into a backup of archivelog in the storage location
PLAN_DF Creates a plan for purging or restoring a datafile from a Delta Pool
a ble
RESTORE Assists with constructing a backup piece from one or more Delta Pools
fe r
POLL Performs a scan of a polling directory
an s
BACKUP_SBT Copies a backup piece to a tape or downstream replication ZDLRA
n - t r
o
RESTORE_SBT
an
Restores a backup piece from a tape or downstream replication ZDLRA
s
RECONCILE
ha eฺ
Synchronizes the upstream/downstream catalog data, so the upstream can have access to the
)
om Guid
downstream backups
ฺ c
CROSSCHECK_DB
n k ng ent
Verifies that the catalog reflects the set of backups stored on tape and downstream replicants
VALIDATE
n b a
S t ud
Validates the backups stored for a database
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Database Scheduler
• Task Type: PURGE

Name Descriptions
PURGE_SB Removes backups that are beyond RECOVERY_WINDOW_SBT from tape storage
T
PURGE Asynchronously purges old backups from Delta Store to free space for new backups
PURGE_IM Synchronously purges to meet an urgent space shortage

M
a ble
PURGE_DF Frees space occupied by blocks that are no longer needed
fe r
an s
PURGE_DU Removes duplicate blocks from a Delta Pool that may occur when multiple Level 0 incremental backups are
n - t r
P indexed
o
OPTIMIZE Manages OPT_DF tasks
s an
OPT_DF
) ha eฺ
Reorders blocks in a Delta Pool to improve the restore I/O efficiency of the most recent virtual backup
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Database Scheduler
• Task Type: MISCELLANEOUS

Name Descriptions
DEFERRED_DELETE Removes backup files per user request
GCOPY_COMPUTE Computes the amount of data not backed up to tape for one Protected DB (used for
Guaranteed Copy)
DELETE_DB Removes a database from the ZDLRA (used for delete_db API)
DB_STATS_REFRESH Computes database metrics such as recovery_window_space or deduplication_factor

a ble
fe r
RESTORE_RANGE_RE Computes the current restore ranges for a database
an s
FRESH
n - t r
o
an
REBUILD_INDEX Rebalances indexes for BLOCKS and CHUNKS tables
CHECK_FILES Validates ZDLRA metadata for consistency s

ha eฺ
)
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
b el li
m
a kee

Database Scheduler
Task Priorities
• Schedulers try to run tasks with lowest number (highest priority) first.
• Tasks with the highest numbers are Busywork tasks.

– DB_STATS_REFRESH, RESTORE_RANGE_REFRESH, OPT_DF, OPTIMIZE,
REBUILD_INDEX, VALIDATE, CHECK_FILES, CROSSCHECK_DB
• Busywork tasks validate and improve performance of the ZDLRA rather than
process recovery data.
• Operations are not hurt if Busywork tasks don’t execute on time.
a ble
fe r
Tasks ordered by
an s
highest priority
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Database Scheduler
RA_TASK View
• Contains a number of columns that are useful. Favorites are task_id, task_type, state,
waiting_on, creation_time, last_execute_time, completion_time, interrupt_count,

error_count, and archived.
• Can be useful for answering questions such as “what tasks were active during a
problem time window?” or “why hasn’t the validation of db x completed lately?”
• Underlying tables are task and task_history that have even more columns.
What tasks have

executed in the
last day?
a ble
fe r
INDEX_BACKUP Task
a n s
n r
-t If
- This task is critical in the ZDLRA operation.
o
an
indexing tasks are not completed, strange
s
behaviors maybe observed.
hatrue ifeyouฺ schedule a copy to tape
)
- This is especially
-ฺc
om TASK_VIEW
job before
Use the RA G
id to understand whether
indexing is complete.
u
n k ng Indexing
e n tis complete.
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
b
Examples el li
k e e•m What tasks have been waiting the longest?
a select task_id, task_type, interrupt_count, systimestamp-
creation_time wait_time from ra_task where creation_time =
(select min(creation_time) from ra_task where archived='N') and
archived='N';
• What types of tasks currently need to be processed?
select task_type, count(*) from ra_task where archived=’N' group
by task_type order by task_type;
• Which tasks are being blocked by other tasks?
select t.task_id, t.task_type, b.task_id blocker, b.task_type
blocker_type from ra_task t join ra_task b on (t.waiting_on =
b.task_id) where t.archived = ’N‘ order by t.task_id;

• What tasks have executed on the last day?
select task_type, count(*) from ra_task where completion_time >
(sysdate - 1) group by task_type order by task_type;
• What tasks are thrashing?
select task_id, task_type, state, interrupt_count from ra_task
where interrupt_count > 300 and archived='N';
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

Database Scheduler
• Task States
The processing
state for the task
Name Descriptions
EXECUTABLE Task is eligible to run
RUNNING Task is currently being executed by a scheduler
COMPLETED Task has finished running. RA_TASK.ARCHIVED will be “Y”
a ble
RETRYING Session has failed, but task has not been cleaned up yet
fe r
an s
FAILED Task did not succeed, but is not eligible to be retried
n - t r
o
CANCEL
CANCELING an
Task is in the process of being run down
s
CANCELED ) ha eฺ
Some agent killed the task
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Database Scheduler
• Task Wait States
The wait state
for the task
Name Descriptions
ORDERING_WAIT An INDEX_BACKUP task is waiting for previous backups to be processed into the Delta Pool
TASK_WAIT Task has been suspended until a more important task completes or until a lock held by the
blocking task is released. The blocking task is identified by RA_TASK.WAITING_ON
SERVLET_WAIT Task is being blocked by an HTTP Servlet that is locking an object needed by the task
a ble
SPACE_WAIT Task is waiting for space to be released in the storage location
fe r
an s
OPT_DF_WAIT Task is an OPT_DF task that is waiting for space to be released in the storage location
n - t r
o
an
TEST_WAIT Task has been queued by the test_task mechanism
RESOURCE_WAIT
s
Task is in penalty box because undo or temp tablespace is temporarily out of space
ha eฺ
STALL_WHEN_WAIT Used by stall_when mechanism )
LIBRARY_WAIT
ฺ c om Guid
A PLAN_DF task is running for a BACKUP_SBT task whose library is stalled
POOL_FULL_WAIT
n k ng ent
The task cannot proceed because the RA_POOL tablespace is full
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Agenda

Appliance
r a ble
a - tr
no n
h a sa
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li
m be
e
ake

Storage Locations
• Characteristics
– Shared among multiple Protected
Databases
– More efficient disk usage
– Database backup deduplication and
compression
• Storage Locations
– DELTA Store (default)
— Usable Capacity: Base Rack 94TB - Full Rack 580TB
(32TB/Storage Node)
a ble
Normal Redundancy fe r
—
an s
– Additional Stores (non default)
n - t r
— Expansion and space allotment
a no
–
—
Backup Pooling
as
Storage Nodes same/different rack and/or Multirack
) h eฺ
External NFS Storage ฺ c om Guid
ngbackups t to DELTA Store based on
—
Processes backups and copies n k d e n

foraspecific
—
Backup Polling Policy n b S t uProtected Databases

n i o i s
@ u e th
( a ak nse
Recovery e l lo
Appliance l i ce locations occupy space in Oracle ASM disk groups. By default,
storage
m b
k e e
the delta pool is stored in a normal redundancy Oracle ASM disk group called DELTA, which
a means that the Recovery Appliance maintains two copies of all on-disk backups. Database
backups can survive the loss of any one disk or storage server. The Recovery Appliance
metadata database, which tracks the files and blocks, is stored in a high redundancy Oracle
ASM disk group called CATALOG.
A more complex implementation may require multiple storage locations, for reasons such as
expansion and space allotment. For example Recovery Appliance with two storage locations:
DELTA (default) and DELTA2 (nondefault).
A Backup Polling Location is a file system directory where a protected database places
backups without interacting directly with Recovery Appliance. The backup polling directory is
an NFS mount point, and is not in a Recovery Appliance storage server. The polling policy
defines the file system path to the storage and how often it will be searched for new backups.

Creating a DELTA Store
• Create an additional DELTA Store (nondefault)
• Prerequisites
– Provision more Storage Nodes

– Configure cell disks
– Create Grid disks (ASM disks)
– Add Cell Server to Compute Nodes configuration
– Create new ASM disk group
• Attributes
– Free Space (GB)
– Storage Location Space (GB): If not specified, all available space will be used. If
amount is specified, you could increase at a later time but never decrease.
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
b el
Create Additional li
Storage Locations - DELTA Store
m
e create a Recovery Appliance storage location, follow these steps:
ake
To
1. From the Targets menu, choose Recovery Appliance. Enterprise Manager displays the
Recovery Appliance page. You may first see the Recovery Appliance Login page, where
you can enter the required credentials.
2. Click the Name of the Recovery Appliance for which you want to create a storage
location. The Recovery Appliance home page displays.
3. From the Recovery Appliance menu, choose Storage Locations. Enterprise Manager
displays the Storage Locations page.
4. Click Create. In the Create Storage Location dialog box, enter the Name of the new
storage location. Next, use the Search icon to select a Disk Group from the ASM Disk
Group dialog box and then click OK. Enter the amount of space in the Space field and
choose the unit of measure from the drop-down list.
5. Click OK to add the new storage location.
You could edit an existing DELTA Store to increase the storage location space but never to
decrease.

Deleting a DELTA Store
Deleting a DELTA Store

• Prerequisites
– You cannot delete a storage location that is used by a

Protection Policy.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li
ea DELTA Store
Deletingb
k e em
To delete a storage location from a Recovery Appliance, follow these steps:
a 1. From the Targets menu, choose Recovery Appliance. Enterprise Manager displays the
Recovery Appliance page.
2. Click the Name of the Recovery Appliance for which you want to delete a storage
location. The Recovery Appliance home page displays.
3. From the Recovery Appliance menu, choose Storage Locations. Enterprise Manager
displays the Storage Locations page.
4. Select the Storage Location from the table that you want to delete and then click Delete.
5. Click OK to delete the storage location.

Provisioning More Storage Nodes
• Expand Recovery Appliance Storage
• Preparation
– Obtain current configuration info: image history, IP addresses, cell

servers conf, compute servers HugePages mem conf, net switches
conf, OS users/groups, cluster status, and patch info
– OEDA to extend storage nodes
– Sync Release and Patch levels
• Add Storage Nodes to the Rack
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
el
ExpandbRecovery li
Appliance Store
m
e steps
ake
Overall
• Preparing to Extend Recovery Appliance
- Estimating InfiniBand Cable Path Lengths: Ensure that you have the required
number and lengths of InfiniBand cables for your installation.
- Obtaining the Current Configuration Information: Use the current configuration
of the Recovery Appliance rack to plan patching requirements, configure new IP
addresses, and so on (image history, IP addresses, cell servers conf, compute
servers HugePages mem conf, net switches conf, OS users/groups, cluster status,
and patch info)

• imagehistory sample:
[root@cell01 ~]# imagehistory

Version : 12.1.2.1.0.141206.1
Image activation date : 2015-03-13 19:18:22 -0600
Imaging mode : fresh
Imaging status : success
Version : 12.1.2.2.0.150917
Imaging mode : patch
a ble
fe r
[root@cell01 ~]#
ans
n - t r
opatch info sample: a no
•
h a s
m ) ideฺ
[oracle@host1 OPatch]$
g ฺ co t Gu
[oracle@host1 OPatch]$ ./opatch n
a kn den -oh /u01/app/12.1.0.2/grid
lsinventory
o nb versionS tu 12.1.0.1.9
Oracle Interim Patch Installer
u i h i s
n Corporation.
@
Copyright (c) 2016, Oracle
e t All rights reserved.
l o
el to u s
k b
a nse
( a
e l
Oracle lo Home lice : /u01/app/12.1.0.2/grid
b
m Central
e Inventory: /u01/app/oraInventory
ake from : /u01/app/12.1.0.2/grid/oraInst.loc
OPatch version : 12.1.0.1.9
OUI version : 12.1.0.2.0
Log file location: /u01/app/12.1.0.2/grid/cfgtoollogs/opatch/opatch2016-
07-28_01-14-54AM_1.log

Lsinventory Output file location:
/u01/app/12.1.0.2/grid/cfgtoollogs/opatch/lsinv/lsinventory2016-07-28_01-14-
54AM.txt
-------------------------------------------------------------------------
-------
Local Machine Information::
Hostname: host1.us.oracle.com
ARU platform id: 226
ARU platform description:: Linux x86-64
Installed Top-level Products (1):
a ble
Oracle Grid Infrastructure 12c
fe r
12.1.0.2.0
ans
There are 1 products installed in this Oracle Home.
n - t r
a no
h a s
m ) ideฺ
Interim patches (5) :
g ฺ co t Gu
a kn Mayd25
nWed e n
Patch 22304421 : applied bon
n S t u 17:16:25 MDT 2016
Unique Patch ID: 19700676
u nio this
Created on 22 o
l l @2015,
Dec
u s e03:47:46 hrs PST8PDT
Bugs fixed:
a k be e to
l o (a cens
22304421
l patch
beThis li overlays patches:
e m
ake
21694919
This patch needs patches:
21694919
as prerequisites
…

Patch level status of Cluster nodes :
Patching Level Nodes

-------------- -----
48024365 slcm03adm02,host1
-------------------------------------------------------------------------
-------
OPatch succeeded.
[oracle@host1 OPatch]$
- Generating the Network Configuration Files: Use the Oracle Exadata Deployment a ble
fe
Assistant to list the IP addresses for the new servers. Specify only the number of new r
ans
n - t r
servers. Define all other items, such as the Oracle home location and owner, the same as
a no
the existing rack configuration. In particular, the bonding configuration of the new servers
must match the existing servers in the rack. The Oracle Exadata Deployment Assistant
h a s
InfiniBand enables you to select the type of bonding. You must install the Recovery
m ) ideฺ
Appliance software in the new storage servers, after installing them in the rack.
ฺ o Gu
cdelete
- Moving the Audit and Diagnostic Files: Move
k g
n ent files in the
to
$GRID_HOME/rdbms/audit directory
b a n and the
u d$GRID_HOME/log/diagnostics
n t
Srecommends doing this task a day or two
u n io Oracle
directory before extending a cluster.
h i s
before the planned extension to allow
@ e t sufficient time.
- Synchronizing Release l o
el andtoPatch s
u Levels: The servers probably have a later release or
k b
a thenrack
patch levelathan
( e se where they will be added. You can either update the rack to the
b eallloservers,
laterlrelease
that
oric
l reimage the new servers to match the current rack. In either case, ensure
and all Sun Datacenter InfiniBand Switch 36 switches, are at the same
m
a kee patch level. Also consider how to handle the Grid Infrastructure and database home
releases and bundle patch updates.
Refer to the document: Zero Data Loss Recovery Appliance Owner's Guide for additional
information.

Storage Nodes: Disk Storage Entities and Relationships
PHYSICALDISK (Disk) LUN CELLDISK GRIDDISK ASM disk

Exadata Cell CellCLI> CREATE GRIDDISK ...
First two
Data LUNs only
Storage Grid
Partition Disk
System Area
a ble
e r
nstof
OR Cell OR Visible
Disk
- tr aASM
no n
a Disk
sGrid
LUN h a
) ideฺ (hot part)
Other m
co t GuGrid Disk
LUNs
g
n enฺ (cold part)
n k
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
Each Exadata
b i
el High lCapacity Storage Server contains 12 hard disk drives. Exadata cell
k e em automatically senses the physical disks in each storage server. As a cell

software
a administrator you can only view a predefined set of physical disk attributes. Each physical
disk is mapped to a logical abstraction called a Logical Unit (LUN). A LUN exposes additional
predefined metadata attributes to a cell administrator. You cannot create or remove a LUN;
they are automatically created.
Each of the first two disks contains a system area that spans multiple disk partitions. The two
system areas are mirror copies of each other which are maintained using software mirroring.
The system areas consume approximately 29 GB on each disk. The system areas contain the
OS image, swap space, Exadata cell software binaries, metric and alert repository, and
various other configuration and metadata files.
A cell disk is a higher level abstraction that represents the data storage area on each LUN.
For the two LUNs that contain the system areas, Exadata cell software recognizes the way
that the LUN is partitioned and maps the cell disk to the disk partition reserved for data
storage. For the other disks, Exadata cell software maps the cell disk directly to the LUN.
After a cell disk is created, it can be subdivided into one or more grid disks, which are directly
exposed to ASM.

Storage Nodes: Flash Storage Entities and Relationships
PHYSICALDISK (Flash) LUN CELLDISK FLASHCACHE

OR
FLASHLOG
Exadata Cell CellCLI> CREATE FLASHCACHE ...

CellCLI> CREATE FLASHLOG ...
a ble
fe r
Flash
a n s
Flash Cell Cache -t r
LUN Disk n on
a
s Flash
) h a ฺ Log
om Guid e
ฺ c
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
el X6-2 lHigh
Each Exadata
b i Capacity Storage Server contains 12.8 TB of high performance flash
k e em distributed across four PCI flash memory cards. Therefore, each flash card has a
memory
a capacity of 3.2 TB.
Essentially, each flash device is like a physical disk in the storage hierarchy. Each flash
device is visible to the Exadata cell software as a LUN and the initial cell configuration
process creates flash-based cell disks on all the flash devices.
By default, nearly all the available space on each flash-based cell disk is allocated to Exadata
Smart Flash Cache and only a small portion (128 MB on each flash-based cell disk, 512 MB
in total) is allocated to Exadata Smart Flash Log.

Storage Nodes: Disk Group Configuration
SQL> CREATE DISKGROUP
a ble
fe r
Exadata Cell CELL1 Exadata Cell CELL1 Exadata Cell
a n s
CELL1
n-t r
Failure Group
DELTA_2
a no
CELL1
Disk Group
CELL2 Failure Group
h a s CELL3 Failure Group
DELTA_3
m ฺ Failure Group
) ideCELL3
CELL1 Failure Group
Disk Group
CELL2 Failure Group
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
After grid l l o li
edisks are configured, ASM disk groups can be defined to consume Recovery
b
em storage. The slide illustrates an example where two ASM disk groups are defined.
eAppliance
ak The DELTA_1 disk group is defined across all of the red grid disks, and the DELTA_2 disk
group is defined across the blue grid disks. When data is loaded into each disk group, ASM
evenly distributes the data across all of the grid disks in each disk group.
Using NORMAL or HIGH ASM redundancy in conjunction with at least 3 failure groups for
each disk group is recommended on Recovery Appliance. This ensures that at least two
copies of data are maintained to protect from storage failure. EXTERNAL redundancy disk
groups are not supported because they provide no protection from storage failures and
prohibit the use of online storage maintenance procedures, such as rolling patches.
Note that the initial configuration process for Recovery Appliance creates 2 disk groups using
NORMAL(DELTA) or HIGH(CATALOG) ASM redundancy.
To protect against the failure of an entire Exadata cell, separate ASM failure groups are
automatically associated with each cell, which ensures that mirrored ASM extents are placed
on different Exadata cells. This is also illustrated in the slide. By default, when failure groups
are automatically created, their names correspond to the cell name. So, different disk groups
can have the same failure group names. For example, the DELTA_1 disk group has a failure
group named CELL1, and the DELTA_2 disk group also has a failure group named CELL1.

Storage Nodes: CellCLI
• Each Exadata Storage Server is administered individually.

• Most administration is performed using CellCLI.
– CellCLI can be executed only on the cell being administered.

– CellCLI works in conjunction with MS to perform administration
tasks.
Example
[celladmin@zdlra1cel01 ~]$ cellcli

CellCLI: Release 12.1.2.1.0 - Production ...
a ble
fe r
CellCLI> list cell
zdlra1cel01 online
an s
CellCLI> exit
n - t r
quitting
a no
[celladmin@zdlra1cel01 ~]$
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Inside b e l l
Recovery
o li
Appliance, each Exadata Storage Server runs independently from all the
k e em Exadata Storage Servers. In line with that autonomy, each cell is administered
other
a individually. Most administration functions are performed using the Exadata cell command-
line interface (CellCLI). CellCLI can be used only from within a cell to manage that cell.
However, you can run the same CellCLI command remotely on multiple cells with the dcli
utility, which is described later in this lesson.
CellCLI works in conjunction with the Exadata Storage Server Management Server. CellCLI
provides the command interface while MS performs the administrative functions, such as
creating and dropping grid disks.

Storage Nodes: User Accounts
Three operating system users are configured for each Exadata Storage
Server:
• The root user can:

– Edit configuration files such as cellinit.ora and cellip.ora
– Change network configuration settings
– Run support and diagnostic utilities located under the
/opt/oracle.SupportTools directory
– Run the CellCLI CALIBRATE command
a b le
– Perform all the tasks that the celladmin user can perform fe r
n s
• The celladmin user can:
n - tra
– Perform administrative tasks (CREATE, DROP, ALTER,
a noand so on)
using the CellCLI utility
h a s
– Package incidents for Oracle Support using m ) the adrci
d e ฺutility
ฺ i
co t Gu Exadata cell objects
• The cellmonitor user can only view (LIST,
n g DESCRIBE)
n
using the CellCLI utility. ank de
nb s Stu
n i o i
u e t h
a a kb se t
l o ( cen
Three b e l
operating li users are configured for each Exadata Storage Server: root,
system
k e em
celladmin, and cellmonitor. The slide describes the function of each user account.
a After Exadata is initially configured, the operating system user accounts are set with initial
passwords. The default initial password for root is welcome1. The default initial password
for the cellmonitor and celladmin users is welcome. It is recommended that the initial
passwords for all the user accounts should be changed to more secure passwords after initial
configuration is completed.

Storage Nodes - Running Commands from Compute Nodes
ExaCLI enables cell management from remote hosts:

• Enables tighter control to cell access:
– No SSH connection to the cell is required

– Communication uses https
• Requires access through a user with role-based privileges
• Supports CellCLI command syntax, with some restrictions
$ exacli -c [username@]host [-l username] [—xml] [--cookie-jar [filename]] [-e CellCLI_command]
$ exacli -l celladministrator -c zdlra1cel01

Password=************ Examples
a ble
ExaCLI>
fe r
$ exacli -c celladministrator@zdlra1cel01
an s
Password=************
n - t r
no
ExaCLI>
s a
$ exacli -c celladministrator@zdlra1cel01 --cookie-jar -e list cell
Password=************
zdlra1cel01 h a
) ideฺ
online
$ m
co t Gu
g ฺ
a
CD_00_zdlra1cel01 kn den
$ exacli -l celladministrator -c zdlra1cel01 -e list celldisk
n normal
...
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Exadata l l o li 12.1.2.2.0 introduces the ExaCLI utility. ExaCLI is essentially the
esoftware release
b
k e em as CellCLI; however, the main difference is that ExaCLI manages cells from a remote
same
a host, typically an Exadata database server, whereas CellCLI runs directly on a cell.
ExaCLI enables users to perform most management functions without the requirement to
establish an SSH connection to the cell. Consequently, access to the cell operating system
can be tightly controlled to address various operational and compliance requirements.
To use ExaCLI, non-default users must be created on the cell, and the users must be
assigned roles that grant appropriate privileges. Creation of users and roles can only be
performed directly on the cell using CellCLI.
All communication between ExaCLI and Management Server (MS) running on the cell is over
https. Security certificates allow the cells to confirm their identity to ExaCLI. MS is deployed
with a default self-signed security certificate, or you can upload a different security certificate
issued by a Certificate Authority (CA).
ExaCli supports the same command syntax as CellCLI. However, not all CellCLI commands
can be executed through ExaCLI. Unsupported commands include:
• ALTER CELL with the RESTART, STARTUP or SHUTDOWN options
• CREATE USER, ALTER USER, DROP USER
• CALIBRATE

Storage Nodes: Distributed CLI
The dcli utility allows you to simultaneously execute a command on multiple Exadata servers.
• Command types:
– Operating system commands

– CellCLI commands
– Operating system scripts
– CellCLI scripts
• Commands are executed in separate parallel threads.
• Interactive sessions are not supported.
• Python 2.3 and SSH user-equivalence are required.
• Command output is collected and displayed in the terminal session executing the dcli utility.
a ble
$ dcli –g mycells date
fe r
exa1cel01: Sun Oct 30 20:48:09 CDT 2015
an s
exa1cel02: Sun Oct 30 20:48:09 CDT 2015
n - t r
no
$ dcli –c exa1cel01,exa1cel02 cellcli –e list cell
a
exa1cel01: exa1cel01
online
online
h a s
Examples
m ) ideฺ
co t Gu
$ dcli –g mycells –x cellclicommands.scl
g ฺ
a n kn den
$ dcli –g mydbservers –l root –x dbwork.sh
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li centralized management across Recovery Appliance by
e utility facilitates
The dcli
b
em
eautomating the execution of a command on a set of servers and returning the output to the
ak centralized management location where dcli was run. The types of commands supported by
dcli includes operating system commands, CellCLI commands, operating system scripts,
and CellCLI scripts. However, it does not support interactive sessions.
The dcli utility runs commands on multiple servers in parallel threads. By default, the dcli
utility is located at /opt/oracle/cell/cellsrv/bin/dcli on each Exadata Storage
server and at /usr/local/bin/dcli on each database server. You can also copy the
dcli utility to a server outside of Recovery Appliance and launch commands from that
server.
The dcli utility requires Python version 2.3 or later on the server running dcli. You can
determine the version of Python by running the python -V command. In addition, dcli
requires prior setup of SSH user-equivalence between all the servers. You can use the dcli
utility initially with the -k option to set up SSH user-equivalence between a group of servers.
Command output (to stdout and stderr) is collected and displayed after command execution is
finished on all the specified servers. The dcli options allow command output to be
abbreviated to filter output, such as removing messages showing normal status.

Storage Nodes: Running Distributed CLI Commands from Compute Nodes
The exadcli utility allows you to simultaneously execute a command on multiple Exadata Storage
Servers.
• Command types:
– ExaCLI commands
– ExaCLI scripts
• Commands are executed in separate parallel threads using ExaCLI:
– Requires access through a user with role-based privileges
– Uses the same cookie jar mechanism as ExaCLI
• Interactive sessions are not supported.
• Command output is collected and displayed in the terminal session executing the exadcli utility.
a ble
$ exadcli –c exa1cel01,exa1cel02 -l celladministrator
fe r
--cookie-jar –e list cell
an s
r
Password=************
online
online
n - t
$ cat mycells
a no
exacel01
exacel02
h a s
Examples $ cat cmd.txt
m ) ideฺ
co t Gu
list cell
g ฺ
kn den
$ exadcli -g mycells -l celladministrator -x cmd.txt
exa1cel01: exa1cel01 online
a n
exa1cel02: exa1cel02 online
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Exadata l l o li 12.1.2.2.0 introduces the exadcli utility. Like dcli, exadcli
esoftware release
b
k e em centralized management across Exadata by automating the execution of a
facilitates
a command on a set of cells and returning the output to the centralized management location
where exacli was run.
The main difference is that dcli uses SSH for communication, while exadcli uses https in
the same way as ExaCLI. By removing the requirement for SSH, exadcli enables tight
control over access to the cell operating system, which is useful in addressing various
operational and compliance requirements.
Another difference is that exadcli can only be used to issue ExaCLI commands to be run on
multiple cells. Unlike dcli, exadcli cannot be used to execute other commands, such as
shell commands.
To use the exadcli utility, you must set up non-default users and roles on all the cells in the
same way as required by ExaCLI. Also, exadcli uses the same cookie jar mechanisms as
ExaCLI.

By using exadcli, you can run one ExaCLI command or you can run a series of commands in
order. You can specify multiple commands on the command line by separating them with a
semicolon (;) character, or you can specify a file that contains a list of ExaCLI commands. You
cannot use exadcli interactively.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Storage Nodes: Testing by Using CALIBRATE
Disk bandwidth Channel bandwidth

Metric = IOPS Metric = MBPS
Need large
Need high RPM and
I/O channel
fast seek time
a ble
fe r
an s
OLTP Analytics n - t r
(Small random I/O) no
(Large sequential I/O)
a
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
nb s Stu
CALIBRATE
n i o i
u e t h
a a kb se t
l o ( cen
e l
The CALIBRATE
b li
command runs raw performance tests on Exadata Storage Server disks and
emmodules. This enables the measurement of two important metrics – IOPS and MBPS:
eflash
ak • IOPS (I/O per second): This metric represents the number of small random I/O that can
be serviced in a second. The IOPS rate typically depends on how fast the disk media
can spin and how many disks are present in the storage system.
• MBPS (megabytes per second): This metric represents the rate at which data can be
transferred between the server and storage array. This mainly depends on the capacity
of the I/O channel that is used to transfer the data.

Sample output:
[root@zdlra1cel01 ~]# cellcli
CellCLI: Release 12.1.2.1.0 ...
CellCLI> CALIBRATE FORCE

Calibration will take a few minutes...
Aggregate random read throughput across all hard disk luns: 1604 MBPS
Aggregate random read throughput across all flash disk luns: 4242.9 MBPS
Aggregate random read IOs per second (IOPS) across all hard disk luns: 4927
Aggregate random read IOs per second (IOPS) across all flash disk luns:
148695 Controller read throughput: 1608.05 MBPS
Calibrating hard disks (read only) ...
Lun 0_0 on drive [20:0 ] random read throughput: 153.41 MBPS, and 412 IOPS
a ble
fe r
ans
...
n - t r
no
a
a s
h
) ideฺ
Calibrating flash disks (read only, note that writes will be significantly
m
slower)
ฺ o Gu
cthroughput:
Lun 1_0 on drive [[10:0:0:0]] random read g
kn read n t 269.11 MBPS, and 19635
n
IOPS Lun 1_1 on drive [[10:0:1:0]] arandom
u d e
n b t throughput: 268.86 MBPS, and
19648 IOPS
u n io his S
...
@ e t
l o s
el to u random read throughput: 268.33 MBPS, and 19717
k b
Lun 5_2 on drive [[11:0:2:0]]
IOPS Lun 5_3 ona drive s e
( a enresults
19693 IOPSoCALIBRATE
[[11:0:3:0]] random read throughput: 268.14 MBPS, and
b ell lic are within an acceptable range.
e m
eCALIBRATE
a k stress test is now running...
Calibration has finished.

Storage Nodes: Starting and Stopping Cell Services
[celladmin@exa1cel01 ~]$ cellcli

CellCLI: Release 12.1.2.1.0 ...
CellCLI> ALTER CELL RESTART SERVICES ALL
Stopping the RS, CELLSRV, and MS services...

The SHUTDOWN of services was successful.
Starting the RS, CELLSRV, and MS services...
Getting the state of RS services...
running
a ble
Starting CELLSRV services...
The STARTUP of CELLSRV services was successful.
fe r
Starting MS services...
an s
The STARTUP of MS services was successful.
n - t r
CellCLI>
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e you may
Occasionally, li need to start, stop, or restart the Exadata Storage Server software
b
k e embe achieved
services:
can
Restart Server (RS), Management Server (MS), and Cell Server (CELLSRV). This
by using the CellCLI ALTER CELL command. An example is shown on the
a
slide. Following is a summary of the relevant syntax:
ALTER CELL { SHUTDOWN | RESTART | STARTUP } SERVICES { RS | MS |
CELLSRV | ALL}

Storage Nodes: Configuring Cell Disks
CellCLI> CREATE CELLDISK ALL HARDDISK

CellDisk CD_00_exa1cel01 successfully created
...
CellCLI> LIST CELLDISK

CD_00_exa1cel01 normal
...
CD_10_exa1cel01 normal
a ble
CD_11_exa1cel01
FD_00_exa1cel01
normal
normal
fe r
...
an s
FD_14_exa1cel01 normal
n - t r
no
FD_15_exa1cel01 normal
CellCLI>
s a
h a
) ideฺ
m
co t Gu
g ฺ
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li in an Exadata Storage Server can be used, cell disks must be created
e disk storage
Beforebthe
k e em
by using the CREATE CELLDISK command.
a Note that in most cases, the initial configuration process, performed by using the Oracle
Exadata Deployment Assistant, will configure Exadata Storage Server cell disks and grid
disks.
The example in the slide shows the use of the CREATE CELLDISK ALL HARDDISK
command to create 12 disk-based cell disks with default names. In most cases, the default
cell disk names are used.
The preceding example also shows the use of the LIST CELLDISK command. For a High
Capacity Exadata Storage Server, output from the LIST CELLDISK command shows the 12
disk-based cell disks along with 16 flash-based cell disks that are normally used in
conjunction with Exadata Smart Flash Cache. The command should return a status of normal
for all the cell disks.

Storage Nodes: Configuring Grid Disks
CellCLI> CREATE GRIDDISK ALL PREFIX=delta_1, SIZE=2000G

GridDisk delta_1_CD_00_zdlra1cel01 successfully created Before After
...
GridDisk delta_1_CD_11_zdlra1cel01 successfully created
Cell Grid
CellCLI> CREATE GRIDDISK ALL PREFIX=delta_2 disk disks
...
CellCLI> LIST GRIDDISK

delta_1_CD_00_zdlra1cel01 active
… …
a ble
...
fe r
delta_1_CD_11_zdlra1cel01
delta_2_CD_00_zdlra1cel01
active
active
an s
...
n - t r
no
delta_2_CD_11_zdlra1cel01 active
CellCLI> exit
s a
[celladmin@zdlra1cel01 ~]$
h a
) ideฺ
m
co t Gu
g ฺ
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
After cell l l o li
edisks are created, grid disks must be provisioned using the CREATE GRIDDISK
b
em Like cell disks, grid disks are typically created as part of the initial configuration of
ecommand.
ak Recovery Appliance. However, there are times when existing grid disks may be removed and
replaced by new ones, such as when the disk groups are resized. This case is considered in
further detail later in the lesson.
The slide shows examples of how to create grid disks on empty new cell disks. In the example
in the slide, the ALL PREFIX option is used to automatically create one grid disk on each
cell disk. When the ALL PREFIX option is used, the generated grid disk names are
composed of the grid disk prefix followed by an underscore (_) and then the cell disk name.
It is best practice to use the planned ASM disk group name as the prefix name for the
corresponding grid disks. In the example, prefix values delta_1 and delta_2 are the
names of planned ASM disk groups that will consume the grid disks. Grid disk names must be
unique across all cells within a single Recovery Appliance deployment. By following the
recommended naming conventions for naming the cell disks and grid disks, you automatically
get unique names.

The optional SIZE attribute specifies the size of each grid disk. If omitted, the grid disk will
automatically consume all the space remaining on the corresponding cell disk.
The LIST GRIDDISK command shows all the grid disks that are defined on the storage
server.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Storage Nodes: Configuring Hosts to Access Grid/ASM Disks
• Configuration files on each database server enable access to

Exadata storage.
– cellinit.ora identifies the storage network interfaces on the

database server.
– cellip.ora identifies the Exadata cells that are accessible to
the database server.
$ cat /etc/oracle/cell/network-config/cellinit.ora
a ble
ipaddress1=192.168.10.1/24
fe r
ipaddress2=192.168.10.2/24
an s
$ cat /etc/oracle/cell/network-config/cellip.ora
n - t r
no
cell="192.168.10.5; 192.168.10.6"
cell="192.168.10.7; 192.168.10.8"
cell="192.168.10.9; 192.168.10.10"
s a
h a
) ideฺ
m
co t Gu
g ฺ
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l o
eof the initialli configuration for Recovery Appliance, two important configuration files
As a part l
b
emcreated on each database server, which enable it to access Exadata Storage Servers:
eare
ak • The cellinit.ora file contains the database server IP address that connects to the
storage network. This file is host specific, and contains the IP addresses of the
InfiniBand storage network interfaces for that database server. The IP addresses are
specified in Classless Inter-Domain Routing (CIDR) format.
• The cellip.ora file contains the IP addresses for the InfiniBand storage network
interfaces of the Exadata Storage Servers that are accessible to the database server.
If there are ever any issues with connectivity between a database server and Exadata
Storage Servers, then verify these files to ensure that the correct settings are present. If
changes are ever required to these files, then:
1. Stop the database and the Oracle ASM instances on the database server host.
2. Make the required file changes.
3. Restart the database and the Oracle ASM instances on the database server host.

Storage Nodes: Configuring ASM Instances to Access Grid/ASM Disks
• Ensure that a compatible version of the Oracle Database

software is being used:
– Zero Data Loss Recovery Appliance Supported Versions (Doc

ID 1927416.1)
• Set the ASM_DISKSTRING ASM initialization parameter:
– ASM_DISKSTRING='o/*/*'
• Set the COMPATIBLE database initialization parameter: le
– COMPATIBLE='11.2.0.0.0' (or later) r a b
s fe
- t r an
no n
s a
h a
) ideฺ
m
co t Gu
g ฺ
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e appropriate
At all times, li versions of the various Exadata software components must be
b
em
emaintained in order to ensure correct operation. This includes the version of Oracle Database
ak software used for the ASM and database instances. Refer to My Oracle Support note Zero
Data Loss Recovery Appliance Supported Versions (Doc ID 1927416.1) for an up-to-date list
of the supported versions for the Exadata software components.
To ensure that ASM discovers Exadata grid disks, set the ASM_DISKSTRING initialization
parameter. A search string with the following form is used to discover Exadata grid disks:
o/<cell IP address>/<grid disk name>
Wildcards may be used to expand the search string. For example, to explicitly discover all the
available Exadata grid disks set ASM_DISKSTRING='o/*/*'. To discover a subset of
available grid disks having names that begin with data, set
ASM_DISKSTRING='o/*/data*'.

Note that if the ASM_DISKSTRING initialization parameter is not set, then the default is to
discover all the currently accessible Exadata grid disks; that is, the grid disks for the cells listed
in cellip.ora and that are not prohibited by a security setting. Exadata Storage Server
security is discussed later in this lesson.
To configure a database instance to access cell storage, ensure that the COMPATIBLE
parameter is set to 11.2.0.0.0 or later in the database initialization file.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Summary

• Identify various configuration tasks
• Use OEDA to install the Recovery Appliance

• Create different administration accounts
• Use metadata information in the Recovery Appliance
• Use the Recovery Appliance storage structures
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

6
ZDLRA Protected Databases Configuration a ble

fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake
Objectives

• Enroll Protected Databases
• Install the Recovery Appliance Module in the Protected

Database
• Enable Real-Time Redo Transport in the Protected Databases
• Execute Backup and Recovery jobson Protected Databases
e
• Monitor Backup and Recover jobs abl fer
a n s
n - tr
no
h a sa
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li
m be
e
ake

Agenda
• Protected Databases Overview

• Install the Recovery Appliance Module for Protected
Databases
• Enable Real-Time Redo Transport in a Protected Database
• Take a Backup of a Protected Database
• Recover a Protected Database a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Oracle Database 12c
Multitenant Database Architecture Overview
Non-Container Database (CDB) Architecture
• Multiple monolithic or non-CDBs share nothing:
– Too many background processes

– High shared/process memory
– Many copies of Oracle metadata
CDB Architecture - Multitenancy
• Operates multiple databases in a centrally managed
platform to lower costs:
– Less instance overhead RAC PDB
– Less storage cost Instance Services
• Reduces DBA resources costs and maintains security
a ble
– No application changes fe r
a n spdbc
– Fast and easy provisioning
– Time saving for patching and upgrade
r
-t crac3
crac1 ncrac2
pdba pdbb
o
– Backup and Recovery operations
s an
• Maintain separation of duties between:
) ha eฺ
PDB
– Different application administrators
– Application administrators and DBA ฺ c om Guid PDBA PDBB PDBC
k n g nCDB t CRAC
– Users within application
a n ud e
• Provides isolation
n b S t
u nio this
l l o @ use
be e to
a k
l o (a cens
Non-CDB
b li
elArchitecture
eIn m 11g database, the only kind of database that is supported is a non-CDB. The old
eOracle
a k architecture is referred to as the non-CDB architecture. The term non-CDB will be used as a
shorthand for an occurrence of a pre-12.1 database that uses the pre-12.1 architecture. This
database requires its own instance and therefore its own background processes, memory
allocation for the SGA. It needs to store the Oracle metadata in its data dictionary. The
database administrator can still create Oracle 12c non-CDBs with the same pre-12.1
architecture. These databases are not multitenant container databases or non-CDBs.
CDB Architecture
Consolidating many non-CDB databases onto a single platform reduces instance overhead,
avoids redundant copies of data dictionaries, and consequently storage allocation. This
benefits from fast provisioning, time-saving upgrading, better security through the separation
of duties and application isolation. The new 12c database that consolidates databases
together is a multitenant container database (CDB), and a database consolidated within a
CDB, a pluggable database (PDB).

DBA resource costs are reduced with:
• No application change and very fast provisioning: A new database can be provisioned
very quickly. A clone of a populated database can be created very quickly. A populated
database can be quickly unplugged from its CDB on one platform and quickly plugged
into a CDB on a different platform. A non-CDB can quickly be plugged into a CDB.
• Fast upgrade and patching of the Oracle Database version: The cost (time taken and
human effort needed) to upgrade many PDBs is the cost of upgrading a single Oracle
Database occurrence. You can also upgrade a single PDB by unplugging it and
plugging it into a CDB at a different Oracle Database version.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

Protected Database Administration
Task Overview
• Install the Recovery Appliance Backup Module on Protected
Databases
• Decide on a strategy to migrate your protected database backups to

• Enroll the protected database with a Recovery Appliance
• Configure backup and recovery settings for the protected database
• Perform a test backup to verify that your protected database
configuration is accurate
r a ble
• Back up the protected database to the Recovery Appliance
n s fe
• Perform a test restore and recovery to assure yourself that-tthe
n ra
protected database can be recovered using the backups
a no stored on
the Recovery Appliance h a s
• In the event of a failure (caused by a mediam )failureidor
e ฺdata
g ฺ co tusing
corruption), recover the protected database G u backups stored on
the Recovery Appliance a n kn den
b tu
on is S
n i
u e th
@
llo to us
b e Copyright © 2016, Oracle and/or its affiliates. All rights reserved.
( a ak nse
e
This slide l lo
describes l i
the
cehigh-level tasks in using the Recovery Appliance to store and
m b
ak ee
manage backups for multiple protected databases in the enterprise. These tasks can be
performed using Cloud Control or RMAN. Depending on the management interface used,
there may be minor variations in the steps to perform certain tasks.

Protected Database Administration
Tools
Multiple interfaces
• Oracle Enterprise Manager Cloud Control
• RMAN Client
• SQL*Plus or SQL Developer
Oracle Databases
Cloud Control
Tape Library
a ble
RMAN SQL*Plus
fe r
an s
n - t r
a no
h a s
) ideฺ
Recovery Appliance
m
g ฺ co t Gu
SuperCluster Exadata a n kn den Recovery Appliance
n b S t u Replica
n i o i s
u e t h
a a kb se t
l o ( cen
l
e Appliance
The Recovery
b li provides multiple interfaces to manage backup and recovery
em for protected databases.
eoperations
ak • Oracle Enterprise Manager Cloud Control (Cloud Control): Cloud Control provides a
GUI for administering, managing, and monitoring a Recovery Appliance environment. It
also enables you to configure, back up, and recover protected databases.
• RMAN client: The Recovery Appliance is integrated with RMAN and you can use the
RMAN client installed on your protected database to configure, back up, and recover
• SQL*Plus: SQL*Plus is a command-line tool that you can use to query the Recovery
Appliance catalog and run the DBMS_RA PL/SQL package.

Protected Databases
Supported Features
• Features available per release

12.1.0.2
Features Available for Oracle Database Releases 10.2.0.5 11.2.0.3 11.2.0.4
and later
Backups to the Recovery Appliance, with support for

x x x x
virtual full backups (1)
Real-time redo transport to the Recovery Appliance

a ble
from Linux, Windows, Solaris x86, SPARC, HP/UX and x x x
fe r
AIX platforms (2)
an s
n - t r
Backup polling from NFS-mounted fast recovery area
n o x x
h a sa
Cross-endian backup polling
m ) idexฺ x
o
ฺc t G u
n g
k den
a n
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l o
e databaseli can be Standard or Enterprise Edition on any storage or volume
l
A protected
b
m (such as filesystem, ASM, ACFS, ZFS) and use any of the following: Oracle Real
kee
manager
a Application Clusters, Oracle Data Guard, Oracle Streams and/or Oracle GoldenGate. Note
that polling locations are only supported on NFS-based locations, not ASM.
Note
1. Virtual full backups are not supported for RMAN encrypted backup sets.
2. Real-time redo is available for 11.2.0.3, but not as an option to be configured using
Enterprise Manager, so you need to do it manually. Enterprise Manager provides the
option to configure real-time redo in versions 11.2.0.4+.

Protected Databases
Supported Features
• Features available per release

12.1.0.2
Features Available for Oracle Database Releases 10.2.0.5 11.2.0.3 11.2.0.4
and later
Redo encryption during transport x x
Oracle Data Guard Broker support for configuring

a ble
redo transport from a primary database, and for
x x
fe r
preserving redo transport on the new primary
an s
database after an Oracle Data Guard role transition
n - t r
Oracle Data Guard Broker support for configuring o
redo transport from a standby database, and for
s an x
preserving redo transport on the new standby
database after an Oracle Data Guard role transition
) ha eฺ
ฺ c om Guid
ng ent
Real-Time redo cascading, i.e. transferring live redo
from a Far Sync instance or physical standby
n k x
database to the Recovery Appliance (3)
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
Note bel li
e m
3. The Recovery Appliance itself cannot serve as a cascader that transfers received redo
ake to a standby database.
For additional information about features supported by version please review the following
support note: Zero Data Loss Recovery Appliance Features Available per Oracle Database
Release (Doc ID 1995866.1)
Also check the following support note for recommended database patches: Recommended
Protected Database Patches for Zero Data Loss Recovery Appliance (Doc ID 1982542.1)

Agenda

Databases
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Recovery Appliance Backup Module
Protected Database Configuration
• The Recovery Appliance backup module is a shared library that transfers
backup data to and from the Recovery Appliance.
• It is installed in the host of the Protected Database.

• It must be installed into a location within the shared library search path that
is visible to every protected database instance ($ORACLE_HOME/lib).
• The Recovery Appliance backup module location is used with the
SBT_LIBRARY parameter in the ALLOCATE CHANNEL or CONFIGURE
CHANNEL command.
• One or more channels can be allocated implicitly or explicitly for each
r a ble
backup and recovery operation. sfe n
n - tra
o
s an
Download RA ) ha eฺ
Backup Module
from OTN gฺ
comt Guid
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
The Zero l l o i
eData LosslRecovery Appliance Backup Module (Recovery Appliance Backup
b
em is an Oracle-supplied SBT library that RMAN uses to transfer backup data over the
eModule)
ak network to the Recovery Appliance. An SBT library transfers data to and from a backup
device type, either a tape device or the Recovery Appliance. RMAN performs all backups to
the Recovery Appliance, and all restores of complete backup sets, by means of this module.
The Recovery Appliance Backup Module must be installed in the following locations:
• In the Oracle home of every protected database that sends backups to a Recovery
Appliance. For example, a single host might have an Oracle Database 11g Oracle
home, and an Oracle Database 12c Oracle home. Each Oracle home might support five
protected databases, for a total of ten databases running on the host. In this case, only
two Recovery Appliance Backup Modules must be installed: one in each Oracle home.
• For Recovery Appliance replication environments, on every upstream Recovery
Appliance that sends backups to downstream Recovery Appliances

To download the Recovery Appliance Backup Module from OTN navigate to the following
URL:
http://www.oracle.com/technetwork/database/availability/oracle-zdlra-backup-module-
2279224.html
The file downloaded from OTN is called “ra_installer.zip” and includes two files:
“ra_install.jar” and “ra_readme.txt”
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Sample Installation
• Installs the credentials, config file (network) and software required to use the
Recovery Appliance software
• Prerequisites
– ra_install.jar file
– Java 1.5 or later
• Input parameters
– ZDLRA Username and Password (RASYS or other)
– ZDLRA Host, Port and ServiceName (SCAN)
• Output parameters
– Wallet File Location: $ORACLE_HOME/dbs/ra_wallet (default) a ble
– Config File Location: $ORACLE_HOME/dbs/ra<ORACLE_SID>.ora (default) fe r
an s
– Lib File Location: $ORACLE_HOME/lib (default)
n - t r
a
$ java -jar ra_install.jar -dbUser rasys -dbPass ra -serviceName zdlra4 no -port 1521 -host zdlra-scan5 \
h a s
-walletDir $ORACLE_HOME/dbs/ra_wallet -libDir $ORACLE_HOME/lib/ -proxyHost www-proxy.demo.com
Recovery Appliance Install Tool, build 2014-09-16

Recovery Appliance credentials are valid.
m ) ideฺ
ฺ co t Gu
Recovery Appliance wallet created in directory /u01/app/oracle/product/11.2.0.3/dbhome_1/dbs/ra_wallet.
g
Recovery Appliance initialization file /u01/app/oracle/product/11.2.0.3/dbhome_1/dbs/raLUZ11.ora created.
a n kn den
Downloading Recovery Appliance Software Library from file ra_solaris_sparc64.zip.
Downloaded 28540438 bytes in 44 seconds. Transfer rate was 648646 bytes/second.
nb s Stu
Download complete.
n i o i
u e t h
a a kb se t
l o ( cen
e installs theli credentials and software that are required to use the ZDLRA library.
This utility
b l
em output for arguments:
eSample
ak host:ra_installer user$ java -jar ra_install.jar
Recovery Appliance Install Tool, build 2015-11-03
No arguments supplied
Usage: java -jar ra_install.jar
-dbUser: Database Username
-dbPass: Database Password
-host: SCAN hostname of ZDLRA Server
-port: ZDLRA Listening port
-servicename: ZDLRA service name of database
-walletDir: Directory to store wallet
-configFile: File name of config file
-libDir: Directory to store library
-libPlatform: Platform of library to download Recovery
Appliance Install Tool.
…
The file downloaded from OTN is called “ra_installer.zip” and includes two files:
“ra_install.jar” and “ra_readme.txt”.
Review the ra_readme.txt file for additional information.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Oracle Wallet
Wallet creation
• Created automatically by ra_install.jar when you install the Recovery Appliance
backup module.
• Can be created manually when missing or damaged
– Use the mkstore tool
– Parameters:
— Wallet location
— credential name
— VPC User [oracle@host1 ~]$ mkdir -p $HOME/oracle/wallet
[oracle@host1 ~]$ mkstore -wrl $HOME/oracle/wallet \
> -createALO -createCredential crmdb vpccrmuser
Oracle Secret Store Tool: Version 11.2.0.4.0 - Production
a ble
Copyright (c) 2004, 2013, Oracle and/or its affiliates. All rights reserved.
fe r
Your secret/Password is missing in the command line
an s
Enter your secret/Password:
Re-enter your secret/Password:
n - t r
a no
Create credential oracle.security.client.connect_string1
[oracle@host1 ~]$
h a s
) ideฺ
[oracle@host1 ~]$ mkstore -wrl $HOME/oracle/wallet -listCredential
m
co t Gu
ฺ
Copyright (c) 2004, 2013, Oracle and/or its affiliates. All rights reserved.
g
a n kn den
List credential (index: connect_string username)
1: crmdb vpccrmuser
nb s Stu
[oracle@host1 ~]$
n i o i
u e t h
a a kb se t
l o ( cen
l li the credentials of the Recovery Appliance user that will be used by
e wallet stores
An Oracle
b
emprotected database to authenticate with the Recovery Appliance. These same credentials
ethe
ak are used for sending backups and redo, if configured. When you install the Recovery
Appliance backup module, an Oracle wallet is automatically created. You can also create the
wallet and add required entries manually.
Sample output for arguments:
[oracle@host ~]$ mkstore
Copyright (c) 2004, 2013, Oracle and/or its affiliates. All rights
reserved.
mkstore [-wrl wrl] [-create] [-createSSO] [-createLSSO] [-createALO]
[-delete] [-deleteSSO] [-list] [-createEntry alias secret] [-
viewEntry alias] [-modifyEntry alias secret] [-deleteEntry alias] [-
createCredential connect_string username password] [-listCredential]
[-modifyCredential connect_string username password] [-
deleteCredential connect_string] [-help] [-nologo]
[oracle@host ~]$

Agenda

Databases
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Real-Time Redo
Configuration
• Using Oracle Enterprise Manager

Unprotected Data Window

a b le
r
is maintained < 1sec
e
a n sf
during the load
n - tr
Real-Time no
Redo Transport
h a sa
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li - Oracle Enterprise Manager
e Redo setting
Real-Time
b
k e emeach Database in the Backup Settings:
For
a 1. Select the Recovery Appliance that you are protecting.
2. Ensure you select the Correct VPC user.
3. Select Enable Real-Time Redo Transport. This is an OEM procedure. You can view its
progress in OEM.
Ensure that you take a backup after completing the REDO Transport operation to ensure all
Logs are collected.
During the load, the Recovery Appliance tries to maintain a lag of less than a second for the
unprotected data window.
Remember that when you configure real-time redo transport, redo data from the protected
database is directly transported and stored on the Recovery Appliance. This reduces the
window of potential data loss that exists between successive archived log backups.

Real-Time Redo
Configuration
• Using the CLI

Archived Log
Destination points to
a ble
fe r
an s
Redo Transport User
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
Real-Time
b li - CLI
el Redo Setting
k e emeach database, execute the following steps:
For
a 1. Ensure that the Recovery Appliance user that the protected database uses to send
backups to the Recovery Appliance is configured. This same user will be used for redo
transport.
- You have learned how to create this user in the previous lesson. In this lesson you
will learn how to assign the privileges (enrolling) needed to send backups and for
redo transport.
- Also ensure that an Oracle wallet is created on the protected database that
contains credentials for the Recovery Appliance (and redo transport) user. This
wallet is created automatically when you installed the Recovery Appliance Backup
Module in the Protected Database’s Oracle Home directory.
2. Ensure that the following conditions are met for the protected database:
- ARCHIVELOG mode is enabled
- DB_UNIQUE_NAME parameter is set

3. Ensure that the REMOTE_LOGIN_PASSWORDFILE and LOG_ARCHIVE_FORMAT
initialization parameters are set for the protected database:
- REMOTE_LOGIN_PASSWORDFILE = exclusive -or- shared
- LOG_ARCHIVE_FORMAT = ‘log_%d_%t_%s_%r.arc' -or- similar
4. (Only on RAC) Set the LOG_ARCHIVE_CONFIG initialization parameter to include a
DG_CONFIG list. Also set the DB_UNIQUE_NAME for the protected database.
- ALTER SYSTEM SET DB_UNIQUE_NAME = <protected_database_name>

SCOPE=BOTH;
- ALTER SYSTEM SET LOG_ARCHIVE_CONFIG =
‘DG_CONFIG=(<ra_database_name>,<protected_database_name>)’
SCOPE=BOTH;
5. Configure an archive log destination that points to the redo staging area on the Recovery
Appliance.
a ble
- ALTER SYSTEM SET LOG_ARCHIVE_DEST_<n>=‘SERVICE=<service_name>
fe r
VALID_FOR=(ALL_LOGFILES, ALL_ROLES) ASYNC ans
DB_UNIQUE_NAME=<ra_database_name>’ SCOPE=BOTH; n - t r
a no
6. Enable logging for the archived redo log destination configured in Step 6 by setting the
h a s
LOG_ARCHIVE_DEST_STATE_n parameter, where n matches the value used for the
m ) ideฺ
LOG_ARCHIVE_DEST_n parameter specified in previous step:
g ฺ co t Gu
n k n en
- ALTER SYSTEM SET LOG_ARCHIVE_DEST_STATE_<n> = 'ENABLE'
SCOPE=BOTH;
n b a
S t ud
io Recovery
7. Set the redo transport user tonthe
u h is Appliance user that was created for this
@ use t
protected database <protected_database_name>:
l l o
- ALTER SYSTEM
a k be eSETtoREDO_TRANSPORT_USER = <vpc_ra_user>
l o (a cens
SCOPE=BOTH;
eldown theliprotected database and restart it.
8. Shut
b
m - SHUTDOWN IMMEDIATE;
a kee - STARTUP;
Ensure you take a backup after completing REDO Transport operation to ensure all Logs are
collected.
During the load the Recovery Appliance tries to maintain a lag of less than a second for the
unprotected data window.

Agenda

Databases
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Enrolling Protected Databases
Registration
Steps Using Enterprise Manager Cloud Control
1. Create an EM Administrator to manage the Protected Database.
2. Enroll the Protected Database.
a ble
fe r
an s
n - t r
a no
h a s
m ฺ
) ide•Reserved
ฺ c o Gu Space
k n g nt •Recovery Appliance User
n b an tude •Credential Access Grantee
u nio this S
l l o @ use
be e to
a k
l o (a cens
Enrolling
b l
eProtected li
Databases with the Recovery Appliance - Cloud Control
k e emeach database, execute the following steps:

For
a 1. Create an Enterprise Manager Administrator to manage protected database operations.
You learned how to do this in lesson 5.
2. Enroll a Protected Database with the Recovery Appliance using Cloud Control:
a) Access the Protected Databases page.
b) Click Add. The Add Protected Databases page appears.
c) Click Add. The Select Targets page appears.
d) In Target Type, select Database Instance.
e) Optionally, narrow the database instances by entering values in the Target Name
and On Host fields.
f) In the table of targets, click the desired databases while pressing the Ctrl key.

g) Click Select.
h) In the Protection Policy section, click the policy to which you want to add the
databases, and then click Next.
i) Set the required attributes of the protected database:
j) In the Reserved Space field, enter the minimum amount of disk space to be
reserved for each protected database. When you add a database to a Recovery
Appliance using Cloud Control, the Recovery Appliance allocates a default reserved
space of 2.5x the database size. You can accept or change this amount.
k) In the Recovery Appliance User section, enter the credentials for the appropriate
virtual private catalog account.
l) In the Credential Access Grantee section, in Enterprise Manager Users, select the
Enterprise Manager user accounts that need access to the Recovery Appliance user
credentials.
a ble
m) Click OK.
fe r
n) Click Close to return to the Protected Databases page. ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

Enrolling Protected Databases
Registration
Steps Using the CLI

1. Install the Recovery Appliance Backup Module.
2. Add Protected Databases to the Recovery Appliance.

3. Grant the privileges required for performing backup and
recovery operations to the Recovery Appliance user that the
protected database will use for authentication.
4. Register the protected database with the Recovery Appliance ble
catalog. fe ra
s n
n - tra
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
Enrolling
b ethe li Databases with Recovery Appliance - CLI
l Protected
k e emeach Database execute the following steps:
For
a 1. Install the Recovery Appliance Backup Module. Follow the steps explained previously in
this lesson.
2. Only for Oracle Database 10g: Enrolling these protected databases requires alternate
manual configuration steps for the first part of the enrolment process.
a. Add a connect descriptor for the Recovery Appliance to the tnsnames.ora file.
Sample:
ZDLRA=
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = scaz15ingest-scan1)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = zdlra9)
)
)

b. Create a directory for an Oracle wallet in the $ORACLE_HOME/dbs directory.
c. Create a wallet in the subdirectory you created in the previous step.
For example:
$ /u01/app/oracle/product/10.2.0/db_1/bin/mkstore
-wrl /u01/app/oracle/product/10.2.0/db_1/dbs/ra_wallet/ -create
d. Add the credentials for the Recovery Appliance user (virtual private catalog user) to
the wallet.
For example :
$ /u01/app/oracle/product/10.2.0/db_1/bin/mkstore -wrl
/u01/app/oracle/product/10.2.0/db_1/dbs/ra_wallet/ -
createCredential "zdlra9" "rauser10" "welcome1"
e. Ensure that the sqlnet.ora file contains the location of the Oracle wallet.
For example : a ble
fe r
$ cat /u01/app/oracle/product/10.2.0/db_1/network/admin/sqlnet.ora
ans
# sqlnet.ora Network Configuration File:
n - t r
a no
/u01/app/oracle/product/10.2.0/db_1/network/admin/sqlnet.ora
h a s
# Generated by Oracle configuration tools.
m ) ideฺ
ฺco t Gu
NAMES.DIRECTORY_PATH= (TNSNAMES, EZCONNECT)
SQLNET.WALLET_OVERRIDE = true ng
a n k den
WALLET_LOCATION =
i o nb s Stu
(SOURCE =
u n t h i
(METHODll=o@
FILE) s
u e
k b e t o
( a a nse
(METHOD_DATA =
b lice
o (DIRECTORY
ell/u01/app/oracle/product/10.2.0/db_1/dbs/ra_wallet)
=
ee m )
ak )
f. Copy the libra.so file from the ORACLE_HOME/lib directory of the Recovery
Appliance to the ORACLE_HOME/lib directory of the protected database.

3. Registering a Protected Database with the Recovery Appliance Catalog.
a. Connect to the protected database as TARGET and to the Recovery Appliance
catalog as CATALOG.
b. Register the protected database using the REGISTER DATABASE command
Sample:
REGISTER DATABASE;
database registered in recovery catalog

starting full resync of recover catalog
full resync complete
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

Configuring Protected Databases
Backup Settings
Steps Using Enterprise Manager Cloud Control:

1. Access the Protected Database Home Page.
2. In the Availability menu, select Backup & Recovery, and then

select Backup Settings.
• Recovery Appliance
• Recovery Appliance User
• Enable Real-Time Redo
Transport
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
Configuring
b li Databases Backup Settings - Cloud Control
el the Protected
For
a 1. Access the home page for the protected database.
2. From the Availability menu, select Backup & Recovery, and then select Backup
Settings. The Backup Settings page for the protected database is displayed.
3. In the Device tab, configure the following optional settings:
a) To write redo data asynchronously from the protected database to the Recovery
Appliance, in the Recovery Appliance Settings section, select Enable Real-Time
Redo Transport.
b) If you want to configure backup polling for the protected database, then specify the
polling location in the Disk Backup Location setting of the Disk Settings section.
4. In the Maximum Backup Piece (File) Size field of the Backup Set tab, specify the
maximum size of each backup set that RMAN creates.

5. In the Policy tab, configure settings that define the objects that must be backed up.
Although it is not mandatory to configure the settings in this section, it is strongly
recommended that you configure automatic backups for the control file and server
parameter file.
6. Click Apply to save the backup settings.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

Configuring Protected Databases
Backup Settings
Steps Using the CLI

1. Connect to the TARGET Protected Database using RMAN.
2. Use the CONFIGURE command to configure the settings.

3. (Optional) Configure Real-Time Redo Transport.
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
Configuring
b li Database Settings - CLI
el the Protected
For
a 1. Use RMAN to connect to the protected database as TARGET.
Sample local connection:
rman target /
2. Use the CONFIGURE command to configure the required backup settings.
Sample command:
CONFIGURE CHANNEL DEVICE TYPE 'SBT_TAPE'
PARMS
'SBT_LIBRARY=/u01/app/oracle/product/11.2.0.4.0/dbhome_1/lib/libra.s
o,
ENV=(RA_WALLET=location=file:/u01/app/oracle/product/11.2.0.4.0/dbho
me_1/dbs/zdlra
credential_alias=ra-scan:1521/zdlra5:dedicated)' FORMAT '%U_%d';
3. (Optional) Configure Real-Time Redo Transport. Follow the steps described in previous
slides in this lesson.

Agenda

Databases
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Backing Up Protected Databases
Task
• Steps Using Cloud Control

Oracle Suggested
Backup
Include Archived Logs

(optionally delete them)
a ble
fe r
an s
n - t r
a no
Edit RMAN script h
a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
After you l l o li protected database, you can create and schedule protected database
econfigure the
b
em The Recovery Appliance uses the incremental-forever backup strategy for protected
ebackups.
ak database backups. In this strategy, an initial level 0 incremental backup is followed by
successive level 1 incremental backups. Except when using real-time redo transport, include
archived redo log files in all backups to ensure that you can perform complete recovery for the
protected database.
If the protected database is running in NOARCHIVELOG mode, then you must perform
consistent backups which requires shutting down the protected database.

Backing Up the Protected Databases - Cloud Control
Oracle-Suggested Recovery Appliance Backup Strategy:
1. Access the home page for the protected database.
2. Ensure that the configuration steps for the Enrolling and Configuring tasks are completed.
3. From the Availability menu, select Backup & Recovery, and then Schedule Backup.
4. In the Oracle-Suggested Recovery Appliance Backup section, select Schedule Oracle-
Suggested Backup.
5. Specify the method used to back up redo data.
- If real-time redo transport is not configured for the protected database, select Also
back up all archived logs on disk.
- If required, select Delete all archive logs from disk after they are successfully
backed up.
6. Click Next to display the Schedule Oracle-Suggested Recovery Appliance Backup: a ble
fe r
Schedule page.
ans
7. Schedule the backup job.
n - t r
o
an
8. Click Next to display the Schedule Oracle-Suggested Recovery Appliance Backup:
s
Review page.
) ha eฺ
9. Click Submit Job.
ฺ c om Guid
10. Click View Job to display the status of the
k g job.
nbackup n t
a n ud e
n b S t
u nio this
l l o @ use
a k be e to
l o (a cens
b el li
m
a kee

Backing Up Protected Databases
Task
• Steps Using the CLI

Recovery Appliance
Backup Module
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
After you l l o li protected database, you can create and schedule protected database
econfigure the
b
em The Recovery Appliance uses the incremental-forever backup strategy for protected
ebackups.
ak database backups. In this strategy, an initial level 0 incremental backup is followed by
successive level 1 incremental backups. Except when using real-time redo transport, include
archived redo log files in all backups to ensure that you can perform complete recovery for the
protected database.
If the protected database is running in NOARCHIVELOG mode, then you must perform
consistent backups which requires shutting down the protected database.
Backing Up the Protected Databases - CLI
To create a full backup of the whole protected database:
1. Connect RMAN to the protected database as TARGET and the Recovery Appliance
catalog as CATALOG.
2. Ensure that the configuration steps for the Enrolling and Configuring tasks are
completed.

3. Run the following command to allocate three SBT channels for the Recovery Appliance
and then create a full backup of the protected database including archived redo log files.
The BACKUP ... INCREMENTAL LEVEL 1 command automatically creates a level 0
backup if no level 0 backup already exists.
Sample:
RUN
{
ALLOCATE CHANNEL c1 DEVICE TYPE sbt_tape
PARMS='SBT_LIBRARY=/u01/oracle/product/11.2.0.4/dbhome_1/lib/libra.so,
ENV=(RA_WALLET=location=file:/u01/oracle/product/11.2.0.4/dbhome_1/dbs/ra
credential_alias=ra-scan:1521/zdlra5:dedicated)'
FORMAT'%U_%d';
a b le
PARMS='SBT_LIBRARY=/u01/oracle/product/11.2.0.4/dbhome_1/lib/libra.so, fe r
n s
n - tra
a no
FORMAT'%U_%d';
h a s
m ) ideฺ
g ฺ co t Gu
PARMS='SBT_LIBRARY=/u01/oracle/product/11.2.0.4/dbhome_1/lib/libra.so,
a n kn den
i o nb s Stu
u n t h i
FORMAT'%U_%d';
l lo@ o us e
BACKUP DEVICE TYPEesbt
a a kb se t
l o ( cen
TAG 'db_full_incr'
b e l
CUMULATIVE li
INCREMENTAL LEVEL 1
m
kee
DATABASE FORMAT '%d_%U'
a PLUS ARCHIVELOG FORMAT '%d_%U' NOT BACKED UP;
}

To create and schedule a level 1 incremental backup that includes archived redo log files:
1. Connect RMAN to the protected database as TARGET and the Recovery Appliance catalog
as CATALOG.
2. Ensure that the configuration steps for the Enrolling and Configuring tasks are completed
3. Ensure that at least one RMAN SBT channel that corresponds to the Recovery Appliance
is configured
4. Open a text editor and create and save a file with the following contents.
Sample:
export ORACLE_HOME=/u01/app/oracle/product/11.2.0.4/dbhome_1
export ORACLE_SID=db1124sm
export PATH=$PATH:$HOME/bin:$ORACLE_HOME/bin:$ORACLE_HOME/Opatch
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$ORACLE_HOME/rdbms/lib:/lib:/usr/lib;
a ble
export LD_LIBRARY_PATH
fe r
export LOG_TRACE_DIR=$HOME/RA_TEST/RMAN_SCRIPTS/LOG
ans
dt=`date +%y%m%d%H%M%S`
n - t r
a no
$ORACLE_HOME/bin/rman log=$LOG_TRACE_DIR/rman_bkincr_log_db1124sm_$dt.log
<<EOF
h a s
m ) ideฺ
co t Gu
CONNECT TARGET /
g ฺ
a n kn den
CONNECT CATALOG rauser/welcome1@ra-scan:1521/zdlra5:dedicated
i o nb s Stu
un e thi
BACKUP DEVICE TYPE SBT
TAG 'db_full_incr' @
b e o us 1
llo tLEVEL
ak %d_%U'
CUMULATIVE INCREMENTAL
a
DATABASE (FORMAT e n se
b
PLUS
llo lic FORMAT '%d_%U' NOT BACKED UP;
eARCHIVELOG
em
keEOF
a 5. Log in to the protected database host as a user who is a member of the OSBACKUPDBA
operating system group.
6. Open a text editor, create a file with the following contents, and save the file using the
name.crontab into your home directory. This example uses the crontab utility to
schedule the RMAN script.
Sample:
MAILTO=first.last@example.com
# MI HH DD MM DAY CMD
00 1 * * * /u01/app/oracle/product/11.2.0.4/db_home1/db_incr_daily.sh
7. In a command window, change directory to your home directory and enter the following
command to create a crontab file for this user from the contents of .crontab.
Sample:
# crontab .crontab

Monitoring Protected Databases
Job

a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Cloud b l l o
e provides
Control li reporting capabilities and also enables you to monitor and change the
k e em of protected database backup and recovery jobs.
status
a Cloud Control provides a Backup Report that displays the details of all backup and recovery
jobs that were run for a particular protected database. You can filter the data displayed in this
report depending on the time period for which you want to see the reports.
Monitoring the Protected Databases Backup - Cloud Control
To display the Backup Report for a protected database:
1. Access the home page for the protected database
2. From the Availability menu, select Backup & Recovery, then select Backup Reports.
3. Filter the data displayed in the backup report using the fields in the Search section. You
can filter by job status, type of backup job, or start time.
4. Click Go to display the backup report.
5. To view the details of a particular job execution, select the job and click View Results.

Monitoring Protected Databases
Job
• Steps Using CLI

a ble
Monitor backup
fe r
job progress
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Monitor thel l o
e progressliof backups, copies, and restores by querying the V$SESSION_LONGOPS
b
k e em RMAN uses detail and aggregate rows in V$SESSION_LONGOPS. Detail rows describe
view.
a the files that are being processed by one job step. Aggregate rows describe the files that are
processed by all job steps in an RMAN command. A job step is the creation or restoration of
one backup set or datafile copy. The detail rows are updated with every buffer that is read or
written during the backup step, so their granularity of update is small. The aggregate rows are
updated when each job step is completed, so their granularity of update is large.
Monitoring the Protected Databases Backups - CLI
Note: Set the STATISTICS_LEVEL parameter to TYPICAL (the default value) or ALL to
populate the V$SESSION_LONGOPS view.
The relevant columns in V$SESSION_LONGOPS for RMAN include:
• OPNAME: A text description of the row. Detail rows include RMAN:datafile copy,
RMAN:full datafile backup, and RMAN:full datafile restore.

• CONTEXT: For backup output rows, the value of this column is 2. For all the other rows
except proxy copy (which does not update this column), the value is 1.
Sample:
SQL> SELECT SID,SERIAL#,CONTEXT,SOFAR,TOTALWORK,
2 ROUND(SOFAR/TOTALWORK*100,2) "%_COMPLETE"
3 FROM V$SESSION_LONGOPS
4 WHERE OPNAME LIKE 'RMAN%'

5 AND OPNAME NOT LIKE '%aggregate%'
6 AND TOTALWORK != 0
7 AND SOFAR <> TOTALWORK;
SID SERIAL# CONTEXT SOFAR TOTALWORK %_COMPLETE

--- ------- ------- ------- --------- ----------
a ble
13 75 1 9470 15360 61.65
fe r
12 81 1 15871 28160 56.36
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Protected Databases
Compression and Encryption
• Compression
– No RMAN compression is recommended. Use Database
compression (ACO) instead

• Encryption
– No RMAN encryption is recommended. Use Database TDE
(ASO) instead.
a ble
fe r
Data in the Database
No RMAN Encryption and RMAN RMAN RMAN Encryption and
an s
No RMAN Compression Encryption Compression RMAN Compression
n - t r
Not Encrypted YES NO
a no YES NO
TDE Tablespace
YES a
YES
h s NO NO
Encryption
m ) ideฺ
g ฺ co t Gu
n kn dForever
Support for Incremental
a e n
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l
Compression
l o li
b
k e em recommends that you enable Compression on tables and indexes in the database,
Oracle
a and then take incremental backups as usual. Table/Index or Tablespace compression
requires the Advanced Compression Option.
Encryption
Oracle recommends that you enable TDE on table spaces in the database, and then take
incremental backups as usual. TDE requires the Advanced Security Option. The benefits of
TDE are as follows:
• TDE is transparent to applications.
• Backups of encrypted table spaces, and redo describing changes to these table spaces,
are encrypted. The TDE-encrypted data blocks are secured on the protected database,
Recovery Appliance storage, tape devices, and replicated appliances, and also when
transferred through any network connections.
• TDE on the source database reduces overhead on downstream servers.
• This technique supports an incremental-forever strategy and virtual full backups.

Protected Databases
Long-Term Backups
• Why make long-term backups?

– Higher backup and recovery retention policy
– Business requirement: standards such as HIPPA

– Auditing and security reasons
• Solution:
– Create a KEEP database backup for long-term retention
– Create a SBT job template that will be used to move the tagged le
backups r a b
sfe
– Move the backup from the Recovery Appliance storage a n
tr(such
n -
no
as Delta Store) to Tape using the SBT job template
h a sa
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Backups
Long-Term li
b
k e emcustomers requiring a much higher backup and recovery retention policy when using the
For
a Recovery Appliance, please follow this solution. Longer backup and recovery retention may
be mandated by a specific business requirement, for governmental or financial regulatory
standards or for standards such as HIPPA. Regardless of the reason, setting a very large
recovery window goal (such as one year) may not be cost effective for any protected
database; so, this solution describes how to KEEP certain TAGGED backups beyond the
Recovery Appliance's recovery window goals to meet higher backup and recovery retention
requirements.
Implementing a long-term backup solution consists of the following steps:
1. Create a KEEP database backup for long-term retention: The recovery appliance
handles this type of requirement via the use of RMAN KEEP Backups. An RMAN keep
backup will backup all datafiles, as well as the control files / spfiles and archivelogs and
these are given a specific TAG. The RMAN run block looks like this:
backup device type sbt tag 'KEEP_DEC2015' database keep until time
‘to_date(sysdate+1830)';

2. Create an SBT job template that will be used to move the tagged backups: Once the
RMAN backup has finished, the backupsets will be residing in the Delta Store. These
backupsets can be moved to tape using the API dbms_ra.move_backup. Before using
the move_backup API, an SBT Job Template must be created that identifies an
attribute set that is configured to go to a specific tape pool (aka media family) for the
specific database.
exec dbms_ra.create_sbt_job_template(template_name=>'KEEP',
db_unique_name=>'<dbname>', attribute_set_name=>'KEEP', backup_type=>'ALL');

3. Move the backup from Recovery Appliance storage (such as Delta Store) to Tape by
using the SBT job template: With the SBT Job Template defined, the backups can be
moved to this template using the move_backup API
exec dbms_ra.move_backup(tag=>'KEEP_DEC2015', format=>'KEEP_%U',
template_name=>’KEEP');
Considerations in a Replicated Environment
a ble
e
If the tape library is installed on the downstream recovery appliance, then after the above
f r
procedure has been completed a KEEP backup still resides on the UpStream Recovery ans
n - t r
Appliance. If it is desirable to purge the backup pieces from the UpStream RA while keeping
no
the KEEP backup in the UpStream RMAN Catalog, the following steps can be performed
a
Using SQL*Plus connect to the Zero Data Loss Recovery Applianceh a s as the VPC User who
owns the database: m ) ideฺ
g ฺ co t Gu from
SQL> select 'delete noprompt backuppiece
rc_backup_piece_details where db_name
a n k=n'<dbname>'
d e n and tag = 'KEEP_DEC2015'
'||bp_key||';'
and BA_ACCESS = 'Local';

i o nb s Stu
u n t h i
l lo@ o us e
e
'DELETENOPROMPTBACKUPPIECE'||BP_KEY||';'
kb se t
a a
---------------------------------------------------------------------
l l o l i c en
( backuppiece
b e
delete noprompt 21711952;
e e m noprompt backuppiece 21711928;

delete
a k delete noprompt backuppiece 21711883;

delete noprompt backuppiece 21711970;
Now connect to the target database and the Zero Data Loss Recovery Appliance catalog
using RMAN and run the above commands.

Migration
Planning Migration
• Adapt the existing backup strategy for the Recovery Appliance
– Minor modifications - reuse RMAN scripts
— Replace Disk/Tape Channels by SBT Channels

— Modify or remove RMAN commands not supported
— Remove RMAN commands not required
• Migrate backup metadata to the Recovery Appliance catalog
– Import Metadata
• Migrate existing protected database backups to the Recovery Appliance
– Configure a backup polling location and poll them
– Back up local image copies to Recovery Appliance as backup sets
a ble
fe r
RMAN Scripts
an s
n - t r
a no
h a s
RMAN Recovery
m ) ideฺ
Catalog
g ฺ co t Gu
a n kn den
RMAN Local Recovery
b t u
n i on is S
COPIES Appliance
@ u e th
( a ak nse
liceinvolving Recovery Manager (RMAN) consists of a set of backup
ello strategy
A basicbbackup
m that are scheduled to run at specified intervals. When you decide to use the Recovery
kee
scripts
a Appliance for data protection, you need to develop a plan to migrate your existing backup
strategy to one that uses the Recovery Appliance.
Your existing RMAN backup strategy typically involves storing backup metadata in an RMAN
recovery catalog. When migrating to a data protection strategy that uses the Recovery
Appliance, you need a strategy to manage existing backup metadata that is stored in an
RMAN recovery catalog.
Your existing backup strategy may store protected database backups in a local disk location
or on a shared disk. After you import the metadata for the protected database backups into
the Recovery Appliance catalog, you must migrate existing backups that are within the
recovery window goals to the Recovery Appliance storage.

Migration
RMAN Differences
• Changes and limitations to standard RMAN commands

RMAN Command Notes
CHANGE BACKUP … UNAVAILABLE DO NOT USE with Virtual Full Backups

CHANGE BACKUP ... UNCATALOG DO NOT USE with Virtual Full Backups
If RETENTION POLICY is set to NONE recommended use FRA for local
CONFIGURE ... RETENTION POLICY
archived logs
a ble
DELETE DO NOT USE
fe r
an s
DROP DATABASE
Use the DBMS_RA.DELETE_DB procedure to unregister the protected database
from the Recovery Appliance and then use the DROP DATABASE command.
n - t r
MAXPIECESIZE clause NOT SUPPORTED a no
h a s
Use the DBMS_RA.DELETE_DB procedure, ) instead eofฺ UNREGISTER
UNREGISTER DATABASE
c m i d
o Gu the Recovery Appliance.
DATABASE, to unregister a protected database from
ฺ
ng ent
n k
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
The table
b l
eabove li
describes the changes and limitations to standard RMAN commands when
k e em the Recovery Appliance.

using
a

Migration
RMAN Channels
Types
• DISK: Stored on disk. Not used for the Recovery Appliance
• SBT: Stored on tape, Oracle Cloud and the Recovery Appliance

CONFIGURE CHANNEL DEVICE TYPE ‘SBT_TAPE'
PARMS 'SBT_LIBRARY=/u01/app/oracle/product/11.2.0.4.0/dbhome_1/lib/libra.so,
ENV=(RA_WALLET=location=file:/u01/app/oracle/product/11.2.0.4.0/dbhome_1/dbs/zdlra
Sample Allocate Sample Configure

a ble
Channel Channel
fe r
an s
RUN
n - t r
{
a no
a s
PARMS='SBT_LIBRARY=/u01/app/oracle/product/12.1.0.2/dbhome_1/lib/libra.so,
h
) ideฺ
ENV=(RA_WALLET=location=file:/u01/app/oracle/product/12.1.0.2/dbhome_1/dbs
m
co t Gu
g ฺ
BACKUP INCREMENTAL LEVEL 1 DATABASE PLUS ARCHIVELOG;
}
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
An RMAN l l o li
e channel represents one stream of data to or from a backup device. The channel
b
em data from the input device, processes it, and then writes it to the output device. RMAN
ereads
ak supports the following types of channel configurations:
• DISK: backups are stored on disk.
• SBT: backups are stored on one of the following: tape using media management
software such as Oracle Secure Backup, the Recovery Appliance using the Recovery
Appliance backup module, or on the Oracle Cloud.
To back up to or restore from a Recovery Appliance, you must configure and use an SBT
channel that corresponds to the Recovery Appliance backup module installed on the
protected database host. In a Recovery Appliance environment, your backup and recovery
scripts must allocate an SBT channel with the SBT_LIBRARY parameter pointing to the
Recovery Appliance backup module.

Configure
You configure RMAN SBT channels for the Recovery Appliance by using the RMAN
CONFIGURE command. Configuring channels for a protected database creates persistent
settings that are applicable to all backup, restore, and maintenance operations on that
protected database.
Allocate
Use the RMAN ALLOCATE command to allocate RMAN SBT channels that will be used to
back up to or recover from the Recovery Appliance. For a particular operation, you can
override the persistent configuration that was set using the CONFIGURE command by
explicitly allocating an RMAN SBT channel before the operation. Enclose the ALLOCATE
command and the other commands in a RUN block.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Migration
Import Backup Metadata
Steps
1. Connect to the Protected Database and the Recovery
Appliance by Using the CLI.

2. Import Protected Database Metadata into the Recovery
Appliance Catalog.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l o
e RMANlibackup strategy typically involves storing backup metadata in an RMAN
l
Your existing
b
em catalog. When migrating to a data protection strategy that uses the Recovery
erecovery
ak Appliance, you need a strategy to manage existing backup metadata that is stored in an
RMAN recovery catalog.
Connect
To connect to a protected database (non-CDB) and Recovery Appliance:
RMAN> CONNECT TARGET sys as sysdba;
RMAN> CONNECT CATALOG ra_rman_user@ra1;
To connect to a protected database (CDB) and Recovery Appliance:
RMAN> CONNECT TARGET c##bkuser@my_cdb;
RMAN> CONNECT CATALOG ra_rman_user@ra1;

Import
1. If the source RMAN recovery catalog is not an contained in an Oracle 12c Release 1
database, then upgrade the recovery catalog database to Oracle 12c Release 1.
2. Install the Recovery Appliance backup module that creates the shared library required to
transfer backup data to the Recovery Appliance.
3. Connect as TARGET to the protected database and as CATALOG to the RMAN recovery
catalog that stores metadata for the protected database.

rman target / catalog rman_cat11/rmancat11_pswd@dbrcat11
4. Ensure that no backups from the protected database are being created to the RMAN
recovery catalog.
RMAN> SELECT username, module
FROM v$session
WHERE username = ‘RMAN_CAT11';
r a ble
5. Determine the number of backup pieces contained in the catalog for the protected
n s fe
database.
n - tra
SELECT db_name, COUNT(*)
a no
FROM rc_backup_piece_details
h a s
WHERE db_name=‘MY_PTDB';
m ) ideฺ
6. Verify that the backups in the recovery catalog g ฺ careo valid
t G u can be used for a successful
and
recovery operation. Use RESTORE … n
a kn detonverify them.
VALIDATE
Connect to SQL*Plus as the owner of o nbsource
the S tu recovery catalog and run the
RMAN
dbmsrmansys.sql script. This u
i
n grants h s
iadditional privileges that are required for the
script
e t
RECOVERY_CATALOG_OWNER
e l lo@ o role.
us
b
ak nse t
# sqlplus rman_cat11/rmancat11_pswd@dbrcat11
( a
l lo l i ce
SQL> $ORACLE_HOME/rdbms/admin/dbmsrmansys.sql
e
m b
SQL> exit
e
ake 7. On the Recovery Appliance, start an RMAN session and connect to the Recovery
Appliance as TARGET using the RASYS user and to the source RMAN recovery catalog
as CATALOG.
# rman target rasys/rasys_pswd
RMAN> CONNECT CATALOG rman_cat11/rmancat11_pswd@dbrcat11

8. Upgrade the source RMAN recovery catalog to Oracle Database 12c Release 1
(12.1.0.2). The UPGRADE CATALOG command needs to be entered twice for
confirmation.
UPGRADE CATALOG;
UPGRADE CATALOG;
9. Repeat Steps 4 through 6 on the upgraded source RMAN recovery catalog to verify that
the upgraded catalog is fine and can be used to recover the protected database.
10. Start RMAN and connect as CATALOG using the rasys user. Note that rasys is the
owner of the Recovery Appliance catalog.
# rman CATALOG rasys/ra_pswd@ra-scan:1521/zdlra5
11. Import the source RMAN recovery catalog into the Recovery Appliance catalog. The
credentials of the source RMAN recovery catalog are provided by the protected database
administrator.
a ble
IMPORT CATALOG rman_cat11/rmancat11_pswd@dbrcat11 NO UNREGISTER;
fe r
IMPORT CATALOG rman_cat11/rmancat11_pswd@dbrcat11
ans
n - t r
DB_NAME 'MY_PTDB' NO UNREGISTER;
o
an
12. Verify that all the backup pieces are included in the Recovery Appliance catalog by
s
querying the RC_BACKUP_PIECE_DETAILS view.
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Migration
Import Existing Backups
• Prerequisites
– Migrate existing level 0 incremental backup
– Migrate level 1 incremental backups

– Migrate archived log files
• Techniques
– Backup Polling Location
– Local Image Copies as RA Backup Sets
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l o
e backupli strategy may store protected database backups in a local disk location
l
Your existing
b
ak eorem
on a shared disk. After you import the metadata for the protected database backups into
the Recovery Appliance catalog you must migrate existing backups that are within the
recovery window goals to the Recovery Appliance storage. You can migrate backups for only
a subset of the protected databases contained in the imported RMAN recovery catalog.
However, for all the databases represented in the imported catalog, you can begin using the
Recovery Appliance catalog as your recovery catalog.
To begin using an incremental-forever backup strategy with Recovery Appliance, you must
first submit a level 0 incremental backup. If a recent level 0 incremental backup already exists
for a particular protected database, it might be more convenient to migrate that backup into
the Recovery Appliance, rather than take another level 0 backup from the database. After
migrating the level 0 backup and any required existing level 1 backups and archived log files,
you can then begin the incremental-forever strategy by sending level 1 incremental backups
and archived log files.

The recommended strategy is to migrate all existing backups and switch immediately to the
Recovery Appliance. Any backups created after you migrate existing backups must be stored on
Backup Polling Location
1. Create a polling policy corresponding to the polling location that contains the protected
database backups to be migrated.
2. On the Recovery Appliance, use the DBMS_RA.UPDATE_PROTECTION_POLICY

procedure to assign the polling policy created in Step 1 to the protection policy that is
associated with the protected database.
BEGIN
DBMS_RA.CREATE_POLLING_POLICY (
polling_policy_name => 'MIGRATION_LINUX',
polling_location => '/polling/shared_backup_location',
a ble
polling_frequency => INTERVAL '1' MINUTE,
fe r
ans
delete_input => FALSE);
n - t r
END;
a no
h a s
3. Mount the polling location directory on the Recovery Appliance database nodes. (you
have learnt how to do this previously)
m ) ideฺ
g ฺ co t Gu
4. Verify that the backup sets have been imported by querying the ra_task view.
a n kn den
Backup Copies as Backup Sets ion
b S tu
u n t h i s
Back up image copies that are
l l o @ us
stored e
on local disk storage as backup sets to the Recovery
k b e
Appliance using the RMAN
t o
BACKUP AS BACKUPSET COPY OF DATABASE command. You
must configure anaSBT channel
( a e n se that corresponds to the Recovery Appliance backup module.
b ello lic
k e em
a

Agenda

Databases
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Recovering Protected Databases
Task

a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Cloud b l l o
e provides
Control li the following techniques to recover protected databases:
k e e•m Oracle Advised Recovery: Oracle Advised Recovery enables you to recover the
a protected database by using the automatic repair actions recommended by Data
Recovery Advisor. The Data Recovery Advisor automatically diagnoses data failures,
assesses their impact, reports these failures to the user, determines appropriate repair
options, and executes repairs at the user's request.
• User Directed Recovery: This technique performs manual recovery based on the
specified criteria. You must provide information such as the objects that must be
recovered (database, datafiles, tablespaces, archived redo logs), whether to perform
complete recovery or point-in-time recovery, location to which database files must be
recovered, and so on.

Prerequisites
• The protected database must be enrolled and registered with the target Recovery
Appliance.
• Backups required to restore and recover the protected database must be stored on the
Recovery Appliance.
Entire Database with Existing Control File
To perform complete recovery of the protected database:

1. Access the home page for the protected database
2. Ensure that the prerequisites are met.
3. From the Availability menu, select Backup & Recovery, and then select Perform
Recovery.
4. In the User Directed Recovery section, select the following options:
5. In Recovery Scope, select Whole Database. a b le
fe r
6. In Operation Type, select Recover to the current time or a previous point-in-time.
a n s
7. Click Recover to display the Perform Whole Database Recovery: Point-in-timen -t r page.
n o
8. Select Recover to the current time and click Next.
h a sa
9. Select No. Restore the files to the default location and)click Next.
m d e ฺ
ฺ cothe Job i
u and Job Description.
10. (Optional) Modify the default names providedgfor
n eRecovery:
n t G Name
11. Click Next to display the Perform Whole
a n kDatabase
d Review page.
n b t u
S for this recovery job, click Edit RMAN
u n io generated
12. (Optional) To edit the RMAN script
h i s
Script. @ e t
13. Click Submit Job.
l o
el to u s
k b
a nse
14. Click View( aJob.
e l lo l i ce
m b
a kee

Recovering Protected Databases
Task
• Steps Using the CLI

a ble
fe r
an s
Recovery Appliance
n - t r
no
Backup Module
s a
h a
) ideฺ
m
co t Gu
g ฺ
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
When b l l o li Appliance for restore and recovery operations, the RMAN
e the Recovery
using
k e em
connection syntax used is the same as with a regular RMAN recovery catalog connection.
a The only difference is that you connect to the Recovery Appliance catalog and configure
RMAN channels.
Prerequisites
• The protected database must be enrolled and registered with the target Recovery
Appliance.
• Backups required to restore and recover the protected database must be stored on the
Recovery Appliance.
• The Oracle wallet containing credentials used to authenticate with the Recovery
Appliance must be configured on the protected database.
• Configure or allocate RMAN SBT channels that correspond to the Recovery Appliance
backup module.

Entire Database with Existing Control File
2. Use RMAN to connect to the protected database as TARGET and the Recovery
Appliance catalog as CATALOG.
3. Restore and recover all the datafiles using the following command:
STARTUP MOUNT;
RUN
{
RESTORE DATABASE;
RECOVER DATABASE;
ALTER DATABASE OPEN;
}
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Entire Database to a Specified Point-In-Time (PIT)
2. Use RMAN to connect to the protected database as TARGET and the Recovery
Appliance catalog as CATALOG.
3. Determine the point-in-time to which the protected database must be recovered. You
can use an SCN, a time, or log sequence number to specify the point-in-time.
- DB available - Protected Database

SQL> set linesize 222
SQL> select name, current_scn, scn_to_timestamp(current_scn) "Time"
from v$database;
NAME CURRENT_SCN TIME
--------- ----------- ------------------------------------
ORA121 122019556 22-APR-14 12.30.15.000000000 PM
a ble
fe r
- DB not available - Recovery Catalog views
ans
SELECT
n - t r
a no
a.db_key,a.db_name,a.sequence#,a.first_change#,a.next_change#,a.completion_
time
h a s
FROM rc_archived_log a, db db_key m ) ideฺ
g ฺ co t= G u
kn den HH24:MI:SS') BETWEEN
WHERE reg_db_unique_name = 'PTDB2' AND a.db_key db.db_key
a n
nb s Stu
AND to_date('16-Jul-2014 06:55:23','DD-Mon-YYYY
n i o
a.first_time AND a.next_time; i
u e t h
e l lo@ oFIRST_CHANGE#
us
DB_KEY DB_NAME
a k b
SEQUENCE#
e t NEXT_CHANGE# COMPLETION_TIME
------ --------
l o c ns ------------- ------------ ---------------
(a ----------
e
24201 be l
PTDB2 li 9911 288402086 288430116 14/07/2014 5:27:49 PM
m
a ke4.eRestore and recover the control file and the protected database.
STARTUP NOMOUNT;
RUN
{
SET UNTIL TIME "TO_DATE('2014-14-07:17:27:49','yyyy-dd-mm:hh24:mi:ss')";
RESTORE CONTROLFILE;
ALTER DATABASE MOUNT;
RESTORE DATABASE;
RECOVER DATABASE;
ALTER DATABASE OPEN RESETLOGS;
}

Summary

• Identify various configuration tasks
• Enroll a Protected Database into the Recovery Appliance

• Perform Backup and Recovery operations of Protected
Databases by using the Recovery Appliance
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

7
Optimizations and Performance Tuning a ble

fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake
Objectives

• Explain the Recovery Appliance internal optimization
• Describe and apply performance-tuning techniques and best

practices
• Use the rastat.pl script to monitor Recovery Appliance
statistics
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Agenda
• Optimization: Overview
• Performance Tuning Best Practices
• Rastat.pl and network_throughput_test.sh Scripts

• Oracle Enterprise Manager Performance Pages for the
Recovery Appliance
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Recovery Appliance Sizing
• Initial and ongoing capacity planning
5
Monitor
Use the Recovery
4 Discuss
Appliance Capacity
Planning Reports
3
Discuss the results;
tune and finalize
Define a ble
fe r
Define the size of the Recovery
s
2 Validate
Appliance based on the collected
information
a no n - t r an
1 s
Validate the collected data
h a
) ideฺ
Collect m
o Gu
cinformation
g
n entฺ
Collect existing database’s
n k
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
el
Methodologies
b li
k e e•m Comparative Sizing (vs. an existing system): Sizing estimate based on current
a utilization; capacity and throughput of existing backup appliance(s). It considers capacity
and throughput differences based on standardization of backups leveraging an
incremental forever strategy.
• Predictive Sizing (for a new deployment): Holistic approach to centralizing and
standardizing Oracle database backups. It starts with a “clean state” to architect a new
centralized backup management strategy.

Recovery Appliance Sizing
• Sizing estimates is a matter of doing the math: • Add InfiniBand-connected racks to scale
- Size and growth rate of database – Add a spine switch when connecting
- Minus temp, free space, committed undo an additional rack

- Percentage of database that changes each day
• Can expand with new-generation
- Redo generation rate
hardware
- Desired recovery window in number of days
- Compressibility of database • Scale to 18 racks
- Annual data growth rate – Up to 216TB/hour Delta Ingest and
- System lifecycle (three years) restore
Up to 120 TB/hour Virtual Backup Rate a ble

Up to 12 TB/hour Sustained Delta Ingest fe r
Up to 12 TB/hour Restore Rate an s
n - t r
a no
h a s
580TB Usable Delta Space/Rack
5.8PB of Virtual Backup Space m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Appliance
The Recovery li supports up to 5.8 PB of virtual full backups in a single full rack, and
b
k e em100 PB of virtual full backups in a maximum configuration of 18 full racks.
over
a If additional capacity is required, a second base rack can be connected via high-speed
InfiniBand to the first rack. The second rack includes its own pair of compute servers that add
connectivity and processing power to the configuration. As with the first rack, storage capacity
can be easily expanded by incrementally adding storage servers. Up to 18 fully configured
racks can be connected together into a single appliance, providing more than 10 PB of usable
capacity, i.e. more than 100 PB of virtual full backups.
A single rack Recovery Appliance can achieve a sustained Delta Ingest rate of up to 12
TB/hour. In other words, it can receive 12 TB/hour of change data, and convert it into
120TB/hour of virtual backups. It is able to support a restore rate of up to 12 TB/hour.
As racks are added to the configuration, both performance and capacity increase linearly. An
18-rack Recovery Appliance achieves virtual full backup rates of up to 2 PB/hour, and 216
TB/hour of Delta Ingest and restore.

Optimization
• Configuring the number of channels that the RMAN uses

– Increasing RMAN channels can improve performance, but it
also can be detrimental

• The Recovery Appliance has internal optimizations.
• The Recovery Appliance is not an Exadata machine, it’s an
APPLIANCE.
• We limit by the number of storage cells. le
r a b
– If you need better performance, add more storage cells. s fe
r a
t cann
• There is a limit to the number of parallel connections n -that
be created against the Recovery Appliance. a n
o
h a s
– If all channels for a database are not available,
) idethe ฺ backup will
wait in a queue until the channelsฺcare m
o released.
u
g t G
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e describes,
As thisbslide li optimization should focus on RMAN channel parallelism, the number
m cells, connection parallelism and the internal optimizations of the Recovery
eofestorage
ak Appliance.

Agenda

Recovery Appliance
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Optimization
Section Size
• Section size can be manipulated but there is a trade off:

– Smaller section sizes enable quicker indexing
• Section size does affect restore performance.

• To calculate the section size for large databases, use:
– Total database size/No. of datafiles = Average backup size per section
• RMAN channels work in parallel, but the virtual full backup needs to be
processed in a sequence.
Channel 1
Section 1 a ble
fe r
an s
Channel 2
n - t r
Section 2
a no
Channel 3 ) ha
s
Section 3 m d e ฺ
ฺ co t Gu i
g
n e4n
kChannel Multipiece
Section 4 a n d
i o nb s Stu backup set
u n
One large datafile
t h i
l lo@ o us e
e
kb se t
a a
( cen
The RMAN l l o li break up large files into sections, and back up and restore these
e can optionally
b
em independently. You can use this built-in RMAN feature with the Recovery Appliance
esections
ak to improve performance. You can use it to create multisection backups, which break up the
files generated for the backup set into separate files. This can be done for backup sets and
image copies.
Each file section contains a contiguous range of blocks of a file. Each file section can be
processed independently, either serially or in parallel. Backing up a file into separate sections
can improve the performance of the backup operation, and it allows large file backups to be
restarted.
A multisection backup job produces a multipiece backup set. Each piece contains one section
of the file. All sections of a multisection backup, except perhaps for the last section, are of the
same size. There is a maximum of 256 sections per file.

Optimization
Best Practices
• Initially, allocate two RMAN channels per database node.

– Do not over-allocate RMAN channels.
• Backup strategy:
– Take only one level 0 backup; ensure that indexing of this
backup has completed before taking another.
– Take a daily cumulative incremental level 1 to reduce database
recovery time.
• The most common bottlenecks are the network and the ra ble
s fe
protected database’s I/O system. r a n
o n -t
• Use block change tracking (BCT) for all protected
a n databases.
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
RMANbChannels li
k e emexample, if you have registered 15 protected databases, you should be able to view a
For
a maximum of 15 x 2 = 30 channels connected to the RAC database. Do not over-allocate
RMAN channels because doing so can result in degraded performance.
Backup Strategy
Use the following query to verify the level 0 backup has been indexed before you create
another level 0 backup:
select bp_key from rc_backup_piece where tag = ‘&tag' and
backup_type ='D' and virtual = 'NO';
Network Bottleneck
The total network throughput per database node is 80GbE/sec. Separate interface
assignments by activity (such as separate replication traffic from ingest traffic, if possible).

BCT
Use change tracking files for all protected databases. Without being BCT enabled, or in cases
when change tracking information cannot be used, RMAN has to read every block in the
datafile during the incremental backup. Each block contains the last system change number
(SCN) of its last modification. The block is copied only if its SCN is higher or equal to the base
backup SCN. To be precise, the backup SCN is a checkpoint SCN that was made right before
the backup.
Usually only a handful of blocks change between incremental backups, so RMAN does a lot of
unnecessary work reading the blocks that are not required for backup. The BCT provides a way
to identify the blocks required for backup without scanning the whole datafile. After that, RMAN
needs only to read blocks that are really required for the incremental backup.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Optimization
Best Practices
RMAN-compressed backups are not needed in Recovery

Appliance environments.
• The Recovery Appliance compresses backups.

• Incremental forever backup strategy reduces network traffic.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e production
Compressed li data remains compressed in the backup. So, this data will not benefit
b
k e emfurther compression during the backup (such as RMAN backup or Recovery Appliance
from
a compression). In addition to this, deduplication software cannot deduplicate compressed data.

Optimization
Best Practices
• Use Transparent Data Encryption (TDE) instead of RMAN

encryption.
• Implement TDE to encrypt datafiles in protected databases.

• Network encryption: Can use SSL (TCPS encrypted while in
transit).
• Currently this is a manual step.
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
TDE bel li
k e emis a feature that enables encryption of individual database columns before storing them
TDE
a in the datafile, or it enables encryption of entire tablespaces. If users attempt to circumvent
the database access control mechanisms by looking inside datafiles directly with operating
system tools, TDE prevents such users from viewing sensitive information.
Network Encryption
Review the following support notes to enable TCPS protocol with SCAN Listeners.
How To Configure Scan Listeners With A TCPS Port (Doc ID 1092753.1)
Step-by-Step Guide: How to Configure SSL / TCPS on ORACLE RAC (with SCAN)(Doc ID
1448841.1)

Optimization
Simple Backup
• A database backup script is typically simple.

$ rman target <target string> catalog <RA catalog string>

backup device type sbt
cumulative incremental level 1
filesperset 1
section size 32g
a ble
database
fe r
plus archivelog not backed up
an s
filesperset 32;
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
el
Keep itbsimple. li
k e em
a

Tasks
• Determine whether the unprocessed task list is growing (over

a seven-day period):
– select count(*) from ra_task where

archived=‘N’;
• Determine whether incidents are active for delays in
performing busywork:
– select error_text from ra_incident_log
a b le
where status=‘ACTIVE’ e r
a n sf
-tr
and error_text like '%has not run%';
on
n
h a sa
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
How tobdetermine i Recovery Appliance is keeping up with a load.
if lthe
ak eem

Recovery Appliance Users
• Import existing backup metadata into the Recovery Appliance

catalog.
• Upgrade the RMAN recovery catalog to Oracle Database

12.1.0.2.
• Use the RMAN IMPORT CATALOG command to import backup
metadata.
• Ensure backups/restores are not in progress during catalog
import.
r a ble
• Dual catalog approach: Keep the existing RMAN catalog and n fe
use
s
the Recovery Appliance catalog for backup metadata under
n - tra
appliance management. a no
a
• Automated catalog backs up daily to the Recovery
h s Appliance
m ) ideฺ
catalog disk group. co t Gu
g ฺ
kn todetape
• If tape drives are attached, backups
a n n are automated as
well. nb Stu
nio this
u
@ use
l l o
be e to
a k
l o (a cens
The Recovery
b li administrator creates database user accounts on the appliance.
el Appliance
em are called virtual private catalog (VPC) users.
eThese
ak These users have privileges to send and receive backups for one or more protected
databases.
Each Recovery Appliance user owns a VPC for accessing metadata in the catalog for only
those databases
The protected database’s administrator with SYSDBA or SYSBACKUP privilege is associated
with the Recovery Appliance user.
Authentication credentials of the Recovery Appliance user are stored securely in an Oracle
Wallet on the protected database host.
The database administrator connects to the Recovery Appliance VPC user using the catalog
role.

Migration
• Migrate existing protected databases to the Recovery

Appliance.
• Migrate the most recent RMAN level 0 backup and

subsequent level 1s as well as archived log backups.
• Migrate via the Recovery Appliance polling capability or by
using RMAN BACKUP AS BACKUPSET.
• Cut-over strategy: le
r a b
– Begin backup operations to the Recovery Appliance on a s fe
selected date. - t r an
n ondate; will
– Restores requiring backup pieces prior to the cut-off
a s apieces have not
use the previous backup location if the backup
h
yet been fully migrated to ZDLRA. om)
ฺ
ฺ ide
c t Gu
g
kn den
a n
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li
m be
e
ake

Protected Databases
• One protection policy can be associated with many

databases.
• Define one protection policy for each class of service.

– Disk recovery window goal and backup copy policy
– Optionally: Tape recovery window, unprotected data window
threshold, maximum disk retention, and backup polling settings
• If an unprotected data window threshold is defined: le
r a b
– Ensure that all databases associated with the policy have real- e
time shipping enabled or backup on the same frequency. a n sf
n - tr
– Replication is configured by the protection policy,nso o that all
databases associated with a protection policy
a
s should have the
h a
) ideฺ
same replication requirement. m
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Extreme l l o li should NEVER occur!
espace pressure
b
k e e•m This could only happen if the Recovery Appliance administrator IGNORES the
a numerous alerts and warnings.
The backup copy policy setting in the protection policy defines prior if extreme space pressure
occurs.
• Always accept new, incoming backups (default)
• The oldest backup within the recovery window could be purged to make room for
incoming backups.
• Refuse new, incoming backups if there is not enough space to meet a recovery window.
Backups within recovery window could only be purged on disk if they had been copied to tape
or replicated.
If this policy is enabled, backups on the Recovery Appliance will never have more than their
reserved space setting.

Protected Databases
• The RMAN backup destination to use depends on the configuration:

– The “SBT_TAPE” destination when backing up directly to the
Recovery Appliance and “DISK” when using polling settings

– Backup and restore operations must connect to the Recovery
Appliance catalog.
– Use RMAN CONNECT as TARGET for protected database and as
CATALOG for communication with the Recovery Appliance.
– Pluggable databases connect to the root of the multitenant container
database (CDB) as TARGET. r a ble
– Backup retention to disk and tape is configured via Recovery nsf
e
a
Appliance protection policies. n-tr o
• Do not use the RMAN “DELETE OBSOLETE” command.
s an
• ) ha eฺ
Protected database configuration recommendations:
m id
ฺ co t Gu
– Enable BCT for fast incremental backups.
g
a n
– Use a local Fast Recovery Area knfor archived
d e n logs, online redo, and
Flashback logs. ion
n
b Stu
is
u
@ use t h
l l o
be e to
a k
l o (a cens
Extreme
b espace li should NEVER occur!
l pressure
k e e•m Could only happen if the Recovery Appliance administrator IGNORES the numerous
a alerts and warning
The Backup Copy Policy setting in the Protection Policy defines prior IF extreme space
pressure occurs:
• Always accept new, incoming backups (default).
• The oldest backup within the recovery window could be purged to make room for
incoming backups.
• Refuse new, incoming backups if not enough space is available to meet the recovery
window.
Backups within the recovery window could only be purged on disk, if they’d been copied to
tape or replicated.
If this policy is enabled, backups on the Recovery Appliance will never have more than their
reserved space setting.

Agenda

Recovery Appliance
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

rastat.pl
The rastat.pl utility:

• Identifies system bottlenecks
– Network data sent/received

– ASM I/O subsystem
– Container files
• Is a perl script
• Runs one or multiple tests in parallel on one or more
r a ble
protected databases
n s fe
• Provides results where each test result representsoan
n -tra
individual client n
sa ha eฺ
)
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
The Recovery
b li ships with two utilities, rastat.pl and
el Appliance
em
enetwork_throughput_test.sh, that can assist you in evaluating the performance of your
ak system.
rastat.pl is a command line utility that runs tests against the Recovery Appliance to gather
performance statistics that can help you identify system bottlenecks.
The tests can generate statistics on:
• Backup data sent to the Recovery Appliance over the network
• Restore data received from the Recovery Appliance over the network
• Recovery Appliance ASM disk group read or write I/O
• Recovery Appliance container file read or write I/O
• Recovery Appliance container file allocation rate

Before you run the rastat utility, ensure that the following requirements are met:
• The platform on which you will be running rastat is either Linux or UNIX.
• If you will be running the utility from a protected database, copy the rastat.pl file from
the /opt/oracle.RecoveryAppliance/client/ directory of a Recovery Appliance
compute server to the protected database.
• Complete the steps to enroll the protected database with the Recovery Appliance.
• Ensure that the $ORACLE_HOME and $ORACLE_SID environment variables are

configured if you do not plan to set them by using the applicable options when you run the
utility.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

rastat.pl
• Examples
>$ORACLE_HOME/perl/bin/perl rastat.pl --test=NETBACKUP --filesize=2048M
--catalog=rman/rman@inst2 --parms='SBT_LIBRARY=/u01/oracle/lib/libra.so,
ENV=(RA_WALLET=location=file:/u01/oracle/dbs/ra_wallet
NETWORK TEST FROM PROTECTED DATABASE TO RECOVERY APPLIANCE
393 MB/s, 2048 MB sent >$ORACLE_HOME/perl/bin/perl rastat.pl --test=CONTAINERWRITE --filesize=2048M

--rasys=admin/admin@inst2 --diskgroup=/:BLOCK_POOL
RECOVERY APPLIANCE WRITE IO TEST TO CONTAINER FILES
a ble
Disk Group: /:BLOCK_POOL
fe r
>$ORACLE_HOME/perl/bin/perl rastat.pl --test=ASMREAD --filesize=2048M
an s
--rasys=admin/admin@inst2 --diskgroup=+RCVAREA
2048 MB, 9.55s write IO time, 3.50s CPU time, 214.35 MB/s, 36.60% CPU usage
n - t r
RECOVERY APPLIANCE READ IO TEST FROM DISK
PL/SQL procedure successfully completed.
a no
Disk Group: +RCVAREA
h a s
2048 MB, 6.06s read IO time, .65s CPU time, 337.99 MB/s, 10.79% CPU usage m
) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
To runb the l l o
e rastat.pl li utility:
ak ee1.m Ensure that the system from which you are running the utility meets the requirements.
2. Open a command prompt window.
3. Enter the applicable command syntax for the tests you want to run, and press Enter.

network_throughput_test.sh
The network_throughput_test.sh utility:

• Identifies network bottlenecks
• Is a shell script
• Uses the QPERF utility part of the Exadata Engineered
System
• Sets up trusted SSH connectivity between the execution node
and send/receive nodes le
r a b
s fe
- t r an
no n
s a
h a
) ideฺ
$ ./network_throughput_test.sh -s sending_nodes -r receiving_nodes -i bondeth0
m
Using 'scam09db01 scam09db02 scam09db03 scam09db04 scam09db05 scam09db06 scam09db07 scam09db08' hosts for sending nodes
co t Gu
ฺ
Using 'scas10adm01 scas10adm02 scas10adm03 scas10adm04' hosts for receiving nodes
g
Validating Trusted SSH to sending nodes scam09db01 scam09db02 scam09db03 scam09db04 scam09db05 scam09db06 scam09db07 scam09db08 ... OK
kn den
Validating Trusted SSH to receiving nodes scas10adm01 scas10adm02 scas10adm03 scas10adm04 ... OK
n
Total Network Bandwidth 8,372,597,121 bytes/sec
a
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l
Assumptions
l o li
b
k e emscript uses the QPERF utility that is provided as a part of an Exadata Engineered System
This
a as well as a part of the Zero Data Loss Recovery Appliance.
This script may be executed from any Linux server with a BASH shell, and requires trusted
SSH to be established between the execution node and the two sets of nodes that are being
tested.
The Execution node may be one of the nodes being tested.
The O/S user used for trusted SSH is the O/S user running the script. There is no support for
the script to use a different O/S user for logging into the nodes being tested.
Installation
If the script is not already available under /opt/oracle.RecoveryAppliance/client, then
download it as instructed in the MOS Note given later in the lesson.

Usage
$ ./network_throughput_test.sh -h
Usage: ./network_throughput_test.sh -s [sending_nodes_file] -r
[receiving_nodes_file] -i [interface]
Parameters
The files sending_nodes_files and receiving_nodes_file are in the format of one
host name per line, similar to the group file used by DCLI utility
The interface defaults to bondeth0 but can be any interface on the receiving nodes, such
as bondeth1 if testing network throughput to a downstream replica.
MOS Note: Zero Data Loss Recovery Appliance Network Test Throughput script (Doc ID
2022086.1)
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Agenda

Recovery Appliance
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Oracle Enterprise Manager: Performance Pages
• Recovery Appliance Home Page

a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Appliance
The Recovery li home page is a virtual command center providing an overview of all
b
em of centralized management of the environment including storage, performance,
easpects
ak recent activity, issues that may need attention, and a summary of the overall environment.
The Home Page is divided into sections, with each section providing a snapshot of the
relevant information.

Summary
In this lesson, you should have learned how to identify various

optimization options.
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

ake
me
bel
l o
a k
li
l l o
(a cens
u
be e to
n
@ use
b a n
nio this
k
S t
ฺ c
ud
ng ent
)
s
om Guid
ha eฺ
an
o n - t r an
sfe r a b
le
8
Disaster Recovery with Replication a ble

fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Objectives

• Describe Replication with Oracle Recovery Appliance
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Agenda
• Replication Topologies: Overview

• Recovery Appliance Replication Process
• Recovery Appliance Replication Interfaces

• Recovery Appliance Replication Configuration
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Data Loss Protection from Site Disasters
• Replication to a Remote Appliance protects data from
disasters or site failures.
• Automated restore from a Local Appliance or directly from
Remote Appliance.
Recovery Appliance supports

most replication topologies. Topology Requirement
Traditional Require backup copies to be retained at an

offsite location for disaster recovery
a ble
fe r
an s
Bi-Directional Multiple data center environments in which each
n - t r
no
is a disaster recovery site for the other
s a
h a ฺ
) centers/offices
m
Hub and Spoke • Multiple remote data
i d e
g ฺ
• Backups arecoaggregated
t G uone central
to
a n kn den
location
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Topologies
Replication li
b
k e em
Replication topologies can be as simple or complex as required. The primary variables are as
a follows:
• The total number of Recovery Appliances in the replication environment, and their
relationships to one another
• The protection policies on the upstream Recovery Appliance that manage the outgoing
replicated backups, and the policies on the downstream Recovery Appliance that
manage the incoming replicated backups
• The replication server configurations that exist on each Recovery Appliance in the
replication environment
• The association between a replication server configuration and a protection policy

You can chain replication so that an upstream Recovery Appliance replicates to multiple
downstream Recovery Appliances, while a downstream Recovery Appliance receives backups
from multiple upstream Recovery Appliances. A downstream Recovery Appliance can receive
both backups from both its own protected databases and replicated backups. Any Recovery
Appliance in the replication topology can simultaneously perform the upstream and downstream
replication roles.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Recovery Appliance Replication Concepts
Replication is driven by a protection policy, which
means that all databases associated with the policy
are replicated.
• It is fully automatic after the initial setup.
• Incoming redo blocks are not replicated when

real-time redo transport is enabled. Only
when an archived log backup is created on
the upstream Recovery Appliance will it be
automatically replicated.
• After a backup is received and placed in a
queue for ingest, replication will begin.
• The upstream (source) catalog is updated ZDLRA 1
with metadata regarding copies on the
Recovery Appliance replica.
(Source)
a ble
fe r
• Backups are not compressed prior to being
Incr
an s
replicated.
Ingest Backup n - t r
• You can assign multiple replication server
configurations to a protection policy. Process a no ZDLRA 2
• The replication server configuration name is
h a s (Replica)
arbitrary.
m ) ideฺ
• The naming convention is us for upstream
g ฺ co t Gu Incr
and ds for downstream.
a n kn den Ingest
Process
Backup
•
process. i o nb s Stu
Data gets validated at every step in the
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Conceptse l l o li
b
k e em
To protect against server or site outage, a Recovery Appliance can replicate backups to a
a different Recovery Appliance.
When the upstream Recovery Appliance sends the incremental backup (full level 0 for the first
time and incremental level 1 subsequently) to the downstream Recovery Appliance, it creates
a virtual full backup as normal.
The downstream Recovery Appliance creates backup records in its recovery catalog. When
the upstream Recovery Appliance requests the records, the downstream Recovery Appliance
propagates the records back.
If the local Recovery Appliance cannot satisfy virtual full backup requests, then it
automatically forwards them to the downstream Recovery Appliance, which sends virtual full
backups to the protected database. DBAs use RMAN as normal, without needing to
understand where or how the backup sets are stored.

Agenda


a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Recovery Appliance Replication Process
When a protected database sends a backup to a Recovery Appliance configured for
replication, the following basic steps occur:
1. The upstream Recovery Appliance ingests the backup, checking the protection policy
to determine whether it is associated with a replication server configuration.

2. If a replication server configuration exists for the protection policy, then the upstream
Recovery Appliance replicates the backup.
3. The downstream Recovery Appliance ingests the backup, thereby creating a virtual
backup.
4. Shortly afterward, the upstream Recovery Appliance sends a reconcile request to the
downstream Recovery Appliance, which in turn sends metadata about the backup to
the upstream Recovery Appliance.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l o
e Processli
l
Replication
m b
eAeprotected database is associated with each Recovery Appliance catalog where its backups
a k reside. In this way, a backup could be restored directly from a downstream Recovery
Appliance by connecting to that Recovery Appliance’s catalog. Furthermore, the replicated
backup information is also stored on the upstream Recovery Appliance. So, if the backup has
been expired from the upstream, it will be automatically retrieved from the downstream
Recovery Appliance.
An initial full backup (original incremental level 0 or the most recent virtual full) is replicated,
and then follow-on incremental backups and archived logs are replicated. The backup would
be ingested on both the upstream and downstream Recovery Appliances as each is
independent of the other.

Agenda


a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Recovery Appliance Replication Interfaces
User Interfaces
• Oracle Enterprise Manager Cloud Control
• DBMS_RA PL/SQL Package

SQL> DESC RASYS.DBMS_RA
…
PROCEDURE CREATE_REPLICATION_SERVER
Argument Name Type In/Out Default?
------------------------------ ----------------------- ------ --------
REPLICATION_SERVER_NAME VARCHAR2 IN
SBT_SO_NAME VARCHAR2 IN
SBT_PARMS VARCHAR2 IN DEFAULT
MAX_STREAMS NUMBER IN DEFAULT
CATALOG_USER_NAME VARCHAR2 IN
WALLET_ALIAS VARCHAR2 IN
WALLET_PATH VARCHAR2 IN
le
PROXY_URL VARCHAR2 IN DEFAULT
b
PROXY_PORT NUMBER IN DEFAULT
…
HTTP_TIMEOUT NUMBER IN DEFAULT
fe r a
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Interfaces
Replication li
b
k e emReplication page in Cloud Control is the recommended interface for configuring the
The
a Recovery Appliance replication, but you can use also the DBMS_RA package to create and
manage replication.
The previous examples show how easy it is to create a Replication Server, add a Protection
Policy to the Replication Server, and a description of the parameters to pass to the
DBMS_RA.CREATE_REPLICATION_SERVER procedure to create a Replication Server.

Recovery Appliance Replication Status
Oracle Enterprise Manager Cloud Control

Webpages related to the replication functionality:
Replication
Enabled
Replication
Summary
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
Replication
n k ng ent
Destination
n b a
S t ud
u nio this 11
l l o @ use
be e to
a k
l o (a cens
el
OraclebEnterprise li
Manager Cloud Control administration web pages
k e em Page
Home
a • Summary: In this section, use the Current Activity tab to display information about
Backup, Copy to Tape, Replication, and Restore operations.
• Replication: Recovery Appliance can replicate backup data to another Recovery
Appliance. This section contains a table that lists other Recovery Appliances that are
replicating to this Recovery Appliance or that are being replicated to or from this
appliance. If replication has not been configured, a link will appear that allows you to
configure replication.
Replication Page
Use this page to configure replication of the backups associated with a protection policy. A
Recovery Appliance can replicate backup data to other local or remote Recovery Appliances.

Protected Databases Page
The Protected Databases table displays the Database name, Version, Protection Policy,
Database Size, Recovery Window information, Unprotected Data Window information, Near-
Zero Data Loss Recovery, Errors/Warnings, Last Complete Backup, Replication, and Copy-
to-Tape.
Protection Policies Page
The Protection Policies table displays the protection policy Name, Recovery Window Goal
(days), Storage Location, Backup Polling Location, and Copy-to-Tape and Replication
Settings.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Agenda


a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Recovery Appliance Replication Configuration
Oracle Enterprise Manager Configuration

Tasks:
• Creating a Replication Server

• Adding Protection Policies to a Replication Server
• Pausing and Resuming a Replication Server
• Removing a Replication Server
Add Protection
a ble
Policies fe r
an s
Pause and
n - t r
Resume
a no
h a s
m ) ideฺ
g ฺ co t Gu
Create
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
Beforebconfiguring li
replication, protected databases for which backups are to be replicated
k e emmeet the following criteria:

must
a • They must be enrolled with the downstream Recovery Appliance prior to configuring
replication on the upstream appliance.
• They must be assigned to a protection policy on the upstream Recovery Appliance for
which replication information is configured.
To configure a replication server, follow these steps:
1. From the Targets menu, select Recovery Appliance. Enterprise Manager displays the
Recovery Appliance page.
2. Click the Name of the Recovery Appliance for which you want to configure a replication
server. The Recovery Appliance home page displays. From the Recovery Appliance
menu, select Replication. Enterprise Manager displays the Replication page.

3. Click Create Replication Server. Enterprise Manager displays the Create Replication
Server dialog box.
a. Select the Downstream Recovery Appliance and specify the required credentials.
b. In the Maximum Streams field, enter the maximum number of simultaneous
replication tasks.
c. Use the Downstream Recovery Appliance Database Credentials field to specify
database user credentials for a virtual private catalog user on the downstream
Recovery Appliance who has been granted permission to manage the protected
databases for which backups will be replicated.
d. Use the Upstream Recovery Appliance Host Credentials field to specify credentials
for the operating system user who owns the Recovery Appliance database
installation.
4. Click OK.
After that you should be able to add Protection Policies and Pause or Resume Replication fora ble
maintenance tasks. fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Recovery Appliance Replication Configuration
SQL> COL REPLICATION_SERVER_NAME FORMAT A30
SQL> COL REPLICAITON_SERVER_STATE FORMAT A30
CLI - DBMS_RA PL/SQL package

SQL> SELECT DISTINCT REPLICATION_SERVER_NAME, REPLICATION_SERVER_STATE
2 FROM RASYS.RA_REPLICATION_SERVER;
REPLICATION_SERVER_NAME REPLICATION_SERVER_ST
Principal Procedures relevant to Replication:

------------------------------ ---------------------
ZDLRA1_REP RUNNING
SQL>
Program Unit Description

Creates a replication server configuration that specifies a
CREATE_REPLICATION_SERVER downstream Recovery Appliance to which this Recovery
Appliance replicates backups
DELETE_REPLICATION_SERVER Deletes a replication server configuration
a ble
Adds a replication server configuration to the protection
fe r
ADD_REPLICATION_SERVER policy that was created by the
CREATE_REPLICATION_SERVER procedure
an s
n - t r
REMOVE_REPLICATION_SERVER a no
Removes a replication server configuration from the
protection policy that was created by the
a s
CREATE_REPLICATION_SERVER procedure
h
m ) ideฺ
PAUSE_REPLICATION_SERVER or ฺ co pauses
Thisgprocedure
t G uor resumes replication to the
RESUME_REPLICATION_SERVER
n n downstream
kspecified e n Recovery Appliance
a d
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l
Command-Line
o li
Interface
b
k e emcan use the DBMS_RA package to create and manage replication.
You
a The Recovery Appliance permits in-progress replication of backup pieces to complete. If the
Recovery Appliance queued backup pieces for replication through this replication server
configuration but did not replicate them, then the Recovery Appliance holds the backup
pieces until you call RESUME_REPLICATION_SERVER. No replication tasks that run against
this Recovery Appliance can execute until you resume replication to the downstream
Recovery Appliance.

In the following example we show how to pause and resume replication:

SQL> COL REPLICATION_SERVER_STATE FORMAT A30
SQL> SELECT DISTINCT REPLICATION_SERVER_NAME,
REPLICATION_SERVER_STATE FROM RASYS.RA_REPLICATION_SERVER;
REPLICATION_SERVER_NAME REPLICATION_SERVER_STATE
------------------------------ ------------------------------
ZDLRA1_REP RUNNING
SQL>
a b le
fe r
ans
SQL> SELECT DISTINCT REPLICATION_SERVER_NAME,
n - t r
a no
h a s
REPLICATION_SERVER_NAME
m ) ideฺ
REPLICATION_SERVER_STATE
g ฺ co t Gu
------------------------------ ------------------------------
a n kn den
nb s Stu
ZDLRA1_REP PAUSED
n i o i
u e t h
e l lo@ o us
SQL> EXEC RASYS.DBMS_RA.RESUME_REPLICATION_SERVER( 'ZDLRA1_REP' );
a a kb se t
l o ( censuccessfully completed.
PL/SQL procedure
b e l li
m
a keeSQL> SELECT DISTINCT REPLICATION_SERVER_NAME,
------------------------------ ------------------------------
ZDLRA1_REP RUNNING
SQL>

Recovery Appliance Replication Status
CLI Views
Different data dictionary views related to Replication functionality:
View Description
This view describes the downstream Recovery
RA_REPLICATION_SERVER Appliances that are directly receiving replicated
backups from this particular Recovery Appliance.
The POLICY_NAME column of this view lists the
a ble
protection policy used by this protected
fe r
RA_DATABASE database. The REPLICATION_USAGE column
an s
shows the cumulative amount of disk space (in
n - t r
a no
GB) replicated for this database.
This view describes the defined protection
RA_PROTECTION_POLICY policies. h a s
m ) ideฺ
RC_BACKUP_PIECE_DETAILS ฺ c o Gu about all available
Lists detailed information
k n g ntrecorded in the recovery catalog

backup pieces
n b an tude
u nio this S
l l o @ use
be e to
a k
l o (a cens
el
CLI Views
b li
k e emcan monitor replication using the Recovery Appliance catalog views.
You
a

Display Replication Server configuration and Protection Policies associated to them:

SQL> COL REPLICATION_SERVER_STATE FORMAT A30
SQL> COL PROTECTION_POLICY FORMAT A30
SQL> SET LINESIZE 100

SQL> SELECT REPLICATION_SERVER_NAME, REPLICATION_SERVER_STATE,
PROTECTION_POLICY
2 FROM RASYS.RA_REPLICATION_SERVER;
PROTECTION_POLICY
a ble
------------------------------ ------------------------------ ------
fe r
------------------------
an s
- t r
non
ZDLRA1_REP RUNNING BRONZE
ZDLRA1_REP RUNNING
h a sa GOLD
ZDLRA1_REP RUNNING
m ) ideฺ SILVER
g ฺ co t Gu
SQL>
a n kn den
i o nb s Stu
u n t h i
l l o @ us
Display cumulative GB replicated for e
several databases …
a k be e to
SQL>
l o (a cens
SQL>be
l li
SELECT DB_UNIQUE_NAME, REPLICATION_USAGE FROM
m
kee
RASYS.RA_DATABASE;
a
DB_UNIQUE_NAME REPLICATION_USAGE
-------------------------------- -----------------
CUST15 0
DBNZDL 0
CUST05 0
ODA1RAC 6601.02258
DB1116 6240.13765
ODA2DBLG 478.316072
DB1124 3314.12441
CUST02 0
ODA2RAC 5269.81718
DB1025 365.495429
CUST09 0
… Zero Data Loss Recovery Appliance Overview 8 - 19
Summary

• Identify multiple replication topologies for the Recovery
Appliance
• Describe the replication process for the Recovery Appliance
• Configure the Recovery Appliance replication process
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

9
Recovery Appliance Integrated

Media Management r a ble
s fe
- t r an
o n
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake
Objectives

• Describe the Recovery Appliance integration with Media
Management software and components

• Use the Recovery Appliance Integrated Media Management
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Agenda
• Media Management Overview

• Media Manager Integration with the Recovery Appliance
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Tape Archival
• Optional low-cost Tape Archival provides unalterable Recovery

Appliance
protection from:
– Intentional attacks by hackers, employees, and viruses

– Accidental data deletion because of using tape as
long-term archive
– Malfunction of primary data protection system
• Autonomous Tape Archival is built-in: just connect 16 Gb FC
tape drives le
r a b
– Offloads tape backup activity from production servers
s fe
– No expensive media manager backup agents on r a n
production systems o n -t
– Tape drives run all day without slowing production s an
systems ) ha eฺ Supports Most
– Integrated Media Manager included, ฺ c om
but G
there’su id
an Major Tape
g t
nkn den
Libraries
option to use your own tape software
n b a tu
n i o i s S
u e t h
a a kb se t
l o ( cen
A robust
b l li protects data against intentional attacks, unintentional user errors
ebackup strategy
m as file deletions), and software or hardware malfunctions. Tape libraries provide
kee
(such
a effective protection against these possibilities.
The fundamental difference in the two approaches is that the Recovery Appliance backs up to
tape, and not to the protected databases. The Recovery Appliance comes with preinstalled
Media Manager software, and supports optional Fibre Channel cards. Thus, installation of a
media manager is not necessary on the protected database hosts.
When the Recovery Appliance executes a copy-to-tape job for a virtual full backup, it
constructs the physical backup sets, and copies them to tape, and then writes the metadata to
the recovery catalog. If you want, the Recovery Appliance can also copy successive
incremental backups and archived redo log file backups to tape. Whereas the backup on the
Recovery Appliance is virtual, the backup on tape is a non-virtual, full physical backup. The
Recovery Appliance automatically handles requests to restore backups from tape, with no
need for administrator intervention.

The advantages of the Recovery Appliance tape solution are as follows:
• The Recovery Appliance performs all tape copy operations automatically, with no
performance load on the protected database host.
• Tape backups are optimized. The Recovery Appliance intelligently gathers the
necessary blocks to create a non-virtual, full backup for tape.
• The Media Manager software is preinstalled, eliminating the need for costly third-party
media managers.
• Tape drives and tape libraries function more efficiently because the Recovery Appliance
is a single large centralized system with complete control over them. In other tape
solutions, hundreds or thousands of databases can contend for tape resources in an
uncoordinated manner.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Recovery Appliance Tape Configuration
• Red Stack
• The Recovery Appliance uses Integrated Media Manager
software
• Components
– 1 x QLogic QLE8362 Converge Network Adapter (configured for FC)
per node
– 2 x Brocade 6510 Switches (16GB x 48 port switches) for HA
– Oracle StorageTek Modular Library System:
a ble
FC-SCSI: SL3000 and SL150 fe r
—
an s
— ACSLS API: SL8500 and SL3000
n - t r
— ACSLS Logical Libraries: SL8500 and SL3000
a no
• Encryption
h a s
– Oracle Key Manager m ) ideฺ
• Compression g ฺ co t Gu
a n kn den
– Tape Library compression
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l o
e Applianceli Media Manager
Recovery l
b
k e emRecovery Appliance Media Manager provides reliable, centralized tape management by
The
a protecting file-system data and Oracle Database files for multiple environments. This Media
Manager is the tape management component for the Recovery Appliance. It is installed with
its components on the Recovery Appliance during its configuration.
Components
• QLogic QLE8362 Converge Network Adapter provides industry-leading native FC
performance with up to 16 Gb throughput at extremely low CPU usage and full hardware
offloads. It also supports 10 Gb FCoE functionality but the Recovery Appliance
implements only native Fibre Channel connectivity. There are two HBAs, one per
compute node installed. The HBA is configured via the ra_install script during
installation. For additional information refer to the following PDF file:
http://www.qlogic.com/OEMPartnerships/Oracle/Documents/Storage/ds_16GbFC-PCIe-
universalHBA.pdf

• The Brocade 6510 Fibre Channel Switch delivers industry-leading 16 Gb/sec
performance within a flexible, cost-effective, and easy-to-use 1U form factor.
Preconfigured by Oracle. For additional information refer to the following PDF:
http://www.oracle.com/us/products/servers-storage/storage/storage-networking/6510-
fibre-channel-switch/6510-ds-1663963-1881253.pdf
• Oracle's StorageTek SL8500 modular library system is an enterprise storage solution
that provides fully automated tape-cartridge storage and retrieval. It supports multiple
different tap drives from Oracle and third-party providers. For additional information refer
to the following web page: http://www.oracle.com/technetwork/documentation/tape-
storage-187555.html
Encryption
Copies to tape could be encrypted by leveraging Integrated Media Manager host-based
encryption or by support of tape drive encryption such as Oracle Key Manager software.
Media Manager manages the encryption keys for encrypted backups on tape or leverages on
a ble
Oracle Key Manager software. ZDLRA recommends using the Oracle Key Manager for
fe r
s
enterprise key encryption needs. This should be configured and functional prior to ZDLRA
an
deployment. Tape drives should already be registered, and encryption enabled.
n - t r
Compression a no
Copies to tape would not be compressed by the appliance as h a
the
sbackups would benefit from
standard tape drive compression. m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o li
b
k e em
a

Recovery Appliance Alternative Tape Configurations
• Connect into existing tape library SAN

– The Recovery Appliance uses Integrated Media Manager
preconfigured software.
– The Recovery Appliance has a dedicated library (partition).
– Each tape drive is accessed via one or two paths (HA).
– Redundant FC SAN is preferred (connected to two different SAN
switches).
• Third-party media management vendor (MMV) le
r a b
– The Recovery Appliance does not connect directly to the tape library
sfe
or the FC SAN infrastructure.
tr a n
n -
– The Recovery Appliance uses third-party MMV.
a no
– Media servers are supplied by the customer and
h a sconnect to the FC
SAN infrastructure.
m ) ideฺ
– Use 10 GigE to send backups fromgthe ฺ coRecovery
t G u Appliance Client to
media servers.
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
Recommendations li
b
emlibrary should have dedicated resources for the Recovery Appliance via either a
keThe
a physically dedicated library or virtualization like partitions and logical libraries. We recommend
using hardware-based drive cleaning with the Recovery Appliance. The tape library should be
online and ready to present to the Recovery Appliance. One or more tape drives should be
deployed. The drives must be dedicated to the Recovery Appliance.

Agenda
• Media Management Overview

• Media Manager Integration with the Recovery Appliance
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Integrated Media Manager Components
Components or host roles:

Administrative
• Administrative server:
server
Maintains configuration
settings and backup OSB catalog
history catalog N
E
• Media server: Transfers T Media
data to or from attached W server
O
R a ble
devices e r
K s f
• Client: Accesses data to - t r an
Storage device
be backed up no n
s
Client a
h a
) ideฺ
Administrative
m
ฺ co t Gu
Domain
g
n k n en Data to backup
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
el
An administrative
b li in Integrated Media Manager is a group of machines on your
domain
em that you manage as a common unit to perform backup and restore operations. An
enetwork
ak administrative domain has one administrative server, one or more clients, and one or more
media servers. In the Recovery Appliance, the two Compute Nodes support all of these
components or host roles.
• The administrative server is one of the two machines in your Recovery Appliance
administrative domain. The Recovery Appliance contains a full installation of Integrated
Media Manager software in a cluster configuration. This host maintains the backup
catalog files and other files for configuration settings and administrative data. The
administrative server runs the scheduler, which starts and monitors jobs within the
administrative domain.
• Each compute node has a media server in your Recovery Appliance that has one or
more secondary storage devices, such as a tape library, connected to it. A media server
transfers data to and from its attached storage devices. The Recovery Appliance
compute servers are media servers for direct attached tape devices.

• Each compute node acts as a client whose accessed data is backed up by Integrated
Media Manager. It constructs the physical backup sets and copies them to tape, and
then writes the metadata to the Recovery Appliance recovery catalog.
In addition to this, Integrated Media Manager is easily customizable for advanced tape
management (such as tape duplication and vaulting, among others).
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Integrated Media Manager Concepts
• Concepts
Media family
Volume set
Volume Volume
Backup image
a ble
Product of a backup operation fe r
an s
Product of a backup operation
n - t r
Physical tape Physical tape a no
h a s
m )images d e ฺ
Set of tapes containing backup
ฺ co t Gu i
Set of volumes with common
g
kn dcharacteristics
n
a n e
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li organizes the backups it creates in a simple hierarchy, which
e Media Manager
Integrated
b
m the following logical concepts:
kee
comprises
a • A “backup image” is the product of a backup operation. A backup image is a file that
consists of one or more backup sections. A backup image can be contained within a
single volume or it can span multiple volumes. The part of a backup image that fits on a
single volume is called a “backup section.”
• A “volume” is a single unit of media, such as an LTO-5 tape. It can store many backup
images.
• A “volume set” is a group of one or more volumes that contain a complete backup
image.
• A “media family” or “tape pool” is a named classification of volumes that share some
common attributes, such as the volume naming method, the policies used for writing
data to volumes, and the length of time volumes are retained in the media family.
When you back up files by using Integrated Media Manager, you generate a volume set that
has some common characteristics defined by the media family that you have associated with
your backup operation.

Integrated Media Manager Interfaces
CLI
• obtool
[root@host1 ~]# which obtool

/usr/bin/obtool
[root@host1 ~]# obtool
ob> lsloc
Media_Recycle_Bin OSB-generated location
robot0 OSB-generated location for library robot0
ob>
BUI a ble
fe r
• Web tool an s
n - t r
• Enterprise Manager
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e
Interfaces l l o li
b
k e e•m Command Line Tool - obtool: Use this tool to perform all Integrated Media Manager
a configuration, backup and restore, maintenance, and monitoring operations.
• Web tool: This is a browser-based GUI that enables you to configure an administrative
domain, manage backup and restore operations, and browse the backup catalog.
• Oracle Enterprise Manager: Use this tool to perform most Media Manager tasks,
including administrative domain and hardware configuration, managing your media, and
backing up and restoring databases. Oracle Enterprise Manager is the preferred Web
interface for Media Manager tasks.

Media Manager Implementation in the Recovery
Appliance
• Media Manager has a new cluster-enabled implementation in
– Key Components
— CRS stack
— RDBMS Media Manager Catalog
— DBFS shared file system
— Media Manager software
• Oracle Clusterware software a ble
fe r
– Use crsctl {status|check|start|stop} res {osbadmin|osbmedia}
a n s
-t r
– During a relocation, running jobs are impacted. non
• Copy-to-Tape Services h a sa
m )Media d e ฺ
– For Copy To Tape to work, we need all
ฺco t Gu i Manager
services up and healthy. kng n
a n d e
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l
Implementation
o li
b
k e em Media Manager is deployed as a clusterware software in the Recovery Appliance.
Integrated
a All services must be up for the Media Manager to start. The CRS service will pull up the
required services.
It has a cluster service for the admin role, which uses the Oracle RAC database as a Media
Manager catalog. Media server and client services are also managed by the clusterware
software.
Oracle Clusterware software commands are used to manage the Media Manager services.
Standard Media Manager commands still work. During a relocation, the job may restart in
Media Manager; otherwise, it will drop back into the Recovery Appliance retry logic and either
fail (if it has exhausted its retries) or re-execute. After the relocation, services and jobs resume
on their own.
To restart services or check their status, use crsctl as root.
For Copy to Tape to work, the Recovery Appliance server must be up and the SBT_LIBRARY
must be “READY”:

SQL> col lib_name format a30
SQL> SELECT lib_name, status FROM rasys.ra_sbt_library;
LIB_NAME STATU
------------------------------ -----
ROBOT0 READY
SQL>
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Oracle Clusterware Media Manager Services in the
Recovery Appliance
• Media Manager services
[oracle@host1 ~]$ crsctl stat res osbadmin

NAME=osbadmin Media Manager
TYPE=cluster_resource Admin service
TARGET=ONLINE
STATE=ONLINE on host1
[oracle@host1 ~]$ [oracle@host1 ~]$ crsctl stat res host1-avip3

NAME=host1-avip3
TYPE=app.appvipx.type
TARGET=ONLINE
a ble
[oracle@host1 ~]$
fe r
an s
n - t r
[oracle@host1 ~]$ crsctl stat res osbmedia
NAME=osbmedia no
Media Manager
a
TYPE=local_resource
h a s
Media services
) ideฺ
TARGET=ONLINE , ONLINE
STATE=ONLINE on host1, ONLINE on host2
m
co t Gu
[oracle@host1 ~]$
g ฺ
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
OraclebClusterware liMedia Manager Services
k e emcrsctl to check the status, start, and stop the Media Manager Admin, Media, and Client
Use
a Services. The Admin service will rotate between the nodes due to switchover or failover
maintenance operations. Each compute node has a media server and a client.
obtool commands
In addition to this you should be able to use Media Manager standard commands.
For example, to list host roles in the Media Manager implementation:
[root@host1 ~]# obtool lsh
host1 mediaserver,client (via OB) in service
host2 mediaserver,client (via OB) in service
host1-avip3 admin,mediaserver,client (via OB) in service
[root@host1 ~]#

Oracle Clusterware Media Manager Directories in the
Recovery Appliance
• Media Manager file system
[oracle@host1 ~]$ crsctl stat res ob_dbfs Media Manager

NAME=ob_dbfs
TYPE=local_resource DBFS cluster file system
TARGET=ONLINE , ONLINE
STATE=ONLINE on host1, ONLINE on host2
[root@host1 ~]# df -h /dbfs_obdbfs
[oracle@slcm03adm03 ~]$ file system Size Used Avail Use% Mounted on
dbfs-@obdbfs.local:/ 300 G 66G 235G 22% /dbfs_obdbfs
[root@host1 ~]#
• Media Manager software binaries

a ble
[root@host1 ~]# cd /dbfs_obdbfs/OSB/backup
[root@host1 backup]# ls
fe r
admin apache bin device drv etc help install lib man samples templates
an s
tools
[root@host1 backup]# cd bin
n - t r
[root@host1 bin]# ls -l
total 66681
a no Media Manager
-rwxr-xr-x 1 root root 65721589 Jan 27 2016 hostinfo
h a s obtool utility
) ideฺ
-rwxr-xr-x 1 root root 78589 Dec 14 2015 obcm
-rwsr-xr-x 1 root root
-rwsr-xr-x 1 root root
150403 Dec 14 2015 obt
777101 Dec 14 2015 obtar m
co t Gu
g ฺ
-rwsr-xr-x 1 root root 1481839 Dec 14 2015 obtool
-rwxr-xr-x 1 root root
[root@host1 bin]#
a n kn den
70192 Dec 14 2015 php.ini
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
OraclebClusterware liMedia Manager Directories
k e em Media Manager is installed on a DBFS file system in the Recovery Appliance to be
Integrated
a accessible by the two compute nodes.
Configuration files and binaries are located in this DBFS file system:
[root@host1 OSB]# pwd
/dbfs_obdbfs/OSB
[root@host1 OSB]# ls -l etc
total 1
drwxr----- 5 root oinstall 0 Jan 28 2016 ob
-rw-r--r-- 1 root root 186 Jan 27 2016 obconfig
[root@host1 OSB]#
[root@host1 OSB]# ls -l etc/ob
total 1
drwxr-xr-x 2 root root 0 Aug 31 06:00 mail
drwxr-xr-x 2 root root 0 Nov 17 2015 report
drwxr----- 2 root oinstall 0 Jul 28 08:52 wallet
lrwxrwxrwx 1 root root 15 Nov 17 2015 xcr -> /usr/etc/ob/xcr
[root@host1 OSB]#
Copy-to-Tape Backup Operations in the Recovery
Appliance
Recovery Appliance Protection Policy Attribute Set(s)

• Defines the recovery window on tape • Associated with a Media Manager Library
• Defines the number of streams to use for the
backup operation
Copy-to-Tape Job Templates
• Media management parameters (optional)
Define: – These override any parameters defined in
• Specific database or all databases in the Media Manager Library setting or via
a protection policy the Media Manager
• Type of backup: Full, incremental a ble

and/or archived logs Media Manager Library fe r
an s
• Job priority, number of copies, and
runtime window
• Associated with the installed Media Manager-
n - t r
• Media Management Library and a no
specific SBT library (OSB by default)
Attribute Set h a s
• Sets maximum number of RMAN channels
m ) ideฺ
that can be used concurrently (based on the
number of available tape drives)
ฺco thetRMAN
• Copy-to-Tape schedule:one time or
recurring g
•nDefines G u media parameters to be
a n k usedden
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e
Operationsl l o li
b
k e em are the main categories of operations required to configure the Recovery Appliance to
These
a execute Copy-to-Tape Jobs:
• Media Manager Library
• Attribute Set(s)
• Copy-to-Tape Job Templates
• Recovery Appliance Protection Policy

Integrated Media Manager Concepts in the Recovery
Appliance
• Media Families or Tape Pools
[root@host1 ~]# obtool lsmf

OSB-CATALOG-MF write 7 days keep 14 days
RMAN-DEFAULT content manages reuse
host1_db write 21 days content manages reuse
host1_fs write 21 days keep 32 days
[root@host1 ~]#
• Media Manager Libraries

ob> lsdev
le
library robot0 in service
drive 1 robot0_tape01 in service
r a b
e
drive 2 robot0_tape02 in service
ob>
s f
• Volume Sets and Volumes - t r an
no n
ob> lsvol -L robot0
s a
Inventory of library robot0:
in 1: h a
) ideฺ
volume host1_db-000036, barcode DEMO10,
m
1121196032 kb remaining, content manages reuse, mediainfo hw
co t Gu
encryptable
in 6:
g ฺ
volume host1_db-000035, barcode DEMO19,
encryptable
a n kn den
1093332992 kb remaining, content manages reuse, mediainfo hw
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Media b e l l
Library
o li
k e em its installation, while the Recovery Appliance is being configured, the Integrated Media
During
a Manager creates a media manager library with default parameters, such as the following:
• Library name (ROBOT0)
• Maximum number of accessible tape drives
• Number of restore drives
• Media manager location (robot0)
Attribute Set
An SBT attribute set helps you to further customize and categorize your backups while
copying them to tape. An attribute set is created for each tape drive associated with its media
manager library. It helps classify backups while storing them on tape by specifying
parameters such as the media pool number, streams, and media manager commands
needed to perform the tape backup operation.

The Recovery Appliance comes with Integrated Media Manager preconfigured as its media
manager. Integrated Media Manager components, which include a media manager library and
attribute sets for each of its tape drives, are also preconfigured. The preconfigured attribute sets
are typically named DRIVE_COUNT_1, DRIVE_COUNT_2, and so on for the number of existing
tape drives.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Integrated Media Manager in the Recovery Appliance
• Administration Tasks
– Media Manager Library ROBOT0 and the Attribute Sets for
each of its tape drives (such as ROBOT0_DRIVE_COUNT_1)

are preconfigured.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
ฺ o Gu
cAttribute
Media Manager
k g
n enTasks t Set
Library Tasks
b a n ud
n S t
u nio this
l l o @ use
be e to
a k
l o (a cens
el Tasksli
Administration
b
k e em
To copy backups to tape by using the Recovery Appliance, you should first be able to create a
a media manager library for your media management software to manage all your tape backup
jobs by adding parameters that apply to a set of jobs. You should also be able to create a
media manager attribute set that helps you control your tape backup jobs further by adding
more job-specific parameters and commands for your media manager software.
In the slide, we illustrate how to execute some administration tasks related to create, edit, and
delete media manager libraries and attribute sets assigned to media manager libraries.
Remember that the Recovery Appliance comes with Integrated Media Manager preconfigured
as its media manager. Integrated Media Manager components, which include a media
manager library and attribute sets for each of its tape drives, are also preconfigured.

Integrated Media Manager
• Copy-to-Tape Job Templates

a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
Job Infrastructure li
b
em created by RMAN, backing up an Oracle database results in a backup set (an RMAN-
eWhen
ak specific logical structure), which contains at least one backup piece (an RMAN-specific
physical file containing the backed up data). In the Recovery Appliance, the Oracle Database
backups are stored as virtual full backups. So, when the Recovery Appliance executes a
copy-to-tape job for a virtual full backup, it constructs the physical backup sets, and copies
them to tape.
Integrated Media Manager backs up and maintains backup metadata for each RMAN backup
piece written to tape within its own catalog. You can browse backup pieces with the obtool
command line, or with Integrated Media Manager Web tool by using Oracle Enterprise
Manager Cloud Control.
Use the Copy-to-Tape Job Templates page to specify copy-to-tape job templates for a
selected database or databases associated with a protection policy. You can schedule the job
to define when the backups are copied to tape. A copy-to-tape job template also specifies a
media manager attribute set to control the copy operation.

Copy-to-tape jobs may be scheduled by database or protection policy.
When initially configuring a copy-to-tape job, the most recent virtual full backup will be copied
(e.g. point of last incremental).
Copy-to-Tape Templates
Parameters to specify when creating a Copy-to-Tap Job Template:
• Name
• Media Manager Library: ROBOT0

• Attribute Set: The settings for your tape backup job
• Protected Policy or DB: Scope of the backup job
• Backup Type: FULL | INCR | ARCH : Implies that all your <Backup Type> database
backups for databases included in the Protected Policy or the specific DB above
mentioned above will be copied to tape
• Full Backup Template: The specified full backup template used to correlate the
a b le
incremental and/or archived log backups to a full backup. Only applies to INCR and fe r
ARCH backup types ans
n - t r
• Priority: Low | Medium | High | Critical (default Medium)
a no
• Copies: The number of copies you need for the backup being copied to tape
h a s
) ideฺ
• Runtime Window: The amount of time in minutes, hours, or days allowed for this job to
m
complete.
g ฺ co t Gu
a n kn den
• From Tag: Specify a tag name to only copy backups associated with a certain tag to
tape
i o nb s Stu
u n h i
• Schedule: Immediately | Later: Specify whether you want this job to run immediately or
t
lo@ o us
at a later specified time
l e
• Repeat
e
kb se t
a a
( cen
l o
Protection Policies
l
e to specify li
m b
Parameters when creating a Protection Policy to include a Media Manager
e
ake
Retention policy:
• Media Manager Recovery Window: Use this field to specify a longer window within
which point-in-time recovery capability from a media manager (Integrated Media
Manager for example) will be maintained.

Pause and Resume

a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l o
e Stateli
l
Maintenance
b
k e em performing tape hardware maintenance, you should be able to disable the
Before
a SBT_LIBRARY and ensure that no more jobs are active. You can choose to pause Media
Manager operations by using Cloud Control, and then resume them when required.
In addition to using Enterprise Manager, you can use DBMS_RA PL/SQL package
procedures.
When the Media Library is running, the query above returns READY.

Example of pausing the ROBOT0 library:
SQL> BEGIN
2 RASYS.DBMS_RA.PAUSE_SBT_LIBRARY( lib_name => 'robot0' );
3 END;
4 /
SQL> COL LIB_NAME FORMAT A40

SQL> SELECT LIB_NAME, STATUS FROM RA_SBT_LIBRARY
2 WHERE LIB_NAME = 'ROBOT0';
a ble
fe r
LIB_NAME STATU
ans
---------------------------------------- -----
n - t r
o
ROBOT0 PAUSE
s an
) ha eฺ
SQL>
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
b el li
k e em
a

• Job Monitoring
SQL> COL STATE FORMAT A10

SQL> COL DB_UNIQUE_NAME FORMAT A20
SQL> COL ERROR_TEXT FORMAT A20
SQL> SELECT TASK_ID, STATE, DB_UNIQUE_NAME, ERROR_TEXT, BS_KEY, PIECE#
2 FROM RASYS.RA_SBT_TASK
3 WHERE TEMPLATE_NAME = 'SF_TAPE_GOLD'
4 ORDER BY DB_UNIQUE_NAME, BS_KEY, PIECE#
5 /
a ble
TASK_ID STATE DB_UNIQUE_NAME ERROR_TEXT BS_KEY PIECE#
fe r
----------
2051036
----------
COMPLETED
-------------------- -------------------- ---------- ----------
ODA1RAC 1049767
an s
1
2051038 COMPLETED ODA1RAC 1049780
n - t r 1
no
2051040 COMPLETED ODA1RAC 1049800 1
2051042
2051044
COMPLETED
COMPLETED
ODA1RAC
ODA1RAC
s a 1049824
1049832
1
1
2051046 COMPLETED ODA1RAC
h a
) ideฺ
1049921 1
2051047 COMPLETED ODA1RAC
m
co t Gu
1049979 1
…
g ฺ
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
Job Monitoring li
b
em an SBT job runs, it generates and queues a task for each backup piece to be copied.
eWhen
ak RA_SBT_TASK lists these tasks and their completion states.
You can determine whether SBT jobs run according to your defined schedule by querying the
RA_SBT_JOB.LAST_SCHEDULE_TIME column. This column indicates the last time that the
SBT job was scheduled to run.

Example:
SQL> COL TEMPLATE_NAME FORMAT A40
SQL> COL BACKUP_TYPE FORMAT A30
SQL> COL LAST_SCHEDULE_TIME FORMAT A40
SQL> SELECT TEMPLATE_NAME, BACKUP_TYPE, LAST_SCHEDULE_TIME

2 FROM RASYS.RA_SBT_JOB;
TEMPLATE_NAME BACKUP_TYPE
LAST_SCHEDULE_TIME
---------------------------------------- -----------------------------
- ----------------------------------------
a ble
SF_TAPE_AL ARCH 02-SEP-16
fe r
12.26.18.772028 AM -06:00
a n s
n -t r
SF_TAPE_SILVER FULL o
29-AUG-16
10.03.13.511986 AM -06:00
s an
SF_TAPE_SILVER_DAILY_INCR INCR ) ha eฺ 30-JUL-16
10.02.56.870003 AM -06:00
ฺ c om Guid
SF_TAPE_GOLD k
FULL
n ng ent 29-AUG-16
10.03.13.841137 AM -06:00 b a tu d
i o n s S
…
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o li
b
k e em
a

Summary

• Describe the integration of the Recovery Appliance with Media
Management software and components

• Use Integrated Media Manager with the Recovery Appliance
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

10
Maintenance and Patching r a ble

s fe
- t r an
o n
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake
Objectives

• Patch the Recovery Appliance
• Display the status of Recovery Appliance components

• Maintain hardware components
• Use the Enterprise Manager plug-in for the Recovery
Appliance
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Agenda
• Patching the Recovery Appliance

• Enterprise Manager Plug-In Installation
• Power On, Power Off

• Maintenance Tasks
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Recovery Appliance Patch Overview
Patching process
Download Bundle Pre-Patch Validations Post-Patch

Patch
a ble
1 2 3 4 5 Complete
fe r
Complete
an s
n - t r
Unpack the Patch File Patching a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn Versions
Zero Data Loss Recovery Appliance Supported
d e n (Doc ID 1927416.1)
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
General l l o i
eInformationlAbout the Upgrade
b
k e emupgrade process should only be completed on clean and functioning Recovery
The
a Appliances. Attempting an upgrade while an appliance is in an unknown service state could
cause stability or upgrade issues.
• Read the full instructions in the readme file included in the patch file.
• The automation tool /opt/oracle.RecoveryAppliance/bin/ra_update.pl is
delivered in the ra_automation rpm. You should ensure that you have the latest
version of ra_automation RPM.
• The upgrade requires access to the compute nodes as root, and access to the rasys
user.
- This is done by setting up upgrade root keys that are password protected by the
upgrade administrator, and by disabling the rasys user to customers during the
install.
- At no time does the upgrade automation need to know the customer set
passwords for the system. Also, at no time are default passwords set.

• You should ensure that you are upgrading to the latest recommended release.
- This release can be found here: Zero Data Loss Recovery Appliance Supported
Versions (Doc ID 1927416.1)
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

Recovery Appliance Patch Planification
Plan the Upgrade

• Get the required files staged on the compute nodes
[oracle@host1 radump]$ ls -lrt

total 2212236
drwx------ 2 root root 16384 Jan 14 14:05 lost+found
-rw-r-xr-- 1 oracle oinstall 14095 Jan 14 15:51 README.txt
-rwxr-xr-x 1 oracle oinstall 5564 Jan 14 15:51 ra_preinstall.pl
-rw-r--r-- 1 oracle oinstall 280267737 Jan 14 15:51 ra_automation-12.1.1.1.7-RELEASE.x86_64.rpm
-r-xr-xr-x 1 oracle oinstall 52774638 Jan 14 15:51 p6880880_121010_Linux-x86-64.zip
-rw-r--r-- 1 oracle oinstall 333056267 Jan 14 15:51 p22552579_121110_Linux-x86-64.zip
-rw-r--r-- 1 oracle oinstall 1596952526 Jan 14 15:57 p22899531_121020_Linux-x86-64.zip
[oracle@host1 radump]$
ab le
• The upgrade is non-rolling, and takes about a little more than an hour to fer
apply. a n s
n -t r
• Do not schedule backup jobs; Redo Shipping might have errors. o
• Upstream replication servers should be paused. s an
) ha eฺ
• Enterprise Manager agents will be stopped onm id servers during
the compute
ฺ c o u
the upgrade.
k n g nt G
b n and
• If Tape was used, pause the tapealibrary
t u dethen wait for the
n
nio this S
BACKUP_SBT tasks to complete.
u
l l o @ use
be e to
a k
l o (a cens
Plan the
b l
eUpgrade li
em there will be downtime, plan to do the upgrade at a time when it will least impact
eBecause
ak users. In addition, it might be useful to notify users when the system will be unavailable.
At the start of the patching process, the ra_update.pl script will stop all services.
Individual services may start and stop throughout the process, but ra_server itself will
remain unavailable.
Required Files
Obtain the required files:
• Get the latest Recovery Appliance patch set from here: Zero Data Loss Recovery
Appliance Supported Versions (Doc ID 1927416.1)
• Example contents of an RA Patch: (RA Patch Example is using Patch 22552579: ZERO
DATA LOSS RECOVERY APPLIANCE: 12.1.1.1.7
PATCHSET: p22552579_121110_Linux-x86-64.zip)
- p6880880_121010_Linux-x86-64.zip
- ra_automation-12.1.1.1.7-RELEASE.x86_64.rpm
- README.txt

• Unzip p22552579_121110_Linux-x86-64.zip on the first node as oracle in
/radump.
• Copy the included RPM "ra_automation-12.1.1.1.7-RELEASE.x86_64.rpm" to
all other compute servers as oracle.
• Download the following patch zip files, and place them in the /radump directory on the
first compute server: Patch 22899531: DATABASE PROACTIVE BUNDLE PATCH
12.1.0.2.160419 (APR2016)
- Ensure that the patches are copied over as the OS user oracle.
- Do NOT unpack the patch zips or create extra directories.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

Recovery Appliance Patching
Phase 1: Pre-Patch Validations

• The pre-install script helps to check if the patches are needed.
[root@host1 radump]# perl ra_preinstall.pl --config_xml=/u01/ra_install/linux-

x64/ZDLRAConfigurations/ZDLRA_INSTALL-zdlraadm.xml --oeda_dir=/u01/ra_install/linux-x64/
FOUND OLD PATCHES RUNNING THROUGH TO REPLACE
[root@host1 radump]#
• The upgrade script performs pre-patch validations first.

– Checks open incidents on the system a ble
fe r
– Validates that all services are up an s
n - t r
a no
[root@host1 bin]# /opt/oracle.RecoveryAppliance/bin/ra_update.pl --patch_type=upgrade
h a s
m ) ideฺ
• If phase 1 fails, you should correctฺcthe o issues
u it identified.
g t G
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Validations
Pre-Patch li
b
k e empre-install script returns “FOUND OLD PATCHES RUNNING THROUGH TO REPLACE”
The
a when the Recovery Appliance system needs to be patched.
After the pre-install verification, update your RPM on all compute nodes by using the following
commands as root:
[root@host ~]# rpm -e ra_automation;
[root@host ~]# rpm -ivh /radump/ra_automation-12.1.1.1.7-
RELEASE.x86_64.rpm
If root authentication between compute nodes isn't configured, the upgrade script sets that
up for the upgrade. This authentication requires entering the root password once for each
compute node.
Open incidents should be reviewed, and if possible resolved prior to patching. Patching won't
generally resolve open incidents, and could actually have issues because of them.
The upgrade script validates that all services are up. This is done to start from a known
working state.

If phase 1 fails, you should be able to correct issues that are generally related to open
incidents, or if a service isn't up correctly. Ensure all services are up and continue to phase 2.
Example output of Phase 1 Pre-Patch Validation:
[root@host1 bin]# /opt/oracle.RecoveryAppliance/bin/ra_update.pl --
patch_type=upgrade
Created log file /opt/oracle.RecoveryAppliance/log/ra_update.log
Welcome to RA Update.
Thu Jan 14 13:08:13 2016: Welcome to RA Update.
Thu Jan 14 13:08:13 2016:
RA Update Start
Thu Jan 14 13:08:13 2016: Patching Start
Thu Jan 14 13:08:16 2016: Pre Patch: Validating Recovery Appliance stack is
online
a ble
Thu Jan 14 13:08:16 2016: Check service status [crs]: Online.
fe r
ans
Thu Jan 14 13:08:17 2016: Check service status [db]: Online.
n - t r
Thu Jan 14 13:08:17 2016: Check service status [rep_dbfs]: Online. o
… s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Phase 2: Patch Installation
• Creates a temporary rasys password
• Stops all services.
• Applies and configures the DB BP patch, RA patch, and RA updates.

• Monitor by using tail –f to view additions to the
/opt/oracle.RecoveryAppliance/log/ra_update.log file
[root@host1 log]# tail -f ra_update.log
Checking VIP configuration.
Checking VIP Subnet configuration.
Check for VIP Subnet configuration passed.
Checking VIP reachability
a ble
Check for VIP reachability passed.
fe r
Sun May 29 17:24:04 2016: vices host2 validated. OK.
an s
Sun May 29 17:24:04 2016: Child oracle: Sun May 29 17:24:04 2016: host2 validated.
n - t r
no
OK.
Sun May 29 17:24:04 2016: Validate post patch CRS environment: Complete
Sun May 29 17:24:04 2016: End RA Update: OK
s a
h a
) ideinฺthe log:
• If phase 2 fails, you should be able to view this m
co t Gu
message
g ฺ
!!! NOTICE NOTICE NOTICE !!!
a n kn den
nb s Stu
If ra_update.pl fails anytime after NOW use --restart when re-running.
i
!!! NOTICE NOTICE NOTICE !!!
n o i
u e t h
a a kb se t
l o ( cen
Patch b e l
Installation li
k e em patch installation, the process creates a temporary password for the rasys user. This
During
a is done so that you can access the system as rasys, and also to block others from changing
the system during the upgrade.
All services stop because the patching processing requires a complete service outage. If
there is any conflict, the new patches roll back. After that the upgrade script applies and
configures the database bundle patch and the Recovery Appliance patch and updates.

Sample ra_upgrade.pl script execution output:
[root@host1 bin]# /opt/oracle.RecoveryAppliance/bin/ra_update.pl --
patch_type=upgrade
Created log file /opt/oracle.RecoveryAppliance/log/ra_update.log
Welcome to RA Update.
…
<Phase 1 output here>

…
Thu Jan 14 13:19:05 2016: Starting opatchauto on host2
Thu Jan 14 13:19:05 2016:
!!! Patching will take up to 30 minutes !!!
Thu Jan 14 13:35:52 2016: Complete opatchauto on host2
Thu Jan 14 13:35:52 2016: Datapatch will be run in the end.
a ble
fe r
Thu Jan 14 13:35:55 2016: Starting opatchauto on host1
ans
Thu Jan 14 13:35:55 2016:
n - t r
!!! Patching will take up to 30 minutes !!!
a no
Thu Jan 14 13:47:46 2016: Complete opatchauto on host1
h a s
m e ฺ
)in theidend.
Thu Jan 14 13:47:46 2016: Datapatch will be run
ฺ o GSkipping.
capply. u
Thu Jan 14 13:49:28 2016: No GI patches to g
kn den t
Thu Jan 14 13:49:28 2016: Relockingan
nb CRSs Stu
CRS
n
Thu Jan 14 13:54:35 2016: Relockedi o i
u
@ us e t h
Thu Jan 14 13:54:35 2016:
l l o Installing the Recovery Appliance Patch
k
Thu Jan 14 13:54:58
a be2016: e o
tInstalled the Recovery Appliance Patch
Thu Jan 14 13:54:58
l o ns Patching Complete
(a ce2016:
b
Thu Jane14 li 2016: Enabling zdlra-name
l 13:54:58
ak e…em
Thu Jan 14 14:09:28 2016: Validating post patch CRS environment
Thu Jan 14 14:10:24 2016: Validate post patch CRS environment: Compete
Thu Jan 14 14:10:25 2016: Post Patch: Checking for EM Agents :
Thu Jan 14 14:10:25 2016: Post Patch: EM Agent Checks: Complete
Thu Jan 14 14:10:25 2016: End RA Update: OK
[root@host1 bin]#
If phase 2 fails carefully review all current logs, correct the issue, and restart the script using –
restart: /opt/oracle.RecoveryAppliance/bin/ra_update.pl –restart –
patch_type=upgrade;

• Phase 3
– Restarts Services
– Re-enables the customers rasys password
SQL> SELECT STATE FROM RASYS.RA_SERVER;
STATE
------------
ON
SQL>
a ble
fe r
• The user performing the installation should then run the a n s
n -t r
cleanup steps. o
a nyou paused.
• Re-enable any upstream replication serversasthat
) h eฺ
com uid
• Re-enable any tape job that you paused.
g ฺ t G
k n n
n b an tude
u nio this S
l l o @ use
be e to
a k
l o (a cens
el
Post Upgrade
b li
k e empatch process automatically restarts Recovery Appliance services. The user performing
The
a the installation should then validate the system and service access, and confirm that services
are functioning.
You should also verify that the Recovery Appliance system is running by querying the
RASYS.RA_SERVER table and verifying that the STATE column value is ON.
On each compute node, clean up the patch files used during the upgrade by using the
following example commands:
[root@host1 ~]# rm -rf /radump/patch
[root@host1 ~]# rm -rf /opt/oracle.RecoveryAppliance/data/upgrade
[root@host1 ~]# rm /radump/ra_automation-12.1.1.1.4-
RELEASE.x86_64.rpm

---And again on the second node---
[root@host1 ~]# rm -rf /radump/patch
[root@host1 ~]# rm -rf /opt/oracle.RecoveryAppliance/data/upgrade
On the first compute node, clean up the zip files by using the following example commands:
[root@host1 ~]# rm /radump/p6880880_121010_Linux-x86-64.zip

[root@host1 ~]# rm /radump/p22899531_121020_Linux-x86-64.zip
[root@host1 ~]# rm /radump/E37457_01.zip
[root@host1 ~]# rm /radump/README.txt
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

• Exadata Storage Server patches:

– Are complete software images
– Maintain consistency across all cell components

– Are released independently from Recovery Appliance
Database patches
– Have dependencies on other software and firmware versions
• Non-rolling patch requirements are mandatory for Exadata
Storage Cells, RAC Compute Nodes, and RA Server. r a ble
n s fe
n - tra
• Firmware levels in Exadata Storage Server are maintained
automatically.
a no
• No additional software, RPMs, and so on a
must
h s be installed on
m ) ideฺ
Exadata Storage Server. ฺco n
u
g nt G
k
an tude
n b
u nio this S
l l o @ use
be e to
a k
l o (a cens
Patching
b l
eOracle li
Exadata Storage Servers
k e em for Exadata Storage Server are provided as complete software images that contain
Patches
a updates to the Linux operating system, cell server software, InfiniBand software, and
component firmware. This ensures that all the software and firmware components on an
Exadata Storage Server remain consistent with each other.
Exadata Storage Server patches are supplied independent of Recovery Appliance Database
Server patches. However, an Exadata Storage Server patch may require a specific Oracle
Database Server patch level, or database server firmware or operating system version.
Details are provided in the patch documentation and in My Oracle Support note 888828.1.
Exadata Storage Server patches can generally be applied in a rolling manner while databases
continue running, but for the Recovery Appliance, they currently cannot be applied so.
Patches may also include instructions for parallel (non-rolling) installation on multiple Exadata
Storage Servers.
Exadata Storage Server automatically maintains firmware within the server. Firmware levels
are periodically checked while Exadata Storage Server is running, and the correct firmware is
automatically applied to components when the server restarts. Disk firmware is also
automatically updated when a disk is replaced.

Oracle recommends that no additional software, whether RPMs or otherwise, should be
installed on Exadata Storage Server.
Note: Do not upgrade to Exadata Storage Server software 12.1.2.3.2 at this time. This release
is not certified at this time. Only Exadata 12.1.2.3.1 release and patch (24306177) (Doc ID
2124734.1) is certified with Recovery Appliance Patch 22552579: ZERO DATA LOSS
RECOVERY APPLIANCE: 12.1.1.1.7 and Database Patch 22899531: DATABASE
PROACTIVE BUNDLE PATCH 12.1.0.2.160419 (APR2016)
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

• Exadata Storage Server Patch

– Download and stage the patch on one of the database servers
– Check the prerequisites by using:

# cd <patch_dir>
# ./patchmgr –cells <cell_group> -patch_check_prereq
– Apply the patch by using:
a ble
# ./patchmgr –cells <cell_group> -patch [-rolling]
fe r
Non-Rolling an s
1. Patch n - t r
Cell 1
a no
patchmgr
1. Patch h a s
Cell 2 m ) ideฺ
1. Patch ng
ฺco t Gu
Cell nnk d e n
a tu (Doc ID 2124734.1)
b patchS(24306177)
nand
i o
Exadata 12.1.2.3.1 release
n i s
u e t h
a a kb se t
l o ( cen
e l
OraclebExadata li Server Patch
Storage
k e em Storage Server patches are typically installed by using a script called patchmgr,
Exadata
a which is supplied with the patch. They may be installed in only one way: non-rolling.
A non-rolling patch is applied to all storage servers simultaneously with databases offline.
This approach requires down time, but may be preferred if a scheduled maintenance window
exists for the system. Because all of the storage servers are patched in parallel, the whole
patch may take approximately two hours to complete.
In either mode, a single invocation of patchmgr can be used to patch all of the Exadata
Storage Servers.
In all cases, ensure that you follow the patch-specific instructions for using patchmgr.

Other components in Exadata that have software or firmware:

• InfiniBand switches: Typically applied by using patchmgr
• Power Distribution Units (PDUs)

• Keyboard, Video, and Mouse (KVM) switch
• Cisco Ethernet switch
• QLogic QLE8362 QLogic 2-port 16Gb Fibre Channel Adapter
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Other b e l l o
Components liPatching
k e emsoftware and firmware on all components must always be maintained in accordance with
The
a the guidelines published in My Oracle Support note 888828.1. In particular, this is critical for
the InfiniBand switches. For other components, there may be less urgency to keep up with the
latest patches.
If the Tape option is included in Recovery Appliance, follow the QLogic Driver Download
Instructions from this MOS support note: Zero Data Loss Recovery Appliance Supported
Versions (Doc ID 1927416.1)

Agenda


a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Recovery Appliance Oracle Enterprise Manager Plug-In
Streamlined enterprise-grade monitoring

• Centralized console to monitor multiple Recovery Appliances
along with other data center assets by using Enterprise

Manager
• New Recovery Appliance target and system page to provide
system view, status, and so on
• Hierarchical viewer to show relationships
r a ble
• Full support for all Recovery Appliance hardware versions sfe
tra n
o n -
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
el
OraclebEnterprise li
Manager Plug-In
k e emOracle Recovery Appliance plug-in for Enterprise Manager Cloud Control features
The
a monitoring dashboards and streamlined target discovery.
See Doc ID 1929507.1 for information about the plug-ins and patches required for Zero Data
Loss Recovery Appliance management, monitoring, and protected database management.

Recovery Appliance Oracle Enterprise Manager Plug-In
Main Pane
Navigation
Pane
a ble
fe r
an s
n - t r
OEM 12c
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den OEM 13c
b t u
n i on is S
@ u e th
( a ak nse
ello
OraclebEnterprise lice Recovery Appliance Dashboard
Manager
k e emdashboard for the Current Status of the Backup Environment includes these sections:
The
a • Summary of current and recent activity
• Protected database backup/recovery issues
• Daily data sent / received
• Performance
• Media Manager status
• Replication status to / from
• Storage Location status
• Incidents and Events

Agenda


a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Recovery Appliance Powering Off, Powering On
Power-off sequence:
1. Check the status of Recovery Appliance services and shut them down.
SQL> SELECT state FROM ra_server;
2. If OSB is configured, stop the service. STATE

------------ SQL> exec dbms_ra.shutdown;
ON
# $GRID_HOME/bin/crsctl status res osbadmin
NAME=osbadmin
TYPE=cluster_resource
# $GRID_HOME/bin/crsctl stop res osbadmin
TARGET=ONLINE
3. Stop DBFS.
$ $GRID_HOME/bin/crsctl status res ob_dbfs rep_dbfs $ $GRID_HOME/bin/crsctl stop res ob_dbfs rep_dbfs
NAME=ob_dbfs CRS-2673: Attempting to stop 'ob_dbfs' on 'host2'
TYPE=local_resource CRS-2673: Attempting to stop 'rep_dbfs' on 'host1'
TARGET=ONLINE , ONLINE CRS-2673: Attempting to stop 'ob_dbfs' on 'host1'
le
STATE=ONLINE on host1, ONLINE on host2 NAME=rep_dbfs CRS-2673: Attempting to stop 'rep_dbfs' on 'host2'
b
TYPE=local_resource CRS-2677: Stop of 'rep_dbfs' on 'host1' succeeded
a
TARGET=ONLINE , ONLINE CRS-2677: Stop of 'ob_dbfs' on 'host1' succeeded
STATE=ONLINE on host1, ONLINE on host2 CRS-2677: Stop of 'rep_dbfs' on 'host2' succeeded
CRS-2677: Stop of 'ob_dbfs' on 'host2' succeeded
fe r
4. Stop the Oracle Cluster Database.
an s
srvctl status database -d zdlradb
n - t r
no
Instance zdlradb1 is running on node host1
Instance zdlradb2 is running on node host2 $ srvctl stop database -d zdlradb
s a
5. Stop the Oracle Clusterware Stack.
h a
) ideฺ
m
# $GRID_HOME/bin/crsctl stop cluster -all
co t Gu
CRS-2673: Attempting to stop 'ora.crsd' on 'host1'
ฺ
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on ‘host1’
g
…
a n kn (CRS).
6. On each computer, stop the Cluster Ready Services
d e n
nb s Stu
# $GRID_HOME/bin/crsctl stop crs
o
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'host2'
u n i h i
CRS-2673: Attempting to stop 'ora.crf' on 'host2'
CRS-2673: Attempting to stop 'ora.mdnsd' on ‘host2'
…
e t
a a kb se t
l o ( cen
e l
OraclebRecovery li
Appliance Power Sequence
k e emslide lists the recommended sequence for powering the Recovery Appliance off and on in
The
a non-emergency situations.
When performing either sequence, ensure that each step is completely finished before
moving to the next step. Failure to execute steps in the proper order may result in the
Recovery Appliance not functioning correctly.
When powering on the Exadata Storage Servers and the Recovery Appliance database
servers, power can be applied by pressing the power button at the front of each server, or
remotely using the ILOM interface for each server. The ILOM can be accessed by using the
web console, the command line interface (CLI), IPMI, or SNMP. For example, you can start a
server from the ILOM CLI by executing the start /SYS command.
To power the rack on or off, use the switches located on the PDUs, which are located at the
rear of the rack.

7. Power down the servers: first compute nodes, then storage

nodes.
# shutdown -h -y now
dcli -l root -g cell_group shutdown -h -y now
8. Power down the network switches and rack by using the

PDUs.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
OraclebRecovery li
k e em Use the dcli utility to run the shutdown command on multiple servers simultaneously.
Note:
a Do not run dcli from a server that will be powered off by the command.

Power-on sequence:
• Rack, including network switches: Apply power for a few
minutes before proceeding.

• Allow 5 to 10 minutes for the storage servers to start all
services. Ensure that they finish initializing before you start the
compute servers.
• When a compute server is powered on, the operating system
r a ble
and Oracle Clusterware start automatically. Oracle fe
n s
Clusterware then starts all resources that are configured
n - tra to
start automatically.
a no
• Power on servers remotely (if necessary).has ฺ
m ) ide
ฺ o
c t Gu
g
kn-U root n power on
a n
# ipmitool -H ra01cel01-ilom
d echassis
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
OraclebRecovery li
k e emcan use the Oracle ILOM interface to power on the Recovery Appliance servers remotely.
You
a To access Oracle ILOM, use the web console, the CLI, intelligent platform management
interface (IPMI), or simple network management protocol (SNMP).

Agenda

• Enterprise Manager Plug-in installation

a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Replacing a Damaged Physical Disk
1 Determine the damaged disk.

CellCLI> LIST PHYSICALDISK DETAIL

name: 20:5
diskType: HardDisk
...
slotNumber: 5
status: failed
Blue LED indicates disk is ready for

removal LIST PHYSICALDISK NORMAL
a ble
Replace the fe r
2 physical disk.
an s
n - t r
3 a
Monitor ASM to confirm the readdition of the disk.
no
h a s
m ) ideฺ
SQL> SELECT NAME, STATE FROM V$ASM_DISK
g ฺ co t Gu
kn den
SQL> SELECT * FROM GV$ASM_OPERATION
a n
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e a Damaged
Replacing li Physical Disk
b
k e em a physical disk, either a hard drive on a high-capacity cell or a flash drive on an
Replacing
a Extreme Flash cell, due to a problem or failure is probably the most likely hardware
maintenance operation that an Exadata Storage Server may require. Because Exadata uses
ASM redundancy, the procedure to replace a problem disk is quite simple.
The first step requires that you identify the problem disk. This could occur in several ways, as
given below:
• For a failed physical disk or a physical disk with poor performance, the blue ready-to-
remove LED is automatically switched on. For a physical disk with predictive failure
status, the blue ready-to-remove LED is automatically switched on after all grid disks
present on the physical disk are dropped from ASM and redundancy is fully restored. In
both cases, the blue LED indicates that it is safe to remove and replace the affected
disk.
• Hardware monitoring using ILOM may report a problem disk.

• If a disk fails, an Exadata alert is generated. The alert includes specific instructions for
replacing the disk. If you have configured the system for alert notifications, the alert will be
sent to the designated email address or SNMP target. The LIST ALERTHISTORY
command could also be used to identify a failed disk.
• The LIST PHYSICALDISK command may identify a disk reporting an abnormal status.
Even if the disk is still functioning, Exadata may detect a problem that may be a precursor
to a disk failure and place the disk in a predictive failure state.
When a failed disk is detected, the Oracle ASM disks associated with the grid disks on the
physical disk are automatically dropped with the FORCE option, and an Oracle ASM rebalance
follows to restore the data redundancy. This process is known as proactive disk quarantine.
After you have identified the problem disk, you can replace it. When you remove the disk, you
will get an alert. When you replace a physical disk, the disk must be acknowledged by the RAID
controller before it can be used. This does not take a long time, and you can use the LIST
PHYSICALDISK command to monitor the status until it returns to NORMAL.
a b
The grid disks and cell disks that existed on the previous disk in the slot will be automatically re-
le
fe r
created on the new disk. If these grid disks were a part of an Oracle ASM disk group with
ans
t r
NORMAL or HIGH redundancy, they will be added back to the disk group and the data will be
n -
no
rebalanced based on disk group redundancy and the asm_power_limit parameter.
a
Recreating the ASM disk and rebalancing the data may take some
h a s time to complete. You can
) monitor
monitor the progress of these operations within ASM. You can
m d e ฺthe status of the disk as
co t You
reported by V$ASM_DISK.STATE until it returns to NORMAL.
ฺ i
ucan also monitor the
g G
rebalance progress by using GV$ASM_OPERATION.
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o li
m b
a kee

Safe Disk Removal
Sometimes a disk must be removed even though the system

detects no fault with it. You can do so by running:
CellCLI> ALTER PHYSICALDISK { disk_1 [ , disk_n ] } DROP FOR REPLACEMENT
• This disables the disk and switches on the blue ready-to-

remove LED only if it is safe to remove the disk.
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l o
e Removalli
Safe Disk l
b
em
eSometimes a disk must be replaced proactively, even though its blue ready-to-remove LED is
ak switched off. For example, the CALIBRATE command may identify a disk delivering
abnormally low throughput or I/Os per second (IOPS).
Nonetheless, removing the disk could lead to a disk group force dismount if the removed disk
has the only copy of some pieces of data, or a cell crashes due to loss of its OS if the disk is
the only system disk.
To mitigate this, Exadata release 11.2.3.3.0 adds a new command.
ALTER PHYSICALDISK ... DROP FOR REPLACEMENT checks whether the specified
physical disks can be removed safely, and if so, the disks are disabled and the blue ready-to-
remove LEDs are switched on. This command replaces checks that were previously required
to ensure that disks could be removed safely. By using this command, disk replacements are
simplified and problems are avoided.
In addition, ALTER PHYSICALDISK ... REENABLE FORCE re-enables normal physical disks.
This command can be used to re-enable physical disks in situations where
ALTER PHYSICALDISK ... DROP FOR REPLACEMENT was executed, but the administrator
does not want to remove the disks.

Replacing a Damaged Flash Card
1 Determine the damaged flash card.
CellCLI> LIST PHYSICALDISK DETAIL

name: FLASH_5_3
diskType: FlashDisk
...
slotNumber: "PCI Slot: 5; FDOM: 3"
status: failed
a ble
fe r
2 Power off 3 Replace the 4 Power on
an s
the cell. flash card.
n - t r
the cell.
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Replacing l l o
e a Damaged li Flash Card
b
k e em a damaged flash module is similar to identifying a damaged physical disk.
Identifying
a Hardware monitoring using ILOM or a drop in performance indicated by the CALIBRATE
command may indicate a problem. If a failed FDOM is detected, an alert is generated.
A bad flash module results in a decreased amount of flash memory on the cell. The
performance of the cell is affected proportional to the size of the flash memory lost, but the
database and applications are not at risk of failure.
As shown in the slide, a damaged flash module can also be reported by using the LIST
PHYSICALDISK DETAIL command. The slotNumber attribute shows the PCI slot and the
FDOM number. In this example, the status attribute indicates a device failure.
Although the PCI slots in an Exadata Storage Server support hot replacement, it is
recommended to power off the cell while replacing a damaged flash card.
After replacing the card and powering on the cell, no additional steps are required to re-create
the Smart Flash Cache and Smart Flash Log areas on the new flash modules.

Moving All Disks from One Cell to Another
Original New Original New

1. Back up the following files and directories:

– Files: /etc/hosts, /etc/modprobe.conf
– Directories: /etc/sysconfig/network, /etc/sysconfig/network-scripts
2. Deactivate the grid disks: CellCLI> ALTER GRIDDISK ALL INACTIVE
3. Shut down the cell.
4. Move the disks, flash cards, disk controller, and CELLBOOT USB flash drive from the
original cell to the new cell.
–
–
Ensure that the system disks occupy the same first two slots.
Ensure that the flash cards occupy the same PCI slots. a ble
5. Power on the new cell. fe r
a n s
6.
7.
Check the operating system
Restart the cell services:
configuration.
n -t r
o
CellCLI> ALTER CELL RESTART SERVICES ALL
8. Activate the grid disks: an
CellCLI> ALTER GRIDDISK ALL ACTIVE
s
9. Validate the configuration: ha CONFIGURATION
CellCLI> ALTER CELL VALIDATE
) e ฺ
10. Activate the ILOM for ASR. m
co t Gu i d
g ฺ
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Moving l l o li One Cell to Another
eAll Disks from
b
k e emmay need to move all drives from one storage server to another. This may be necessary
You
a when there is a chassis-level component failure, or when troubleshooting a hardware
problem. To move the drives, perform the following steps:
1. If possible, back up /etc/hosts, /etc/modprobe.conf, and the files in
/etc/sysconfig/network and /etc/sysconfig/network-scripts. This is
mainly a precautionary step and is also useful if you want to move the disks back to the
original chassis.
2. If possible, deactivate the grid disks by using ALTER GRIDDISK ALL INACTIVE.
Also, ensure that the Oracle ASM disk_repair_time attribute is set sufficiently long
enough so ASM does not drop the disks before the grid disks can be activated in
another Exadata Storage Server.
3. Shut down the original server.

4. Move the hard disks, flash cards, disk controller card, and CELLBOOT USB flash drive to
the new server.
- Caution: Ensure that the first two disks, which are the system disks, are in the same
first two slots. Also ensure that the flash cards are installed in the same PCI slots.
Failure to do so causes the Exadata Storage Server to function improperly.
5. Start the new server. The cell operating system should be automatically reconfigured to
suit the new server hardware.
6. Check /etc/hosts, /etc/modprobe.conf, and the files in

/etc/sysconfig/network and /etc/sysconfig/network-scripts. If any files
are corrupted, restore them from your backup or manually re-create them by using a good
cell as a guide. Also, use the ifconfig command to confirm the hardware addresses for
the Ethernet interfaces (eth0, eth1, and so on). Compare the Ethernet interface
hardware addresses with the HWADDR value stored in the configuration files (ifcfg-
eth0, ifcfg-eth1, and so on) in the /etc/sysconfig/network-scripts directory.
If necessary, edit the configuration files so that the HWADDR value matches the a ble
fe r
corresponding ifconfig command output.
ans
7. Restart the cell services by using ALTER CELL RESTART SERVICES ALL.
n - t r
8. Activate the grid disks by using ALTER GRIDDISK ALL ACTIVE. a no
a s
9. Validate the cell configuration by using ALTER CELL VALIDATE CONFIGURATION.
h
10.If required, activate the ILOM interface of the newocellm )for Auto
i d eฺ Request (ASR).
Service
g ฺ c t Gu
If you are using ASM redundancy and the procedure
a n kn isdecompleted
n before the amount of time
specified in the DISK_REPAIR_TIME ASM
n b S t u
initialization parameter, then the ASM disks will be
automatically brought back online n
u io updated
and
h is with any changes made during the cell outage.
@ use t
l l o
a k be e to
l o (a cens
b el li
m
a kee

Using the Cell Software Rescue Procedure
• Every Exadata Storage Server is equipped with a CELLBOOT USB

flash drive to facilitate cell rescue:
– Required if both system disks fail simultaneously or are corrupt

– Should be used with extreme caution
• To perform cell rescue:
1. Connect to the Exadata Storage Server using the console.
2. Boot the cell, and as soon as you see the Oracle Exadata splash
screen, press any key on the keyboard.
r a ble
3. From the boot options list, select the last option,
n s fe
CELL_USB_BOOT_CELLBOOT_usb_in_rescue_mode. Thenrapress
Enter. o n -t
s
4. Select the rescue option, and proceed with the rescue. an
a ฺ
) hthe
5. At the end of the rescue process, ensurem
o
that
i e boots from the
cell
d
system disks. g ฺc t G u
n n
6. Reconfigure the cell. ank de
nb s Stu
n i o i
u e t h
a a kb se t
l o ( cen
l
e Rescue
Cell Software
b li Procedure
em Storage Server maintains mirrored system areas on separate physical disks. If one
eExadata
ak system area becomes corrupt or unavailable, the mirrored copy is used to recover.
In the rare event that both system disks fail simultaneously, you must use the rescue
functionality provided on the CELLBOOT USB flash drive that is built into every Exadata
Storage Server. The rescue procedure may also be required to recover from a file system
corruption or a corrupt boot area.
It is important to note the following when using the rescue procedure:
• Use extreme caution when using this procedure, and pay attention to the prompts. The
rescue procedure can rewrite some or all of the disks in the cell. If this happens, you can
lose the contents of those disks. You should use the rescue procedure only with
assistance from Oracle Support Services.
• The rescue procedure does not destroy the contents of the data disks or the contents of
the data partitions on the system disks unless you explicitly choose to do so during the
rescue procedure.

• The rescue procedure restores the Exadata Storage Server software to the same
release. This includes any patches that existed on the cell as of the last successful boot.
• The following is not restored using the rescue procedure:
- Some cell configuration details, such as alert configurations, SMTP information,
and administrator email address. Note that the cell network configuration is
restored, along with SSH identities for the cell, and the root, celladmin, and
cellmonitor users.
- ILOM configurations. Typically, ILOM configurations remain undamaged even in

case of Exadata software failures.
• The rescue procedure does not examine or reconstruct data disks or data partitions on
the system disks. If you have data corruption on the grid disks, do not use the rescue
procedure. Instead, use the database backup and recovery procedures.
The following rescue options are available for the rescue procedure:
• Partial reconstruction recovery: During partial reconstruction recovery, the rescue a ble
fe
process re-creates partitions on the system disks and checks the disks for the existence r
ans
n -
boots successfully, you use the CellCLI commands, such as LIST CELL DETAIL, to
r
of a file system. If a file system is discovered, the process attempts to boot. If the cell
t
a no
verify that the cell is usable. You must also recover any data disks, as appropriate. If the
h a
boot fails, you must use the full, original build recovery option.
s
m ) ideฺ
g ฺ co t Gu
• Full, original build recovery: This option rewrites the system area of the system disks to
restore the Exadata software. It also allows you to erase any data on the data disks, and
data partitions on the system disks. a n kn den
i o nb s Stu
• Re-creation of the CELLBOOT USB flash drive: This option is used to make a copy of
the CELLBOOT USB flash drive. u n t h i
l lo@ us e
To perform a rescue by
k b e
using the
t oCELLBOOT USB flash drive:
a
(athe Exadata
1. Connect to e
ns Storage Server by using the console.
l o c e
li as soon as you see the Oracle Exadata splash screen, press any key
elthe cell, and
2. Boot
b
e e m on the keyboard. The splash screen remains visible for only five seconds.
a k 3. In the list of boot options, scroll down to the last option,
CELL_USB_BOOT_CELLBOOT_usb_in_rescue_mode, and press Enter.
4. Select the rescue option, and proceed with the rescue.

5. When prompted at the end of the rescue process, proceed as follows:
a. Choose to enter a shell. Do not select the reboot option at this point.
b. Log in to the shell by using the rescue root password.
c. Run the reboot command from the shell.
d. During reboot, but before you see the Oracle Exadata splash screen, press F8 to
access the boot device selection menu.
e. Select the RAID controller as the boot device.

6. After a successful rescue, you must reconfigure the cell to return it to the pre-failure
configuration. This may include items such as cell configuration parameters, IORM
configuration and metric thresholds. If you selected to preserve the data when prompted
by the rescue procedure, the cell disks are automatically imported during the rescue
procedure. If you selected not to preserve the data, you should create new cell disks
and grid disks.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

Summary
In this lesson, you should have learned to:

• Identify various maintenance tasks
• Power On and Power Off the Recovery Appliance

• Patch the Recovery Appliance
• Describe how the Oracle Enterprise Manager plug-in works
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

ake
me
bel
l o
a k
li
l l o
(a cens
u
be e to
n
@ use
b a n
nio this
k
S t
ฺ c
ud
ng ent
)
s
om Guid
ha eฺ
an
o n - t r an
sfe r a b
le
11
Diagnostics and
Troubleshooting r a ble
s fe
- t r an
o n
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake
Objectives

• Describe the Recovery Appliance diagnostic and troubleshooting
tasks
• Describe the main areas for troubleshooting
• Use diagnostic and troubleshooting utilities
• Describe Automatic Service Requests (ASRs)
• List Oracle Enterprise Manager (OEM) plug-in monitoring
r a ble
capabilities fe
n s
• Describe replication issues
n - tra
o
• List Media Manager issues an s
h a
) ideฺ
m
co t Gu
g ฺ
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li
m be
e
ake

Agenda
• Main Areas for Issues

• Diagnostic Collection
• OEM Diagnostic Framework

• Automatic Service Request
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Recovery Appliance Main Areas for Issues
• The Recovery Appliance topology involves four main areas as

possible sources of issues:
– Protected database systems and available CPU resource and

storage throughput
– Backup or Ingest Network connecting the protected database
systems to the Recovery Appliance
– Recovery Appliance and its associated CPU resources and
storage throughput.
r a ble
s
– Tape Library (when used) that can use the Ingest Networknorfe
a
Private Network through fibre channel n-tr no
s a
h a
) ideฺ
m
co t Gu
g ฺ
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o i areas where a performance issues can be identified.
1.
m be are fourlmain
There
e 2. It is important to identify the area where the problem was initially found and any
ake additional errors across the four main areas.
3. Having an original baseline to compare with the suspicious situation will help
performance investigations.
4. Record the sequence of the steps executed to reproduce the issue and their outcome:
Elapsed time, messages (warning, error, and so on).
5. Provide the information about end-to-end architecture and network topology.
Protected Database
• Connectivity to the Recovery Appliance (Setup, Configuration, and so on).
• Server configuration and capacity planning report (Exawatcher, Oswatcher).

6. Recovery Appliance
- Run exacheck (mandatory) with the version that is shipped with the Recovery
Appliance. Optionally, you can collect the Capacity Planning Report (From OEM,
refer to Lesson 12: Reporting with BI Publisher for Recovery Appliance OEM
reports).
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
em
ake

Generic Troubleshooting Questions
• Questions to formulate:
– Third-Party products?
– Network security attributes in your configuration?

– RMAN Compression?
– Any Encryption: media and network?
– Replication?
– Tape backups? le
r a b
s fe
- t r an
no n
s a
h a
) ideฺ
m
co t Gu
g ฺ
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li questions:
e the following
Formulate
b
k e em a detailed problem description and describe the impact of the problem.
Provide
a • What Recovery Applinace version are you running?
• Do you have any third-party products installed on the Recovery Appliance? What are
they?
• Are there any network security attributes in your configuration: firewalls, ACL, etc.?
What are they? Who is the vendor? What is the configuration?
• Are you using any RMAN compression?
• Are you using any encryption? Are you using RMAN, TDE, or other vendor encryption
product? What is the configuration?
• Are you using Oracle *Net encryption? Are you using TCPS native or any external
authentication?
• Are you using tape backups and which product? If not OSB, did you engage the third-
party vendor?

Agenda


a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Diagnostic Collection
• Trace File Analyzer (TFA) Collector

[oracle@host1 ~]# TFA_HOME=/u01/app/12.1.0.2/grid/tfa/host1/tfa_homegrep TFA_HOME= /etc/init.d/init.tfa oracle@host1 ~]# $TFA_HOME/bin/tfactl

[oracle@host1 ~]# export TFA_HOME=/u01/app/12.1.0.2/grid/tfa/host1/tfa_home tfactl> events
[oracle@host1 ~]# $TFA_HOME/bin/tfactl print status
.--------------------------------------------------------------------------------------------------. Output from host: host1
| Host | Status of TFA | PID | Port | Version | Build ID | Inventory Status | ------------------------------
+-------------+---------------+------+------+------------+----------------------+------------------+ Summary :
| host1 | RUNNING | 2849 | 5000 | 12.1.2.6.3 | 12126320160104141621 | COMPLETE | =========
| host1 | RUNNING | 2666 | 5000 | 12.1.2.6.3 | 12126320160104141621 | COMPLETE | INFO : 0
'-------------+---------------+------+------+------------+----------------------+------------------' ERROR : 0
[oracle@host1 ~]# WARNING : 0
Output from host: host2
[oracle@host1 ~]# $TFA_HOME/bin/./tfactl diagcollect -from "Sep/02/2016" -to "Sep/02/2016 13:00:00" ------------------------------
Collecting data for all nodes Sep/02/2016 20:16:24:[crs.host2]:2016-09-02
Scanning files from Sep/02/2016 00:00:00 to Sep/02/2016 13:00:00 20:16:24.971 [OCTSSD(25068)]CRS-2409: The
Collection Id: 20150402150419host1 clock on host host2 is not synchronous with
Repository Location in host1: $TFA_HOME/repository the mean cluster time. No action has been
2016/09/02 15:04:21 EDT: Collection Name: tfa_Wed_Sep_2_15_04_19_EDT_2016.zip taken as the Cluster Time Synchronization
2016/09/02 15:04:21 EDT: Sending diagcollect request to host: host2 Service is running in observer mode.
2016/09/02 15:04:22 EDT: Scanning of files for Collection in progress... Summary :
2016/09/02 15:04:22 EDT: Collecting extra files… =========
le
… INFO : 0
b
2016/09/02 15:05:22 EDT: host2:Completed Collection ERROR : 0
a
2016/09/02 15:05:22 EDT: Completed collection of zip files. WARNING : 1
Logs are being collected to: $TFA_HOME/repository/collection_Wed_Sep_2_15_04_19_EDT_2016_node_all
$TFA_HOME/repository/collection_Wed_Sep_2_15_04_19_EDT_2016_node_all/host1.tfa_Wed_Sep_2_15_04_19_EDT_2016.zip
tfactl>
fe r
$TFA_HOME/repository/collection_Wed_Sep_2_15_04_19_EDT_2016_node_all/host2.tfa_Wed_Sep_2_15_04_19_EDT_2016.zip
[oracle@host1 ~]#
an s
• Diagnostic Assistant (DA) n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Collection
Diagnostic li
b
k e emTFA and the DA provide the official methods to gather information from ZDLRA for
The
a troubleshooting any service request.
The TFA is a diagnostic collection utility used to simplify diagnostic data collection on Oracle
Clusterware/Grid Infrastructure, RAC, and single-instance database systems.
TFA Collector Features:
• It simplifies diagnostic data collections.
• A single command performs clusterwide diagnostic collection for ALL components.
• Diagnostic files “trimmed” around the incident time.
• Collected diagnostics are consolidated on a single node.
• Increased efficiency of admin staff.

• Includes and integrates the following tools that were previously packaged as the "RAC
and DB Support Tools Bundle":
- ORAchk to promote proactive maintenance
- OSWatcher, Procwatcher, SQLT, ORAtop, and DARDA to ensure that such data is
available for reactive types of issues
The DA simplifies and standardizes end-user experience with various diagnostic tools within
Oracle.
DA benefits:
• Multiple execution methods
• Menu-driven command line interface
• Web User Interface
• Direct command execution
• Enterprise Manager's Support Workbench
a b le
• Knowledge of the syntax of the various data collectors NOT needed fe r
ans
• Proactively troubleshoot by problem type, system components, common issues
n - t r
• Limited prompting (common values stored for re-use)
a no
a s
• Deployed via a single zip as well as being delivered within many product lines
h
m ) ideฺ
• Create a draft service request directly from the DA, without the need for Oracle
Configuration Manager (OCM) g ฺ co t Gu
n kn den
• Package the diagnostic collections and upload directly to an existing or draft service
a
request
i o nb s Stu
u n t h i
• Immediate access to diagnostic menus attached to a MOS Knowledge Article
l @ us
losupport e
k e
Please refer to the following
bGathering to(Docnotes for additional information: TFA Collector - Tool for
Enhanced Diagnostic
( a a n s e ID 1513912.2) and Diagnostic Assistant: Information
e
elloID 201804.2)
Center (Doc
b lic
k e em
a

Other Useful Diagnostic Tools: Exachk
• Exachk is a utility that:

– Collects data regarding Recovery Appliance component
versions and best practices

– Is used by administrators to check their environment against
supported version levels and best practices
– Is preinstalled on new Recovery Appliances
– Should be executed periodically as a part of regular Recovery
Appliance monitoring and maintenance r a ble
– Does not alter any Recovery Appliance configuration settings n s fe
n - t ra
– Is lightweight and has minimal impact on the system
o
• Exachk is NOT: s an
) ha eฺ
– A continuous monitoring tool
ฺ c om Guid
– A replacement for Enterprisekn g nt
Manager
b a n
t u de
n
u nio this S
l l o @ use
be e to
a k
l o (a cens
el dataliregarding key software, hardware, and firmware versions, along with
Exachkbcollects
k e em
information about Recovery Appliance-specific configuration best practices.
a The output assists Recovery Appliance administrators to periodically review and cross-
reference the current data for key Recovery Appliance components against supported version
levels and recommended best practices.
Exachk is preinstalled on new Recovery Appliance systems. The latest updated version of
Exachk is available for download from My Oracle Support note 1070954.1. It can be executed
as desired and should be executed regularly as a part of the maintenance program for
Recovery Appliance.
Exachk is not a continuous monitoring utility and should not be considered as a replacement
for other monitoring or alerting tools, such as Enterprise Manager or ILOM.
It is non-intrusive and does not change anything on the Recovery Appliance apart from writing
output files and some small temporary files, which are deleted after they are used. It also
offers to configure SSH user equivalence if it is not configured.

Exachk is lightweight. Therefore, the impact to the target machine is minimal. However, it is
thorough, and consequently takes some time to execute. In general, Exachk takes a few
minutes for each database server in your environment. Nonetheless, the amount of time taken
can vary depending on numerous factors, such as the number of nodes in the cluster, the
current CPU load, the current network latency, and so on.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Other Useful Diagnostic Tools: Exachk
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
Exachkbproduces anliHTML report of findings, with the most important exceptions listed first by
k e em run.The
component. report is contained in a date-stamped zip file that is produced during each
Typically, administrators will transfer the exachk output ZIP file to a desktop
a exachk
computer and expand it there to review the exachk findings.
The exachk HTML report contains a summary of findings, along with relevant details. The
detailed entries typically describe the issues in detail and provide recommendations for
remediation. A link to documentation or a link to a note in My Oracle Support may also be
included to provide further information, especially if the recommendations are not
straightforward.
The slide shows an example of the output from the exachk HTML report.
For additional information please refer to the following support note: Oracle Exadata
Database Machine exachk or HealthCheck (Doc ID 1070954.1)

Other Useful Diagnostic Tools: ExaWatcher
• ExaWatcher collects OS and network metrics on all Recovery

Appliance servers to aid the diagnosis of performance issues.
• ExaWatcher gathers data on a regular basis, invoking utilities such

as vmstat, netstat, and iostat..
• ExaWatcher runs automatically with the following default settings:
• Installation location: /opt/oracle.ExaWatcher
• Most OS commands are executed every five seconds.
— A few long-running commands have a longer interval.
a ble
– Archive location: /opt/oracle.ExaWatcher/archive fe r
an s
— 3 GB on database servers, 600 MB on storage servers
n - t r
• o
You can customize the commands that are executed,ncollection
intervals, the archive size, and more. s a
h a
• Use the GetExaWatcherResults.shoscript m i d eฺ the archive
) to search
Gu
gฺc ntlocation.
and write a set of observations tokanspecific
n b an tude
u nio this S
l l o @ use
be e to
a k
l o (a cens
ExaWatcher
b li
el is a collection of perl and shell scripts designed to collect and archive OS and
k e em metrics to aid in diagnosing performance issues. ExaWatcher replaces OSWatcher in

network
a the latest software versions. ExaWatcher operates as a set of background processes on the
server and gathers OS data on a regular basis, invoking utilities such as vmstat, netstat,
and iostat.
By default, metric snapshots for most commands are recorded every five seconds. However,
some long-running commands have a longer interval between executions. For example, the
list of files opened by current processes, as determined by the lsof command, has a
minimum collection internal of two minutes.
The default space allocation for the ExaWatcher archive is 3 GB on each database server and
600 MB on each storage server. Typically, the default allocation is large enough to store
approximately two weeks of observations on each storage server, and approximately 10
weeks of observations on each database server. ExaWatcher periodically checks the amount
of space consumed by the archive and deletes old observations to create space, if required.

ExaWatcher is installed under /opt/oracle.ExaWatcher. Metric observations are
maintained in the archive subdirectory.
For more information about ExaWatcher, including how you can use the
GetExaWatcherResults.sh script to search the archive and write a set of observations to a
specific location, and how you can customize ExaWatcher, see My Oracle Support note
ExaWatcher utility on Exadata database servers and storage cells (Doc ID 1617454.1).
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Items to Collect
[oracle@host1 ~]$ /u01/app/oracle/product/12.1.0.2/dbhome_1/OPatch/opatch
lsinventory -oh /u01/app/emagent/plug-ins/oracle.sysman.db.discovery.plug-
in_12.1.0.7.0/
Oracle Interim Patch Installer version 12.1.0.1.9
• OEM Diagnostic
Copyright (c) 2016, Oracle Corporation. All rights reserved.
Oracle Home : /u01/app/emagent/plug-
– Patch level of the agents

ins/oracle.sysman.db.discovery.plug-in_12.1.0.7.0
Central Inventory: /u01/app/oraInventory

from : /u01/app/emagent/plug-
ins/oracle.sysman.db.discovery.plug-in_12.1.0.7.0//oraInst.loc
– Patch level of the

OPatch version : 12.1.0.1.9
…
database and Recovery SQL> COL ERROR_TEXT FORMAT A40
Appliance plug-ins
SQL> SELECT INCIDENT_ID, DB_UNIQUE_NAME, ERROR_CODE, ERROR_TEXT
2 FROM RASYS.RA_INCIDENT_LOG
3 WHERE STATUS = 'ACTIVE' AND SEVERITY = 'INTERNAL';
– All internal incidents INCIDENT_ID DB_UNIQUE_NAME ERROR_CODE ERROR_TEXT

----------- -------------------- ---------- ----------------------------------------
1761673 -45168 ORA-45168: unexpected scheduler exit whi…
• Protected Databases
Diagnostic a b le
e r
$ rman target / catalog rauser11/welcome1@scao09ingest-scan3:1521/zdlra8 log=/tmp/rman_backup.log debug
f
s
trace=/tmp/rman_backup.trc
an
RMAN>
• Recovery Appliance t r
Diagnostic
$ sqlplus rasys/*****
SQL> EXEC DBMS_RA.DUMP(do_dpdump => TRUE);
o n -
an
$ ls -ltr /radump/*dat
-rw-r--r-- 1 oracle dba 24479534 Oct 2 07:47 /radump/radump_time_074708__ospid_112711_RADUMP.dat
• Tape Diagnostic s
ha eฺ
)
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
Items to
b l
eCollect: li
k e e1.m If the OEM is being used:
a a. From the Recovery Appliance and the protected database, get the patch level of
the agents. This can be obtained by running optach from the database home with
the lsinventory option and pointing to the EM agent plug-in directory.
b. Gather all internal incidents from the Recovery Appliance incident log querying the
RASYS.RA_INCIDENT_LOG table.
c. The patch level of the database plug-in and the Recovery Appliance plug-in can be
obtained by using opatch from the OMS home with the lsinventory option and
pointing to the OMS plug-ins directory.
2. From the Protected Database side, if the issue is between the protected database and
the ZDLRA:
a. Capture the output of the netstat -rna command
b. Capture the output of the opatch lsinventory -details for the both
database home and grid home (if the customer is using the GI)

c. RMAN backup and restore commands or script used
d. RMAN configuration.
e. RMAN debug trace
f. System performance reports from the problematic period if this is a performance
concern. These can be obtained by Oracle ExaWatcher (in the event of using an
Oracle Engineered System), OEM, Oracle OSWatcher or any third-party tool used
to monitor the system’s resource utilization: CPU, Memory, IO, Network, etc.
g. If this is a performance problem, provide AWR report from the protected database
during the problematic period, this can be obtained through OEM or manually.
h. The spool files from querying the following views: gv$backup_async_io and
gv$backup_sync_io.
i. Network Performance test throughput result. This test can be performed by using
the qperf utility.
3. From the Recovery Appliance side (repeat the steps below on replica appliance if the a ble
fe r
problem is related to replication):
ans
a. Capture the output of the netstat -rna command.
n - t r
no
b. The Recovery Appliance version the customer is running, this can be obtained by
a
h a s
executing the following on any ZDLRA database node as the oracle user:
) ideฺ
$ORACLE_HOME/OPatch/opatch lspatches ( for the ZDLRA database & GI)
m
ฺ co t Gu
c. Capture the output of: cat $ORACLE_HOME/rdbms/install/zdlra/zdlra-
g
software-id
a n kn den
i o nb s Stu
d. As the root user, run: rpm -qa ra_automation
e. AWR dump from the u n t
problematic h i
period that will help to identify any issue on the
l lo@Database
Recovery Appliance
e
us side.
b e t o
( a ak noutput.
f. An radumper s e On the ZDLRA database nodes execute the
e l o
lDBMS_RA.DUMP
l i ce procedure.
m bg. Recovery Appliance performance reports from the problematic period. These can
a kee be obtained by Oracle ExaWatcher or from the OEM BI - Capacity Planning

Reports.
h. A TFA/DA collection from ZDLRA for the problematic period - Make sure to include
the Storage Cells option in the gathering.

4. If Tape is configured, gather the below information as well:
a. Check the queued tasks with the OEM Copy-to-Tape Templates section or by
querying the RASYS.RA_SBT_TASK table.
b. If you are using OSB, take a snapshot of the running jobs at the problematic
period, if you are using a third-party vendor instead of the OSB, please open a
case with the vendor and provide the relevant logs for investigation.
Please refer to the following support notes for additional information:

Zero Data Loss Recovery Appliance Network Test Throughput script (Doc ID 2022086.1)
ExaWatcher utility on Exadata database servers and storage cells (Doc ID 1617454.1)
OSWatcher (Includes: [Video]) (Doc ID 301137.1)
How to Export and Import the AWR Repository From One Database to Another (Doc ID
785730.1)
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Agenda


a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

OEM Monitoring Home Page
• The Recovery Appliance home page provides a dashboard of current statuses.
– Warnings and alerts that may need user attention are prominently displayed.
• Enables you to effectively manage any issues, assign ownership, and track problems
through to resolution
– It leverages the EM incident and event notification system.
• Default and user-defined metrics trigger warnings or error messages based on
thresholds.
– EM collects metrics on key information, providing out-of-the box monitoring and alerting.
Protected Database Protected
Activity Summary Database
Recovery Status
a ble
Summary
fe r
Filter by an s
Backup Volume and
Incoming Data Rate n - t r
Performance Charts
a no
h a s
m ) ideฺ
g ฺ co t Gu Protected
n n
k de n Database
a
nb s Stu Status Filter
n i o i
u e t h
a a kb se t
l o ( cen
l
e sections
The following
b li of the Home page show monitoring information:
ak ee•m Summary: This section shows the number of databases with no issues, with alerts, and
with warnings. In Cloud Control, an alert is an indicator that a particular metric condition
has been encountered. For example, an alert might indicate that a metric threshold has
been reached.
• Media Managers and Replication: These sections show the status of copy-to-tape and
Recovery Appliance replication services.
• Protected Database Issues: This section summarizes the backup status for protected
databases, and provides a category filter so you can view which databases are affected.
• Incidents and Events: This section displays incidents and events reported for the
Recovery Appliance and all associated targets. You can filter by target and category.
You can click the Summary link to drill down to the Incident Manager to view detailed
information about the incident.

OEM Protected Database Metrics
Recovery Window Goal: 10 Days
Current Recovery Window: 6 Days

Projected Space Needed for Recovery Window Goal: 2.69 TB
a ble
Used Space: 2.5 TB Current RPO: < 1 sec !
fe r
an s
n - t r
a no
h a s Deduplication Ratio: 10:1
m ) ideฺ
Reserved Space: 7.9 TB
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Databases
The Protected li table displays information related to the database name, version,
b
em policy, database size, recovery window information, unprotected data window
eprotection
ak information, near-Zero Data Loss Recovery, errors/warnings, last complete backup,
replication and Copy-to-Tape.
You can view detailed information about the currently selected protected database by using
the tabs in the Protected Database Detail section below the Protected Databases table. The
table displays charts showing the status of each protected database on the Status tab, and a
list of Recovery Appliance incidents on the Incidents and Events tab.

OEM Incidents
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e and Events
The Incidents li section shows all incidents, events, and warnings for a Recovery
b
em
eAppliance. Click any incident to open the Incident Manager page. Incident Manager provides,
ak in one location, the ability to search, view, manage, and resolve incidents and problems
affecting your environment.

imageinfo and imagehistory
[root@host1 ~]# imageinfo
Kernel version: 2.6.39-400.264.1.el6uek.x86_64 #1 SMP Wed Aug 26

16:42:25 PDT 2015 x86_64
Image version: 12.1.2.2.0.150917
Image activated: 2015-11-07 13:46:21 -0700
Image status: success
System partition on device: /dev/mapper/VGExaDb-LVDbSys1
[root@host1 ~]#
[root@host1 ~]# imagehistory

Version : 12.1.2.1.0.141206.1
le
Imaging mode : fresh
r a b
Version
Image activation date
:
:
12.1.2.2.0.150917
2015-11-07 13:46:21 -0700
s fe
Imaging mode
Imaging status
:
:
patch
success
- t r an
[root@host1 ~]#
no n
s a
h a
) ideฺ
m
co t Gu
g ฺ
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Software l l o li
eis initially installed on Exadata storage and Recovery Appliance database servers by
b
k e em a Recovery Appliance-specific imaging process. The Exadata Storage Server patch
using
a bundles are also provided as software images.
Two utilities are provided to monitor the Recovery Appliance and Exadata Storage images:
• imageinfo: Displays information relating to the images currently installed on a server
• imagehistory: Displays historical information about all the images installed on the
server
The slide shows examples of the output displayed by imageinfo on a storage server and on
a database server.
There is significantly less output associated with a database server because database
servers have no concept of active and inactive images, and they do not contain a cell boot
USB device.
The behaviour and output of imageinfo can be modified by using a series of optional
parameters. Execute imageinfo –h to obtain a list of the available options.

Agenda


a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Automatic Service Request
• It automatically and securely opens service requests with

Oracle for common server hardware faults.
• Minimal data is collected and transmitted.

• No IP addresses are communicated.
• One-way (customer to Oracle) SSL encrypted
communications are used.
• It provides fast and accurate resolution of hardware faults
r a ble
• It offers improved availability and has less down time. nsfe
• It can be integrated with the existing monitoring tools. n - tra
a no
• The ASR Manager can send service request a snotifications via
SNMP traps to the existing monitoringmtools.
h
) ideฺ
g ฺ co t Gu
• It is included with hardware warranty
k n eand n Oracle Premier
Support for Systems. nba Stud
n
u nio this
l l o @ use
be e to
a k
l o (a cens
el Request
Auto Service
b li (ASR) is a secure, scalable, and customer-installable software solution
emis available as a feature of your Oracle hardware warranty and Oracle Premier Support
ethat
ak for Systems. The ASR software helps to resolve problems faster by using auto-case
generation for server and storage systems when specific hardware faults occur.
The ASR functionality is available for Recovery Appliance in addition to all other Engineered
Systems. ASR automatically opens service requests with Oracle Support when specific
hardware faults occur either in the Exadata Storage Servers or in the Recovery Appliance
database servers. The ASR covers the following server components: CPUs, disk controllers,
disks, flash cards, flash modules, InfiniBand cards, memory modules, system boards, power
supplies, and fans.

ASR Process
Customer Data Center Oracle Support Services
FRU replaced by
Field Engineer
Oracle Support
Engineer
Oracle Field Field Replaceable

Engineer Unit (FRU)
dispatched by
Support Engineer
Service
request
Fault routed to
occurs Support
Customer SR creation
email notification
Engineer
a ble
to customer
fe r
Oracle Case
an
Management s
n - t r
System
a no
ASR
Manager
h a sService request (SR)
ASR Service
SNMP trap sent to
ASR Manager
Fault telemetry
securely m ) ideฺ created
transmitted to co u
Oracle gฺ t G
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li illustrates the ASR process. It shows how a service request is
e in the slide
The diagram
b
em
eautomatically opened by the ASR Manager after it receives an SNMP trap that is triggered by
ak a server hardware fault. Within Oracle Support Services, the ASR is acknowledged via email
and serviced according to normal procedures.
Customers must note that there are occasions when an SR may not be automatically filed.
For example, this can happen due to loss of connectivity to the ASR Manager. Customers can
continue to monitor their systems for faults and engage Oracle Support if they notice a fault
but do not receive notice that a service request has been automatically filed.

Agenda

• OEM diagnostic framework

a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Oracle Configuration Manager Overview
• OCM collects configuration information for propagation to

Oracle Support.
• Benefits include:
– Efficient problem diagnosis
– Easier service request reporting
– Configuration information uploaded to Oracle either
automatically or manually
a ble
– Configuration information remains confidential. fe r
anthe s
• Oracle recommends the use of OCM in conjunction with
- t r
Recovery Appliance. non
h a sa
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
OCM automatically li
collects configuration information from your environment at regular
b
k e em This configuration information can be uploaded to My Oracle Support. This helps
intervals.
a Oracle maintain up-to-date information about your environment, diagnose support issues
more efficiently, and offer consistently better support outcomes. OCM is recommended for
use in conjunction with Recovery Appliance.
Oracle recommends that OCM is set up so that configuration information can be automatically
collected and uploaded to the Oracle Customer Configuration Repository and My Oracle
Support. Alternatively, information can be manually uploaded by an administrator.
No business or personal information is collected and uploaded, except for contact information,
which is used for handling transmission problems. All information is kept strictly confidential.
Note that OCM is separate and distinct from the Enterprise Manager Configuration
Management Pack.

Configuring OCM
a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li of OCM is typically performed as a part of the initial installation and
e configuration
Like ASR,
b
em
econfiguration of the Recovery Appliance. Customers can opt out of configuring OCM.
ak However, this is not recommended by Oracle.
For customers who want to configure OCM against an existing Recovery Appliance system,
the easiest way is to use the resources on the Collector tab in My Oracle Support. The slide
shows the interface presented to users, which they can use to get started with OCM.
The OCM collector is installed on the Recovery Appliance database servers. No OCM
components are installed on the Exadata Storage Servers.

Summary

• Identify various diagnostic tools
• Perform basic troubleshooting

• Handle the ASR and Oracle Configuration Manager
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

ake
me
bel
l o
a k
li
l l o
(a cens
u
be e to
n
@ use
b a n
nio this
k
S t
ฺ c
ud
ng ent
)
s
om Guid
ha eฺ
an
o n - t r an
sfe r a b
le
12
Reporting with BI Publisher r a ble

s fe
- t r an
o n
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake
Objectives

• Describe the reporting framework included with Enterprise
Manager
• Use different Oracle-provided reports for the Recovery
Appliance
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Agenda
• Introduction to BI Publisher
• Using Oracle-Provided Reports
• Reporting on the Recovery Appliance Components

• Reporting on Protected Databases
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Introduction to BI Publisher
BI Publisher's intuitive user interface allows you to create and publish

reports with little effort.
The key benefits of using BI Publisher include:

• A framework for creating content-rich, well-formatted HTML reports
based on management repository data
• Out-of-the-box reports, which let you start generating reports
immediately without any system configuration or setup
• The ability to schedule automatic generation of reports and store
r a ble
scheduled copies and/or email them to intended audiences nsfe
• The ability for Enterprise Manager administrators to share n - tra
reports
with the entire business community, including executives,
a no
a
customers, and other Enterprise Manager administrators
h s
m ) ideฺ
BI Publisher provides you with a feature-rich
g ฺ coframework
t G u that is the
b ankn en
central information source for your enterprise.
ud
n S t
u nio this
l l o @ use
be e to
a k
l o (a cens
BI Publisher,
b li Manager's reporting framework, makes information about your
el Enterprise
em environment available to audiences across your enterprise.
emanaged
ak Strategically, reports are used to present a view of enterprise monitoring information for
business intelligence purposes, but can also serve an administrative role by showing activity,
resource utilization, and configuration of managed targets.
IT managers can use reports to show availability of sets of managed systems. Executives can
view reports on availability of applications (such as corporate email) over a period of time.

Recovery Appliance Reporting Workflow
Recovery Appliance basic tasks for reporting

Planning Ongoing
Setup and Configuration
Phase Maintenance
Capacity Protected Database able

Reports Reports sfe
r
tra n
o n -
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
l perform
eyou
Typically,
b li reporting tasks in the following sequence:
k e e1.m During the planning phase, familiarize yourself with the monitoring and reporting tools
a available through Cloud Control.
2. Setup and Configuration tasks are automated when installing Recovery Appliance
plugins in Enterprise Manager Cloud Control. Pre-configured reports for Recovery
Appliance are available and ready to be used.
3. During the ongoing maintenance phase, review the reports, as needed.
- Review the Storage Capacity Planning Summary every week, using the Capacity
Planning Details to get more detailed information.
- Review the protected database reports as needed.

Agenda
• Reporting on Recovery Appliance Components

a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Oracle-Provided Reports with the Recovery Appliance
The Recovery Appliance provides the following pre-created BI

Publisher reports:
• Capacity Planning Summary

• Capacity Planning Details
• Recovery Window Summary
• Protected Database Details
• Top 10 Protected Databases by Data Transfer a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Through l l o li the Recovery Appliance provides pre-created BI Publisher reports that
eBI Publisher,
b
k e em you to meet the following goals:
enable
a • Ensure that the Recovery Appliance has sufficient storage space for its needs: By using
the capacity reports, you can plan for additional storage, reduce the number of new
protected databases added to the Recovery Appliance, or adjust protection policies so
that the aggregate recovery window space decreases.
• Ensure that the network is not overloaded: The capacity reports also indicate whether
the Recovery Appliance has maximized network capacity. In some cases, you can
reduce network load by redistributing network traffic more evenly throughout the day. If
network traffic is not distributed, and if network peaks are close to maximizing network
bandwidth, then you may need to adjust the backup window times of some protected
databases.
• Provide a good view of system performance and activity for service requests

• Obtain a brief or highly detailed status report for any protected database, which can
sometimes be useful for troubleshooting databases that are not meeting recovery
window goals
The BI Publisher comes with a comprehensive library of predefined report definitions, allowing
you to generate fully formatted HTML reports presenting critical operations and business
information without any additional configuration or setup.
Enterprise manager allows you to view reports interactively and/or schedule generation of
reports on a flexible schedule. You can choose for scheduled reports to be emailed to any
number of recipients.
Note: BI Publisher software included with Cloud Control can only access data in the
Enterprise Manager Repository, not data in the Recovery Appliance schema. Recovery
Appliance schema data is accessible through SQL queries of V$ and recovery catalog views.
a ble
fe r
ans
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
a k be e to
l o (a cens
bel li
e m
ake

Agenda

a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Reporting on Recovery Appliance Components
Storage Capacity report

a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o i
e StoragelCapacity
Reviewbthe Planning Summary section to determine the storage growth rate
k e emdays until capacity is exceeded.

and
a For example, the graphic illustrates the summary for ZDLRA San Francisco: The key metric is
the “Days until Capacity is Exceeded.” Within a week, the Recovery Appliance will either
purge backups or reject incoming backups, depending on the protection policy settings.
The chart also illustrates that over the last five days, the total space needed to satisfy the
recovery window goals for all protected databases has decreased from 2.3 TB to 2.2 TB. The
number of protected databases has been steady at 27. You could also view the historical
storage trends over various time periods (7 days, 31 days, and last year).
The network capacity planning summary provides a view of the aggregated network traffic
over various time periods (24 hours, 7 days, 31 days, and last year). The chart shows that the
receive rate spiked during the last day.

Storage Capacity Details Report

a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e provideslan
This report i even finer granularity of information than the capacity planning
b
em Unlike the capacity planning report, the detailed report also has memory and IOPS
esummary.
ak information.

Recovery Window Report

a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o li finer granularity of information than the capacity planning summary,
e also provides
This report
b
emincludes memory and IOPS information.
eand
ak The pie chart shows that over 18% of the databases are not meeting their service level
agreements. The table shows that out of 27 total databases, one is not meeting its recovery
window goal. Also, four databases are not within their unprotected data threshold.

Agenda
• Using Oracle-provided reports

a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

Reporting on Protected Databases
Protected Database Report

a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
l l o
e Databases
The Protected li section in the Recovery Appliance table displays charts showing
b
emstatus of each protected database on the Status tab.
ethe
ak You have two links to navigate to the Protected Database Report and to the Backup Report.
For example, the chart shows the Summary details and the data sent and received over the
past week by CUST01 database after making click the Protected Database Report link for the
CUST01 database.

Top 10 Protected Database Report

a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
e l l o
Top 10bProtected li
Database reports use multiple dimensions: Top 10 Protected Databases by
emTransfer, Top 10 Databases by Backup Data, Top 10 Databases by Replication Data,
eData
ak and Top 10 Databases by Copy-to-Tape Data.

Backup Database Report

a ble
fe r
an s
n - t r
a no
h a s
m ) ideฺ
g ฺ co t Gu
a n kn den
i o nb s Stu
u n t h i
l lo@ o us e
e
kb se t
a a
( cen
Use the
l o
eView Backupli Report page to:
l
b
ak ee•m Display lists of backup jobs that are known to the database through the information
recorded about them in the database control file
• Filter the backup jobs that display by Status, Start Time, and Type
• Link to Job Details and Job Summaries
You can drill down to a summary page of the backup job by clicking on the Status of the job in
the Result table where you can view the contents of the output log.

Summary
In this lesson, you should have learned how to use

Oracle-provided reports for the Recovery Appliance.
a ble
fe r
an s
n - t r
o
s an
) ha eฺ
ฺ c om Guid
n k ng ent
n b a
S t ud
u nio this
l l o @ use
be e to
a k
l o (a cens
bel li
em
ake

ake
me
bel
l o
a k
li
l l o
(a cens
u
be e to
n
@ use
b a n
nio this
k
S t
ฺ c
ud
ng ent
)
s
om Guid
ha eฺ
an
o n - t r an
sfe r a b
le

D96091GC10 SG

Enviado por

Dados do documento

Título original

Direitos autorais

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

D96091GC10 SG

Enviado por

Direitos autorais:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2019, Oracle and/or its affiliatesฺ

Learn more from Oracle University at oracle.com/education/

Hector Navarro Disclaimer

Restricted Rights Notice

3 Recovery Appliance Functions and Features

Oracle Deduplication and Validation Challenges 3-11

4 Recovery Appliance Installation

Expanding the Recovery Appliance 4-55

5 Zero Data Loss Recovery Appliance Configuration

6 ZDLRA Protected Databases Configuration

Monitoring Protected Databases 6-35

7 Optimizations and Performance Tuning

8 Disaster Recovery with Replication

9 Recovery Appliance Integrated Media Management

Generic Troubleshooting Questions 11-6

After completing this course, you should be able to:

and components, including Delta Push and Delta Store

Zero Data Loss Recovery Appliance Overview 1 - 2

This course comprises the following lessons:

2. The Zero Data Loss Recovery Appliance Solution

Zero Data Loss Recovery Appliance Overview 1 - 3

• Oracle Recovery Appliance Product Page

Zero Data Loss Recovery Appliance Overview 1 - 4

To assess your preparedness to take this course, answer the

are prepared to take this course.

Zero Data Loss Recovery Appliance Overview 1 - 5

Now that you have been introduced to the course, introduce

Zero Data Loss Recovery Appliance Overview 1 - 6

Zero Data Loss Recovery Appliance Solution able

After completing this lesson, you should be able to:

• Explain the benefits of the Oracle Recovery Appliance

Zero Data Loss Recovery Appliance Overview 2 - 2

• Engineered Systems: Overview

• Oracle Recovery Appliance Hardware

Zero Data Loss Recovery Appliance Overview 2 - 3

Engineered Systems are purpose-built hardware and software

Zero Data Loss Recovery Appliance Overview 2 - 4

Zero Data Loss Recovery Appliance Overview 2 - 5

Oracle Recovery Appliance Design Goals Tape Library

Protected Databases Recovery Appliance

• No expensive DB backup agents

Zero Data Loss Recovery Appliance Overview 2 - 6

e e m remote data center.

Zero Data Loss Recovery Appliance Overview 2 - 7

Zero Data Loss Recovery Appliance Overview 2 - 8

Zero Data Loss Recovery Appliance Overview 2 - 9

• Engineered Systems: Overview

• Oracle Recovery Appliance Hardware

Zero Data Loss Recovery Appliance Overview 2 - 10

Private Data Protection Cloud supports up to thousands of databases

• Policy-Based Data Protection: Groups databases by the recovery service

k e em scalable service. The following technology components within the Recovery

Zero Data Loss Recovery Appliance Overview 2 - 11

• Engineered Systems: Overview

• Oracle Recovery Appliance Hardware

Zero Data Loss Recovery Appliance Overview 2 - 12

ZDLRA X4-2 ZDLRA X5-2 ZDLRA X6-2 U40

Cores/Node 2x12 (48) 2x18 (72) 2x22 (88) U34

knto dMulti n Rack (up to U06 U06

Storage Multi Rack (up to

u e t h U01 Solid filler U01

Zero Data Loss Recovery Appliance Overview 2 - 13

Compute Servers (88) Cores (total) (88) Cores (total)

Memory 512 GB Memory (total) 512 GB Memory (total)

Raw Capacity 288 TB 1.73 PB

Zero Data Loss Recovery Appliance Overview 2 - 14