o n l y THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED Oracle Database 10g: Data Guard Administration 11 - 2 Copyright 2006, Oracle. All rights reserved. Objectives After completing this lesson, you should be able to deploy Data Guard in a Real Application Clusters environment. O r a c l e
U n i v e r s i t y
a n d
S Q L
S T A R
I N T E R N A T I O N A L
L I M I T E D
u s e
o n l y THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED Oracle Database 10g: Data Guard Administration 11 - 3 Copyright 2006, Oracle. All rights reserved. Real Application Clusters and Data Guard Possible combinations of instances in the primary and standby databases: Yes Yes Multi-Instance Yes Yes Single-Instance Multi-Instance Standby Database Single-Instance Standby Database Primary Instance Real Application Clusters and Data Guard You can configure your Real Application Clusters (RAC) environment to switch over to another RAC environment or a single-instance environment. If you configure switchover to another RAC environment, the broker ships all redo from all primary database RAC instances to the single standby instance where apply services is running. After switchover or failover, your application is in a RAC, which should give the same performance as the previous configuration. Switchover and failover to a smaller system do not allow your application to perform as in the original configuration. If you are sending redo to a single-instance standby database, be sure to configure the number of standby redo logs on the standby database to be equal to the total number of online redo logs in all the instances of the RAC system. O r a c l e
U n i v e r s i t y
a n d
S Q L
S T A R
I N T E R N A T I O N A L
L I M I T E D
u s e
o n l y THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED Oracle Database 10g: Data Guard Administration 11 - 4 Copyright 2006, Oracle. All rights reserved. Configuration Considerations with RAC Format for archived redo log file names: Thread variable (%t or %T) of LOG_ARCHIVE_FORMAT is mandatory. Data protection modes: Maximum protection or maximum availability mode; all instances could stop shipping redo. Configuration Considerations with RAC Format for archived redo log file names: The thread variable (%t or %T) in the LOG_ARCHIVE_FORMAT parameter is mandatory for Real Application Clusters to uniquely identify the archived redo log files. Data protection modes: In a RAC configuration that is configured for maximum protection or maximum availability mode, any instance that loses connectivity with a standby destination causes all other instances to create a sync point for that destination. This action maintains the data integrity of the data that has been transmitted to that destination and can be recovered. When the failed standby destination comes back up, Data Guard runs the site in resynchronization mode until no gaps remain. Then, the standby destination can participate in the Data Guard configuration again. O r a c l e
U n i v e r s i t y
a n d
S Q L
S T A R
I N T E R N A T I O N A L
L I M I T E D
u s e
o n l y THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED Oracle Database 10g: Data Guard Administration 11 - 5 Configuration Considerations with RAC (continued) Data protection modes: In a RAC configuration that is configured for maximum protection or maximum availability mode, any instance that loses connectivity with a standby destination causes all other instances to create a sync point for that destination. This action maintains the data integrity of the data that has been transmitted to that destination and can be recovered. When the failed standby destination comes back up, Data Guard runs the site in resynchronization mode until no gaps remain. Then, the standby destination can participate in the Data Guard configuration again. O r a c l e
U n i v e r s i t y
a n d
S Q L
S T A R
I N T E R N A T I O N A L
L I M I T E D
u s e
o n l y THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED Oracle Database 10g: Data Guard Administration 11 - 6 Copyright 2006, Oracle. All rights reserved. Multi-Instance Primary Database with a Single-Instance Standby Database Primary database Standby database Multi-Instance Primary with a Single-Instance Standby You can create a single-instance standby database by using either SQL commands or Enterprise Manager. Each instance of the primary database sends redo to the one instance of the standby. The standby database automatically determines the correct order in which to apply the archived redo log files. O r a c l e
U n i v e r s i t y
a n d
S Q L
S T A R
I N T E R N A T I O N A L
L I M I T E D
u s e
o n l y THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED Oracle Database 10g: Data Guard Administration 11 - 7 Copyright 2006, Oracle. All rights reserved. Multi-Instance Primary Database with a Multi-Instance Standby Database Primary database Standby database Multi-Instance Primary with a Multi-Instance Standby Although the Add Standby Database Wizard does not create a RAC standby database, you can use it to add an existing RAC standby database to a Data Guard configuration. Click Add Standby Database to invoke the Add Standby Database Wizard, and then select Add an existing standby database. The Data Guard broker also has the ability to dynamically discover instances of a database and add them to a database profile without user intervention. After they are added, the broker can manage these instances when it comes to conducting state changes, role changes, and so on. If a new primary instance is added, the broker automatically enables the redo transport service on that instance and gets that instance to ship its redo to the configured set of standby databases. During role changes and protection mode upgrades, the broker also manages the instance restarts because it is integrated with the RAC Cluster Ready Services (CRS). O r a c l e
U n i v e r s i t y
a n d
S Q L
S T A R
I N T E R N A T I O N A L
L I M I T E D
u s e
o n l y THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED Oracle Database 10g: Data Guard Administration 11 - 8 Copyright 2006, Oracle. All rights reserved. Redo Transport with RAC to RAC Primary instance A Primary instance B Standby instance C Standby recovery instance D Archived logs Archived logs Online redo files Standby redo files Archived logs LGWR RFS ARCn LGWR Primary database Physical standby database MRP Redo Transport with RAC to RAC The best practice is to send all redo to the recovery instance on the standby RAC. When both the primary and standby databases are in a Real Application Clusters configuration, then a single instance of the standby database applies all sets of log files that are transmitted by the primary instances. In this case, the standby instances that are not applying redo data cannot be in read-only mode while Redo Apply is in progress. O r a c l e
U n i v e r s i t y
a n d
S Q L
S T A R
I N T E R N A T I O N A L
L I M I T E D
u s e
o n l y THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED Oracle Database 10g: Data Guard Administration 11 - 9 Copyright 2006, Oracle. All rights reserved. Setting Up a Primary Database with RAC Primary instance A Primary instance B Archived logs Archived logs Online redo files LGWR LGWR Primary database Setting Up a Primary Database with RAC To configure redo transport services on the primary database: 1. On all instances, define the LGWR attribute for the LOG_ARCHIVE_DEST_n parameter to specify that the LGWR process performs the archival operation. 2. Configure each primary instance to send its redo to the standby recovery instance by setting the LOG_ARCHIVE_DEST_n parameter to an appropriate value. O r a c l e
U n i v e r s i t y
a n d
S Q L
S T A R
I N T E R N A T I O N A L
L I M I T E D
u s e
o n l y THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED Oracle Database 10g: Data Guard Administration 11 - 10 Copyright 2006, Oracle. All rights reserved. Setting Up a Standby Database with RAC Standby instance C Standby recovery instance D Standby redo files Archived logs RFS ARCn Physical standby database MRP Setting Up a Standby Database with RAC To configure redo transport services on the standby database: 1. Create the standby redo log files. In a Real Application Clusters environment, the standby redo log files must reside on disk devices that are shared by all instances. 2. On the recovery instance, define the LOCATION attribute of the LOG_ARCHIVE_DEST_1 initialization parameter to archive locally. 3. Start log apply services on the recovery instance. O r a c l e
U n i v e r s i t y
a n d
S Q L
S T A R
I N T E R N A T I O N A L
L I M I T E D
u s e
o n l y THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED Oracle Database 10g: Data Guard Administration 11 - 11 Copyright 2006, Oracle. All rights reserved. Assigning Threads to Standby Redo Log Groups SQL> ALTER DATABASE ADD STANDBY LOGFILE 2 THREAD 1 'STBY_LOGFILE_1.SRL' SIZE 50M; Primary database Standby database Standby redo logs Assigning Threads to Standby Redo Log Groups Use the THREAD n clause in the ALTER DATABASE ADD STANDBY LOGFILE statement to assign a standby redo log group to a specific thread. With this clause, you can balance the use of standby redo log groups across all threads. The THREAD n syntax is optional. If the syntax is omitted, Data Guard automatically assigns the standby redo log to a thread at run time. This is applicable only if you are using the RAC. Because standby redo log groups are now assigned to a given thread, you may need more standby redo log groups. This is because there is no longer a pool of files available for any thread. If you have threads that generate more redo than others, assign more standby redo log groups to that thread. It is usually sufficient to assign one or two more than there are online log files. O r a c l e
U n i v e r s i t y
a n d
S Q L
S T A R
I N T E R N A T I O N A L
L I M I T E D
u s e
o n l y THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED Oracle Database 10g: Data Guard Administration 11 - 12 Copyright 2006, Oracle. All rights reserved. Apply Instance Failover If the apply instance fails, transport of redo is also halted by default. Broker configuration: Set ApplyInstanceTimeout to avoid downtime. Default is 120 seconds. Set to 0 (zero) to disable. Optionally set PreferredApplyInstance. Non-broker configuration: Set up a list of destination connect identifiers. Apply Instance Failover When the apply instance fails, not only does log apply services stop applying log files to the standby database, but redo transport services stop transmitting redo data to the standby database. To tolerate a failure of the apply instance, the broker leverages the availability of the RAC standby database by automatically failing over log apply services to a different standby instance. To set up apply instance failover in a Data Guard brokercontrolled configuration, set the ApplyInstanceTimeout property to specify the time period that the broker should wait after detecting an apply instance failure and before initiating an apply instance failover. To select an appropriate timeout value, you must consider: If there is another mechanism in the cluster that will try to recover the failed apply instance How long the primary database can tolerate not transmitting redo data to the standby database The overhead that is associated with moving the log apply services to a different instance. The overhead may include retransmitting (from the primary database) all unapplied log files that have accumulated on the failed apply instance if those log files are not saved in a shared file system that can be accessed from other standby instances. O r a c l e
U n i v e r s i t y
a n d
S Q L
S T A R
I N T E R N A T I O N A L
L I M I T E D
u s e
o n l y THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED Oracle Database 10g: Data Guard Administration 11 - 13 Apply Instance Failover (continued) You can also set the PreferredApplyInstance property of Data Guard broker to indicate which instance should take over this task if the current apply instance fails. If PreferredApplyInstance is not set, the broker picks a random instance that is currently running to be the new apply instance. If the Data Guard broker is not enabled for your configuration, Oracles high-availability best practices recommend setting up a list of destination connect identifiers in the tnsnames.ora file on the primary database, as in the following example: CHICAGO= (DESCRIPTION= (ADDRESS_LIST= (ADDRESS=(PROTOCOL=tcp)(HOST=chicago_n1-server)(PORT=1521)) (ADDRESS=(PROTOCOL=tcp)(HOST=chicago_n2-server)(PORT=1521))) (CONNECT_DATA= (SERVICE_NAME=CHICAGO))) In this case, LGWR chooses the next entry in the list to send redo data to, after a timeout period that is specified in the NET_TIMEOUT attribute of the log_archive_dest_n parameter (if NET_TIMEOUT is not specified, it waits until the systems TCP/IP timeout). However, the apply process (Redo Apply or SQL Apply) would need to be manually started in the new instance with SQL*Plus. The primary instance of the standby cluster will be brought down to ensure no data loss if all of the following are true: A connection list is not specified The Data Guard broker is not enabled for the configuration The apply instance for the last standby database fails The LGWR process times out O r a c l e
U n i v e r s i t y
a n d
S Q L
S T A R
I N T E R N A T I O N A L
L I M I T E D
u s e
o n l y THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED Oracle Database 10g: Data Guard Administration 11 - 14 Copyright 2006, Oracle. All rights reserved. Role Transitions with RAC Switchovers: Only one primary instance and one standby instance can be active during a switchover. Failovers: Before performing a failover to a RAC standby database, shut down all but one standby instance. Role Transitions with RAC Switchovers: For a RAC database, only one primary instance and one standby instance can be active during a switchover. Before a switchover, shut down all but one primary instance and one standby instance. After the switchover completes, restart the primary and standby instances that you shut down during the switchover. Failovers: Before performing a failover to a RAC standby database, shut down all but one standby instance. After the failover completes, restart the instances that were shut down. O r a c l e
U n i v e r s i t y
a n d
S Q L
S T A R
I N T E R N A T I O N A L
L I M I T E D
u s e
o n l y THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED Oracle Database 10g: Data Guard Administration 11 - 15 Copyright 2006, Oracle. All rights reserved. Troubleshooting Switchover failure Avoiding downtime during a network outage Troubleshooting Switchover failure: When your database is using RAC, active instances prevent a switchover from being performed. When other instances are active, an attempt to switch over fails and the following error message appears: SQL> ALTER DATABASE COMMIT TO SWITCHOVER TO STANDBY; ALTER DATABASE COMMIT TO SWITCHOVER TO STANDBY * ORA-01105: mount is incompatible with mounts by other instances Query the V$ACTIVE_INSTANCES view to determine which instances are causing the problem: SQL> SELECT * FROM v$active_instances; INST_NUMBER INST_NAME ----------- ------------------------------ 1 nleduc1.nl.oracle.com:orcl1 Connect to this instance and shut it down. O r a c l e
U n i v e r s i t y
a n d
S Q L
S T A R
I N T E R N A T I O N A L
L I M I T E D
u s e
o n l y THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED Oracle Database 10g: Data Guard Administration 11 - 16 Troubleshooting (continued) Avoiding downtime: If you configured Data Guard to support a primary database in a RAC environment and the primary database is running in maximum protection mode, a network outage between the primary database and all of its standby databases will disable the primary database until the network connection is restored. The maximum protection mode dictates that if the last participating standby database becomes unavailable, processing halts on the primary database. If you expect the network to be down for an extended period of time, consider changing the primary database to operate in either maximum availability or maximum performance mode until network connectivity is restored. If you change the primary database to maximum availability mode, it is possible for there to be a lag between the primary and standby databases, but you gain the ability to use the primary database until the network problem is resolved. If you choose to change the primary database to maximum availability mode, it is important to use the following procedures to prevent damage to your data. The following steps describe what to do if the network goes down and you want to change the protection mode for the RAC configuration. The example assumes that you are using a server parameter file (SPFILE) rather than a text initialization parameter file (PFILE). 1. At this point, all RAC primary instances are shut down. Issue the STARTUP MOUNT command to start one instance: STARTUP MOUNT; 2. Change the mode from maximum protection to either maximum availability or maximum performance. For example, the following statement sets the maximum availability protection mode: ALTER DATABASE SET STANDBY DATABASE TO MAXIMIZE AVAILABILITY; 3. Open the RAC primary database for general access. When the network comes back up later, perform the following steps to revert to maximum protection mode: 1. Shut down all instances of the RAC primary database. 2. Mount a single instance of the RAC primary database without opening it for general access. 3. Change the mode on the RAC primary database from its current mode (maximum availability or maximum performance) to maximum protection mode. 4. Open the RAC primary database for general access. O r a c l e
U n i v e r s i t y
a n d
S Q L
S T A R
I N T E R N A T I O N A L
L I M I T E D
u s e
o n l y THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED Oracle Database 10g: Data Guard Administration 11 - 17 Copyright 2006, Oracle. All rights reserved. Summary In this lesson, you should have learned how to deploy Data Guard in a Real Application Clusters environment. O r a c l e
U n i v e r s i t y
a n d
S Q L
S T A R
I N T E R N A T I O N A L
L I M I T E D
u s e
o n l y THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED O r a c l e
U n i v e r s i t y
a n d
S Q L
S T A R
I N T E R N A T I O N A L
L I M I T E D
u s e
o n l y THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED