Logical Standby Database for Reporting

Mark Bole NoCOUG Nov 10, 2005

Introduction
Mark Bole Independent Consultant Oracle, Unix, Perl since 1991 http://www.bincomputing.com

(teaser)
Logical Standby Databases for Reporting - Mark Bole, BIN Computing Not quite ready for a data warehouse, but still want a high performance reporting environment that doesn't impact your online transaction (OLTP) users? New in Oracle 9i, the Data Guard logical standby database is one answer. One or more core schemas are automatically synchronized with your live database, as frequently as you wish, without creating any load on the source database. Once updated, the core data is protected from any alteration, yet any other tables and schemas can be created in the same database to extend your data model and provide reporting performance. Materialized views, query rewrite, and other typical data warehouse techniques are all available.

Todays Session
DataGuard Logical Standby going outside the DG box What is Logical Standby? How to set it up Reporting/Batch: refresh cycle, materialized views, maintenance

DataGuard vs. Streams

From the Oracle docco:

While Streams and Data Guard do share some common underlying [] technology, they are both independent features that are built to solve different business needs. [see references]  Oracle Data Guard is designed for protecting from data failure and disasters.  Streams is designed for information sharing and distribution but can also provide a very efficient high availability solution.

Logical Standby Product Placement?

Integration High Availability
Basic readable standby database Oracle Data Guard Redo Apply (Physical Standby) Basic Replication Advanced Replication

8i 9i 10g

Oracle Streams

Oracle Data Guard SQL Apply (Logical Standby)

Logical Standby Product Placement?

Integration High Availability
Basic readable standby database Oracle Data Guard Redo Apply (Physical Standby) Basic Replication Advanced Replication

8i 9i 10g

Oracle Streams

Oracle Data Guard SQL Apply (Logical Standby)

Why Not Logstdby for HA?

Parity, one-to-one-ness, having a failover environment that is a mirror image of the production [site]. That is the ideal situation.
http://asktom.oracle.com

Too Many Ways for the Logical Standby to deviate from the primary intentional or otherwise!

Bugs A Real-Life Story

The following is an excerpt from a recent posting on the oracle-l list. Need some help or opinions regarding Oracle logical standby database. Logical standby, version 10, release x, patch x. Primary and standby are tightly coupled (maximum availability mode). [...] 2) From time to time, standby breaks, but this seems due mostly to schema changes [...] 3) DB Guard leaks memory on the primary. This was confirmed via the OS, not V$-views. . Had to disable it. Not really happy about that. [] 5) Primary is not happy about unexpected standby restarts.[...] 6) To sum up: something's a bit shaky in my environmentt. Should I give up with maximum availability mode and reduce the database to maximum performance mode? I am interested to hear from others.

Unsupported Datatypes (partial list) LONG LONG RAW BFILE ROWID user-defined types varrays nested tables

Logical Standby has a documented list of restrictions*

Unsupported Tables, Sequences, and Views (partial list)

Tables using data segment compression Index-organized tables

*Note: many removed in 10g

Evolution of Robustness 8i 9i 10g

RMAN Robustness: less more

RAC

SQL Apply

ASM

Assumptions: Logstdby for Reporting

Not running DG Manager/Broker No intention of ever switching DG roles Log transport MAXIMUM PERFORMANCE mode Implies existence of separate Physical Standby for real recovery NOARCHIVELOG mode (all report/batch data can be re-created if necessary) Service Level less than 24 x 7 (planned maintenance outages) Not running SQL Apply continuously

Todays Session
DataGuard Logical Standby going outside the DG box What is Logical Standby? How to set it up Reporting/Batch: refresh cycle, materialized views, maintenance

Physical standby is an image copy of the primary every datafile block is the updated to match as of point in time. Other existing options for logical:  Export / Import  Traditional Advanced Replication  Transportable tablespace (maybe)

The Goal: logical, not physical, replication

Previous Solution
Primary Redo apply (once every 24 hours) Daily Cycle:
1. 2. 3. 4. 5. 6. Close Physical Standby Mount Physical Standby Recover day's worth of redo Open Physical Standby read-only for rest of day Run batch and reporting processes using dblink Loop to step 1

Physical Standby

DB Link

Reporting Database

Previous Solution - Problems

Need to close and reopen physical standby Two databases on same server creates inefficiencies Harder to tune distributed queries Cannot use MV logs in standby, hence cannot use MV's in reporting database
Physical Standby (open read-only)

DB Link

Reporting Database

SQL Apply
SQL Apply technology [] first transforms the redo data into SQL statements and then executes the generated SQL statements on the logical standby database.
Physical

SQL

redo

Block-level logminer changes

Logical

SQL

GUI Log Miner

I:\>Oracle\product\10.1.0\Client_1\BIN\oemapp.bat lmviewer

Select archived redo logs to mine

Example: schema in primary

create user schema_blue identified by xyz default tablespace TESTDB_A temporary tablespace temp quota 100M on TESTDB_A quota unlimited on temp; grant create session, create table to schema_blue; connect schema_blue/xyz@binc01.tree CREATE TABLE test_table (name varchar2(50), value varchar2(50), timestamp date, constraint pk_test_table primary key (name) using index); insert into test_table values ('test1', 'value1', sysdate); insert into test_table values ('test2', 'value2', sysdate); commit;

Logical Change Records

Logical Standby = Streams Made Simple

Streams is configured from the bottom up individual tables, schemas, capture processes, apply processes, queues Logical Standby is configured from the top down start with entire database, then specify only what you dont want Less muss, less fuss

Todays Session
DataGuard Logical Standby going outside the DG box What is Logical Standby? How to set it up Reporting/Batch: refresh cycle, materialized views, maintenance

Logstdby Pre-requisites
Enterprise Edition license Same operating system and platform architecture as primary Same RDBMS version (10g supports rolling upgrades) FORCE LOGGING at database level

Supplemental Logging
If youre going to use supplemental logging, you need to be aware that you are actually asking for quite a lot of new information to be included in the redo stream and all that extra information might just be enough to tip LGWR and ARCH over into serious performance problems.
Howard J. Rogers, New Features in Oracle 9i

Much less of a problem if you have a unique or primary key on every table!

Create the Logstdby

Start with any kind of hot backup Mount or quiesce the primary (requires outage, restriction removed in 10g)
EXECUTE DBMS_LOGSTDBY.BUILD to put the LogMiner dictionary into the redo stream Rename datafiles, online redo logs in standby Reset database name DBNEWID utility Register archive logfile containing LogMiner dictionary, start SQL Apply

My favorite LogStdby settings

INIT.ORA
remote_archive_enable = receive standby_archive_dest = '/opt/oracle/admin/binc01dg/stbyarch'

DBA_LOGSTDBY_PARAMETERS
DBMS_LOGSTDBY.APPLY_SET ('TRANSACTION_CONSISTENCY', 'NONE'); DBMS_LOGSTDBY.APPLY_SET ('MAX_EVENTS_RECORDED', '2000'); DBMS_LOGSTDBY.APPLY_SET ('RECORD_SKIP_DDL', 'FALSE'); DBMS_LOGSTDBY.APPLY_SET ('RECORD_APPLIED_DDL', 'FALSE'); DBMS_LOGSTDBY.APPLY_SET ('_EAGER_SIZE', 1000);

alter database guard standby;

Todays Session
DataGuard Logical Standby going outside the DG box What is Logical Standby? How to set it up Reporting/Batch: refresh cycle, materialized views, maintenance

Previous Solution
Primary Redo apply (once every 24 hours) Daily Cycle:
1. 2. 3. 4. 5. 6. Close Physical Standby Mount Physical Standby Recover day's worth of redo Open Physical Standby read-only for rest of day Run batch and reporting processes using dblink Loop to step 1

Physical Standby

DB Link

Reporting Database

The Logstdby Advantage

SCHEMA_RED TABLES INDEXES PL/SQL SCHEMA_BLUE TABLES INDEXES PL/SQL SCHEMA_GREEN TABLES INDEXES PL/SQL SCHEMA_BLUE TABLES INDEXES PL/SQL SCHEMA_ORANGE TABLES INDEXES PL/SQL

Primary

LogStdby

DG SQL Apply
OLTP application Reporting/Batch application

The Logstdby Advantage

Guard Standby (read-only)
SCHEMA_BLUE TABLES MV LOGS INDEXES PL/SQL SCHEMA_ORANGE TABLES INDEXES MATL_VIEWS PL/SQL

LogStdby

Now we can create Materialized View logs directly in the source schema without impacting the primary!

Reporting/Batch application

Controlling what is replicated

-- workaround for bug in 9.2.0.5 exec DBMS_LOGSTDBY.SKIP('PROCEDURE', 'XYZ', '%', null); exec DBMS_LOGSTDBY.SKIP('SCHEMA_DDL', 'VCS_MONITOR', '%', null); exec DBMS_LOGSTDBY.SKIP('DML', 'VCS_MONITOR', '%', null); -- NON_SCHEMA_DDL (for grants)

Example: MV logs
schema_blue> select table_name from user_tables; TEST_TABLE schema_blue> delete from test_table where name = 'test1'; ORA-01031: insufficient privileges <= table is guarded schema_blue> CREATE MATERIALIZED VIEW LOG on test_table 2 with sequence, primary key, rowid 3 (value, timestamp) 4 INCLUDING NEW VALUES; Materialized view log created.

Example: Materialized View

schema_orange> create materialized view mv_test_table 2 build immediate 3 refresh fast on demand 4 ENABLE QUERY REWRITE 5 AS 6 SELECT 7 count(*) cnt, 8 count(name) cnt_name, 9 tt.value, 10 trunc(tt.timestamp) trunc_timestamp 11 FROM schema_blue.test_table tt 12 13 group by 14 tt.value, 15 trunc(tt.timestamp); Materialized view created.

Fast Refreshable? Check!

schema_orange> SELECT OWNER, 2 MVIEW_NAME, 3 FAST_REFRESHABLE 4 FROM DBA_MVIEWS 5 / OWNER MVIEW_NAME FAST_REFRESHABLE -------------------- --------------- -----------------SCHEMA_ORANGE MV_TEST_TABLE DIRLOAD_DML

Query Rewrite? Check!

schema_orange> exec DBMS_MVIEW.EXPLAIN_REWRITE ('select count(*) from schema_blue.test_table', 'mv_test_table'); PL/SQL procedure successfully completed. schema_orange> commit; schema_orange> SELECT MV_OWNER, MV_NAME, QUERY, MESSAGE, PASS FROM REWRITE_TABLE ; [headings omitted] SCHEMA_ORANGE MV_TEST_TABLE select count(*) from schema_blue.test_table QSM-01033: query rewritten with materialized view, MV_TEST_TABLE YES

The Logstdby Advantage

Guard Standby (read-only)
SCHEMA_BLUE TABLES MV LOGS INDEXES PL/SQL SCHEMA_ORANGE TABLES INDEXES MATL_VIEWS PL/SQL

LogStdby

Now we can create Materialized View logs directly in the source schema without impacting the primary!

Reporting/Batch application

SQL Apply can be slow

Redo contains uncommitted transactions Necessary overhead of logical vs. physical in general Be sure to read and follow Best Practices document Be prepared, in worst case, to recreate your Logical Standby for reporting!

Recover Perl script

recover_logstdby.pl - recover logical standby Connect to database alter database start logical standby apply Die on error unless ORA-16105: Logical Standby is already running in background While (BehindTime = select trunc( (newest_time - applied_time)*1440 ) from dba_logstdby_progress) if (BehindTime < specified recovery window) done, successful if (exceeded timeout) done, unsuccessful, wake me up End alter database stop logical standby apply Disconnect

References
Creating a Logical Standby with Minimal Production Downtime Oracle Data Guard Readme for SQL Apply Release 9.2.0.6 Oracle Data Guard Readme for SQL Apply Release 10.1.0.3 SQL Apply Best Practices (9i) SQL Apply Best Practices (10g) Oracle10g Data Guard SQL Apply Troubleshooting ORA-1 Occurring on Logical Standby Synchronizing tables in a Logical Standby Database Troubleshooting 9i Data Guard Network Issues Streams and Data Guard Role Transitions
Note:278371.1 Note:286787.1 Note:304059.1 http://otn.oracle.com/deploy/availability/pdf/DataGuard SQLApplyBestPractices.pdf http://www.oracle.com/technology/deploy/availability/p df/MAA_WP_10gSQLApplyBestPractices.pdf Note:312434.1 Note:257356.1 Note:271455.1 Note:241925.1 http://www.oracle.com/technology/deploy/availability/p df/MAA_WP_10gDataGuardRoleTransitionsStreams.p df

Thank you!
Copy of presentation will be available at NoCOUG web site

Questions?