Advanced DB2

By:
Parag K Bhayani
 Learning Objectives
Day# 1
Understanding physical architecture
Understanding logical architecture
DB2 and OS/390 environment
Start and Stop database
Estimating the storage space
 Learning Obj contd…
Day# 2
Performance goals
Analyzing performance data
Tuning memory
Tuning application
EXPLAIN
 Learning Obj contd…
Day# 3
DB2 Utilities
 Overview to DBMS

DB2 DBMS Layers

Table,
User Views Logical
View

Catalog Mapping Conceptual

tables

VSAM Physical
Data Sets Storage
 Physical Structure
Storage Groups
Boot Strap Data Sets (BSDS)
Active and Archive Logs
 Storage Groups Storage

Set of volumes on DASD

Volumes hold VSAM LDS
SYSDEFLT – default storage group during installation
Within a group all volumes have same device type

CREATE STOGROUP DSN8G610

VOLUMES (ABC005,DEF008)
VCAT DSNCAT;
 Storage Group contd…
Privilege required for CREATE STOGROUP
CREATESG privilege
SYSADM and SYSCTRL authority
Guidelines:
Frequently needed objects on faster device and vice versa
View Information @
SYSIBM.SYSSTOGROUP
SYSIBM.SYSVOLUMES
 Storage Groups contd…
Trouble shooting
Dynamic Allocation Error:
The volumes are of different types
Any volume ID in STOGROUP is offline
Any volume ID in STOGROUP is invalid
 Active Log
Records all data changes
In case of failure, DB2 uses it for recovery
Choose between single v/s dual logging
Single active log – 2 to 31 active log data sets
Dual – 4 to 62 active log data sets
When log buffers are full, they are written to Active
log data sets.
 Active Log contd…

Application Process
Unit of Recovery
SQL Transaction 1 SQL Transaction 2

Time Line

Application SQLT1 SQLT1 SQLT2 SQLT2 Commit Application

Process Begins Ends Begins Ends Process
Begins Ends
 Archive Log
When active log is full, DB2 copies the contents of
the active log to a DASD or magnetic tape called
archive log
It can consist of 1000 data sets, each of which is
sequential data sets
Archive log is created during off-load process
 Boot Strap Data Sets
VSAM KSDS data set
Contains critical information about DB2
Inventory of all active and archive log data sets. For any log,
list tells range of log records, which active logs are full etc
DB2 checkpoint activity
Information about buffer pools
During Installation DB2 creates 2 copies of BSDS.
Why?
Logical Structure

Database

Tablespace

Pages
Objects

Database

Tablespace

Table

View Index
 Database
Collection of DB2 logical structures
Default database DSNDB04
Catalog resides in DSNDB06
DB2 Directory in DSNDB01
Work file in DSNDB07

CREATE DATABASE DBENR

STOGROUP DSN8G610
BUFFER POOL BP8K4
INDEXBP BP4;
 Tablespace
Simple
Segmented
CREATE TABLESPACE DSN8S61C
Partitioned
IN DBENR
LOB USING STOGROUP DSN8G610
PRIQTY 30720
SECQTY 10240
SEGSIZE 32
LOCKSIZE TABLE
BUFFER POOL BP01
 Simple Tablespace
A tablespace that can contain more than one table.
The space is composed of pages
Each page can contain rows rows from many tables
Can this affect performance?
 Segmented Tablespace
A tablespace that contain more than one table.
The space is composed of group of pages called
segments
Each segment is dedicated to holding rows of a single
table.
Can this affect performance?

Segment
Segment 5
Segment
Segment 4 Table B
Segment 3
2 Table A
1 Table C
Space Table B
Header Table A
Map
Page
 Partitioned Tablespace
A table space that can contain only single table
The space is divided into partitions based on the key
ranges
Each partition can be processed concurrently
Where to use?

Partition 1 Page Page Page Page Page

Key range A-L 1 2 3 4 5

Partition 1 Page Page Page Page Page

Key range M-Z 1 2 3 4 5
 Tablespace Compression
Bit strings occuring frequently are replaced with
shorter strings
Information about the mapping of bit strings to their
replacements is stored in a compression dictionary
Processing overhead – compress before storing
Decompress before retrieving
Restriction – Tablespace containing LOB data
 Deciding on compression
Consider following points before compression:
Table space size
• Can work very well for large table spaces. With small table spaces the
compression dictionary can offset the space savings that compression
provides
Data row size
• Closer that average row length is to the actual page size, the more
inefficient compression might be, why?
 Compression
View Information @
Compression Reports – REORG with KEEPDICTIONARY
Catalog Statistics
• PAGESAVE column of SYSIBM.SYSTABLEPART
• PCTROWCOMP columns of SYSIBM.SYSTABLES and
SYSIBM.SYSTABSTATS
DS1COMP – finds out how much space will be saved and
how much processing it will require to compress your data
Physical & Logical
Non Partitioned Tablespace
Storage
Table T1 Table T2
Group# 1
Index I1 Index I2

Partitioned Index Space Storage

Tablespace
Table Part 1 Index Part 1
Table Part 1 Index Part 2
Table Part 3 Index Part 3
Table Part 4 Index Part 4

v
Storage Storage
Group# 2
 Phy & Logical contd…
Storage Group “G1” Storage Group “G2”

Vol 1 Vol 2

Tablespace: Space1 Tablespace: Space3

Table D
TABLE A
TABLE B Index space: INDXSPC1
Index ID
Tablespace: Space2
Tablespace: Space4
Table C Table E

Database “DBD1” Database “DBD2”

 Page

Header
Design and
Performance Free Space
Implications

Used Space
 Choosing page size
DB2 provides many options for data page size
The size of the data page is determined by the buffer
pool in which you define the tablespace
For example, Tablespace defined in 4 KB buffer pool
has 4K page size
Scenarios:
When the size of individual rows is greater than 4K
2100 byte record – what size of page?
Larger page size can reduce data sharing overhead –
reduces the no. of times coupling is accessed
 DB2 Catalog
Data dictionary tables residing at DSNDB06 database
SYSIBM.SYSTABLES
SYSIBM.SYSCOLUMNS
 Sample queries
SELECT NAME, TBNAME, COLTYPE, LENGTH, NULLS, DEFAULT
FROM SYSIBM.SYSCOLUMNS
WHERE TBNAME = ‘DEPT’
AND TBCREATOR = ‘DSN8610’

SELECT *
FROM SYSIBM.SYSINDEXES
WHERE NAME = ‘XEMPL2’
AND CREATOR = ‘DSN8610’
 Sample queries
SELECT A.DBNAME, A.NAME, B.CREATOR, B.NAME
FROM SYSIBM.SYSTABLESPACE A, SYSIBM.SYSTABLES B
WHERE A.DBNAME = B.DBNAME
AND (A.STATUS = ‘P’ OR A.STATUS=‘S’)
ORDER BY 1,2,3,4;
 DB2 Directory
Residing at DSNDB01
Stored in RDBMS form but not accessible through
SQL
Its consists of following tablespaces:
SCT01 (Skeleton Cursor Table)
• Internal Form of SQL Statements
SPT01 (Skeleton Package Table)
• Created when you bind a package
SYSLGRNX (Long Range Tablespace)
• Used to track opening / closing of tablespaces, indexes, or partitions
SYSUTILX (System Utility)
• Contains row for every utility job running
DBD01 (Database descriptor)
• Internal information about database
 DB2 and OS/390
Address Spaces
Data Services (DSN1DBM1)
System Services (DSN1MSTR)
IRLM (IRLMPROC)
Stored Procedure (DSN1SPAS)
Distributed Data Facility (DSN1DIST)
 Various components

Storage Group Address

Spaces

User Data Sets Database Services

Bootstrap Data Sets System Services
Active Logs Locking Services
Archive Logs Distributed Data Facilities
Catalog Stored Procedures
Directory
 Various Subsystems

CICS IMS/DC IMS/DB TSO

Distributed
Database System
Data
Services Services
Facilities

Locking User
BSDS
Active
Data Logs
Services

Catalog Directory
Archive
Logs
 Buffer Pools (BPnn)
Also known as “Virtual Buffer Pools”
Temporarily stores the pages of table spaces and
indexes
Application program requesting a row of a table, DB2
retrieves of page from DASDs and placing it into
buffer pool
Buffer pool can reside at any of the following:
Within DB2’s DBM1 primary address space – limit 1.6 GB
Partly within DBM1 address space and using ESO HiperSpace
– limit 1.6 GB in DBM1 and 8 GB of ESO
Solely with MVS Data Space – limit 8 million buffers and for
32K Buffer pool – 256 GB
 Decisions related to BPs
Decisions are made during installation
How many virtual buffer pools of each size you need?
50 – 4 KB virtual buffer pool
10 each of 8 KB, 16KB, and 32 KB virtual buffer pools
Do you want buffer pools to use hiperspaces, data
spaces or neither?
Which buffer pool is to be the default for user data
and which is to be for indexes?
Buffer Pool Sizes
 Start Database
Controls availability of objects residing in that
database.
ACCESS(RO)
ACCESS(RW)
ACCESS(UT)

START DATABASE DBENR

SPACENAM(*)
ACCESS(RO)
 Monitoring Database
At console –
DISPLAY DATABASE(DBENR) SPACENAM(TSPU07)
 Stop Database
Make portion of database unavailable for normal
operations
Change access type to portion of database

STOP DATABASE DBENR

SPACENAM(*)
PART (*)
 Starting / Stopping DB2
Start DB2 at console -
START DB2 ACCESS(MAINT)
START DB2 ACCESS(*)
Stop DB2 at console –
STOP DB2 MODE(FORCE)
STOP DB2 MODE(QUIESCE)
Performance Tuning

Day# 2
 Performance Goals
Acceptable response time
A duration within which some percentage of all applications
have completed
First response from system to the application
Average throughput
The total number of transactions or queries that complete
within a given time
System Availability
MTTR (Mean-Time-To-Recover)
MTBF (Mean-Time-Between-Failure)
 Response Time

Response Time = Wait Time + Service Time

Application
Database Tuning
Tuning
 Improving response time
General Tips:
Reduce I/O Operations
Use RUNSTATS to keep access paths statistics current
Reserve free space in table spaces and indexes
Make buffer pool large enough
 Tuning Buffer Pools
90% 95% 97.5%

BPnn Fixed Thresholds SPTH DMTH IWTH

Updated Pages In-use Available Pages

Pages

Normal Read
Queue
Being Sequential prefetech
Handled queue

Queued per
Data set

Unavailable Pages Available Pages

 IWTH Threshold – 97.5%
Is checked whenever the page is to be updated
If exceeded updated page is written to DASD as soon
as the update completes
The write is synchronized with SQL request, and
request waits until the write has been completed and
the two operations are not carried out concurrently
 DMTH Threshold – 95%
Is checked before page is read or updated
If threshold has not been exceeded, DB2 accesses
the page in the buffer pool once for each page
If threshold has been exceeded, DB2 accesses the
page in the buffer pool once for each row
Means – retrieving or updating several rows in one
page causes several page access operations
 Size of Buffer pools
Hit Ratio
Measure of how often a page access(a getpage) is satisfied
without requiring I/O operation
Accounting reports show the hit ratio for specific applications
Accounting trace report shows the hit ratio for single threads
DB2 PM statistics report shows the hit ratio for subsystem
Hit Ratio = getpages – pages_read_from_DASD / getpages
Preferred hit ratio should be anything above 95%
 Tuning EDM Pool
EDM Pool contains:

Database descriptors (DBDs)

Skeleton Cursor Tables (SKCTs)
Cursor Tables (CTs)
Skeleton package tables (SKPTs)
Package Tables (PTs)
Skeleton of Dynamic SQLs
 Problems with EDM Pool
EDM Pool that is too small causes:

Increased I/O activity in DSNDB01.SCT02,

DSNDB01.SPT01 and DSNDB01.DBD01
Fewer threads used concurrently due to lack of
storage
 Efficiency of EDM Pool
CT Requests / CT not in EDM
PT Requests / PT not in EDM
DBD Requests / DBD not in EDM

A value of 5 or more is acceptable

Means hit ratio of 80%
 Tips for EDM Pool
Use Packages
Smaller objects in pool
Use RELEASE(COMMIT) when appropriate
Using bind option RELEASE(COMMIT) for infrequently used
packages and plans can cause objects to be removed from
the EDM Pool sooner
 Sort Pool Size
The maximum size of the sort work area allocated for
each concurrent sort user depends on the value you
specified for SORT POOL SIZE field on installation
panel DSNTIPC.
The default value is 1MB
 Estimating sort pool size
Total bytes =
No. of rows * (12 + sort key length + sort data length + 4)
For Example, tablex = 16000 rows
SELECT C1, C2, C3
FROM tablex
ORDER BY C1, C4;
If C1, C2, C3 and C4 are each 10 bytes in length, then
Sort size = 16000 * (12 + 20 + (10+10+10+100) + 4)
Sort size = 1216000
 Tips for Sort processing
The larger the sort pool, the more efficient the sort is
Minimize I/O contention on the I/O paths to the
physical work file
Allocate additional work files in the excess of the
defaults, and put those work files in their own buffer
pool
Sort only what is required to be sorted
 Application Tuning
Area of focus for SQL Tuning:
Indexes
• Selectivity of index = no. of distinct values in column / total no. of
values in column
• Unplanned full table scan
• Leading column in concatenated index
• Large table indexing???
Views
• Merge Join or sub-query
Joins
• Sequencing of tables
 Concurrency issues
Effects of Locks:
Suspension
• Example:
Using an application for inventory control, two users attempt to reduce
the quantity on hand of the same item at the same time. The two lock
requests are queued. The second request in the queue is suspended
and waits until the first request releases its lock.

• Effects:
The suspended process resumes running when:
– All processes that hold the conflicting lock release it.
– The requesting process times out or deadlocks and the process
resumes to deal with an error condition.
 Concurrency issues
Effects of Locks contd… …
Time out:

• Example:
An application process attempts to update a large table space that is
being reorganized by the utility REORG TABLESPACE with
SHRLEVEL NONE. It is likely that the utility job will not release control
of the table space before the application process times out.

• Effects:
DB2 terminates the process, issues two messages to the console, and
returns SQLCODE -911 or -913 to the process
 Concurrency Issues
Effects of Locks:
Deadlock:
• Example:

• Effects:
After a preset time interval (the value of DEADLOCK TIME), DB2 can
roll back the current unit of work for one of the processes or request a
process to terminate. That frees the locks and allows the remaining
processes to continue.
Concurrency Issues
 Concurrency Issues
Recommendations for design
Keep like things together
Plan for batch inserts
Use LOCKSIZE ANY until you have reason not to
Fewer rows of data per page, but not at the cost of I/O
Commit work as soon as is practical
Retry an application after deadlock or timeout
Close cursors
Bind plans with ACQUIRE(USE)
Bind with ISOLATION(CS)
 EXPLAIN for tuning
EXPLAIN provides information on:
The order in which tables will be accessed
How the tables will be accessed
Whether the index will be used
The types of joins and the order
Internal Sorts
Table locks
Tool which helps understanding optimizer
 EXPLAIN contd…
OPTIMIZER:
Checks the authority
Resolves names of tables, views and columns against the
catalog
Validates the semantics
Determines the best mechanism execute the SQL statement
Strategy decided based upon statistics stored in catalog
tables
 EXPLAIN contd…
Step# 1: Create PLAN_TABLE
 EXPLAIN contd…
Step# 2 Populate PLAN_TABLE table

BIND PLAN(<Plan Name>) MEMBER(<DBRMMEM>)

EXPLAIN(YES)

EXPLAIN PLAN
SET QUERYNO=1
FOR “explainable statement”
DB2 Utilities
 CHECK DATA
The CHECK DATA online utility checks table spaces
for violations of referential and table check
constraints,and reports information about violations
that are detected.
CHECK DATA also checks for consistency between a
base table space and a LOB table space.
 CHECK DATA contd…
Output:
CHECK DATA optionally deletes rows that violate referential
or table check constraints.A row that violates one or more
constraints is copied to an exception table.
If any violation of constraints is found,CHECK DATA puts the
table space being checked in the CHECK pending status.
On successful execution,CHECK DATA resets the CHECK
pending status.

Authorization Required:
STATS privilege for the database
DBADM,DBCTRL,or DBMAINT authority for the database
SYSCTRL or SYSADM authority
 CHECK DATA contd…
Phases:
UTILINIT Initialization
SCANTAB Extract foreign keys; use foreign key
index if it matches exactly, else scan
table
SORT Sort foreign keys if not extracted
from foreign key index
CHECKDAT Look in primary indexes for foreign
key parents,and issue messages to
report errors detected
REPORTCK Copy error rows into exception tables,
and delete them from source table if
DELETE YES is specified
UTILTERM Cleanup
 CHECK DATA contd…
Sample JCL:
 COPY
The COPY online utility creates up to four image
copies of any of the following objects:
Table space
Table space partition
Data set of a linear table space
Indexspace
Index space partition
There are two types of image copies:
A full image copy is a copy of all pages in a table space,
partition, data set, or index space.
An incremental image copy is a copy only of pages that have
been modified since the last use of the COPY utility.
 COPY contd…
The copies are used by the RECOVER utility when
recovering a table space or index space to the most
recent time or to a previous time.
SHRLEVEL
REFERENCE
CHANGE
 COPY contd…
Output:
Up to four sequential data sets containing the image copy.
Rows in the SYSIBM.SYSCOPY catalog table that describe
the image copy data sets available to the RECOVER utility.
Phases:
UTILINIT Initialization and setup
REPORT Reporting for CHANGELIMIT option
COPY Copying
UTILTERM Cleanup
 COPY contd…
Sample JCL:
 COPY contd…
More options:
 COPY contd…
Incremental Copy:
 LOAD
Use LOAD to load one or more tables of a table
space.
LOAD loads records into the tables and builds or
extends any indexes defined on them.
If the table space already contains data, you can
choose whether you want to add the new data to the
existing data or replace the existing data.
 LOAD
Output:
A loaded table space or partition
A discard file of rejected records
A summary report of errors encountered during
processing,generated only if you specify ENFORCE
CONSTRAINTS or if the LOAD involves unique indexes
 LOAD contd…
Sample Example:
 MERGECOPY
The MERGECOPY online utility merges image copies
produced by the COPY utility or inline copies
produced by the LOAD or REORG utilities.
It can also merge incremental copies with a full
image copy to make a new full image copy.
MERGECOPY operates on the image copy data sets of
a table space,and not on the table space itself.
 MERGECOPY contd…
Output:
A new single incremental image copy
A new full image copy
Authorization Required:
IMAGCOPY privilege for the database
DBADM, DBCTRL, or DBMAINT authority for the database
SYSCTRL or SYSADM authority
 MERGECOPY
Sample JCL:
 REORG
The REORG TABLESPACE utility reorganizes a table
space to improve access performance and reclaim
fragmented space.
Can reorganize a |single partition or range of
partitions of a partitioned table space.
 REORG contd…
Output:
Reorganizes all data,entire partitioning index,and all non-
partitioning indexes.

Authorization Required:
REORG privilege for the database
DBADM or DBCTRL authority for the database
SYSCTRL authority
SYSADM authority
 RUNSTATS
Online utility gathers summary information about the
characteristics of data in table spaces,indexes,and
partitions
DB2 records this information in the DB2 catalog and
uses it to select access paths to data during the bind
process.
RUNSTATS TABLESPACE gathers statistics on a table
space and optionally on indexes or columns
RUNSTATS INDEX gathers statistics only on indexes.
Use the STATISTICS keyword with LOAD,REBUILD
INDEX,and REORG jobs to eliminate the need to
execute RUNSTATS for updating catalog statistics.
 RUNSTATS contd…
Sample JCL: