Backing Up WebSphere Application Server With Tivoli Storage Management Redp0149

Front cover
Backing up WebSphere Application Server

with Tivoli Storage Management
WebSphere Application Server V3.5: backup/restore using TDP WebSphere Application Server V4.0: backup/restore using TSM Installation and configuration
Leos Stehlik Charlotte Brooks
ibm.com/redbooks
Redpaper
International Technical Support Organization Backing up WebSphere Application Server with Tivoli Storage Management June 2002
Take Note! Before using this information and the product it supports, be sure to read the general information in Notices on page vii.
First Edition (June 2002) This edition applies to Version 1 of Tivoli Data Protection for WebSphere Application Server, 5698-DPW for use with the Microsoft Windows NT, Microsoft Windows 2000 and IBM AIX. Comments may be addressed to: IBM Corporation, International Technical Support Organization Dept. QXXE Building 80-E2 650 Harry Road San Jose, California 95120-6099 When you send information to IBM, you grant IBM a non-exclusive right to use or distribute the information in any way it believes appropriate without incurring any obligation to you.
Copyright International Business Machines Corporation 2002. All rights reserved. Note to U.S Government Users Documentation related to restricted rights Use, duplication or disclosure is subject to restrictions set forth in GSA ADP Schedule Contract with IBM Corp.
Contents
Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix The team that wrote this Redpaper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Comments welcome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x Chapter 1. Overview of WebSphere Application Server V3.5 and backup. 1 1.1 WebSphere Application Server overview . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1.1 WebSphere Application Server architecture overview . . . . . . . . . . . . 2 1.1.2 Administrative server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1.3 Application server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1.4 Administrative database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1.5 Administrative console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1.6 Standard Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1.7 Advanced Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.1.8 Naming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.1.9 Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.1.10 Transactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.1.11 Workload management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.1.12 Open standards. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.2 Tivoli Storage Manager overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.2.1 TSM basic architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.3 WebSphere Application Server backup strategy . . . . . . . . . . . . . . . . . . . . 10 1.3.1 Why it is important to back up WebSphere Application Server . . . . . 10 1.3.2 What else needs to be backed up? . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.3.3 Considering the right strategy for your environment . . . . . . . . . . . . . 12 Chapter 2. Overview of TDP for WebSphere Application Server . . . . . . . 13 2.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.1.1 TDP for WAS features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.1.2 Design overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.1.3 How backup/restore really works . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.2 Prerequisites and supported environments . . . . . . . . . . . . . . . . . . . . . . . . 18 2.2.1 Known limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.3 Introducing TDP for WAS to your infrastructure . . . . . . . . . . . . . . . . . . . . 19 Chapter 3. Installing TDP for WAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.1 Deploying TDP for WAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Copyright IBM Corp. 2002
iii
3.1.1 WebSphere Application Server V3.5 setup . . . . . . . . . . . . . . . . . . . . 22 3.1.2 Preparing the TSM server for TDP for WAS . . . . . . . . . . . . . . . . . . . 22 3.1.3 Installing TSM client API on WAS nodes . . . . . . . . . . . . . . . . . . . . . 24 3.1.4 Preparing DB2 for using with TDP for WAS . . . . . . . . . . . . . . . . . . . 24 3.1.5 Installing TDP for WAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.1.6 Post-installation steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.2 Backup/restore with TDP for WAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 3.2.1 Our TDP for WebSphere lab scenario . . . . . . . . . . . . . . . . . . . . . . . 37 3.2.2 Backing up WebSphere using TDP for WAS . . . . . . . . . . . . . . . . . . 37 3.2.3 Querying the backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.2.4 Restoring WebSphere using TDP for WAS. . . . . . . . . . . . . . . . . . . . 40 3.2.5 Deleting unwanted backups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.3 Troubleshooting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.3.1 What to do when things go wrong . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.3.2 Recovering from failed TDP for WAS backup . . . . . . . . . . . . . . . . . . 46 Chapter 4. Backing up WebSphere V4.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.1 WebSphere 4.0 considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 4.1.1 WebSphere Application Server 4.0 backup/restore strategy . . . . . . 50 4.2 Planning for backup and recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 4.2.1 Introducing our testing environment . . . . . . . . . . . . . . . . . . . . . . . . . 51 4.2.2 Defining which objects we need to back up . . . . . . . . . . . . . . . . . . . 54 4.2.3 Backup procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 4.2.4 Restore procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.2.5 WebSphere Application Server V4.0 complete recovery . . . . . . . . . 62 4.2.6 Backing up and restoring a non-DB2 WAS environment . . . . . . . . . 63 Appendix A. TDP for WAS config files InitWAS.utl . . . . . . . . . . . . . . . . . . . . . . . . db2uext.utl . . . . . . . . . . . . . . . . . . . . . . . . vendor.env . . . . . . . . . . . . . . . . . . . . . . . . ....... ....... ....... ....... ...... ...... ...... ...... ....... ....... ....... ....... ...... ...... ...... ...... .. .. .. .. 65 66 70 72
Appendix B. WebSphere Application Server V4.0 backup scripts . . . . . . 73 B.1 Windows environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 B.2 AIX environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 Appendix C. Additional material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 Locating the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 Using the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 System requirements for downloading the Web material . . . . . . . . . . . . . . 78 How to use the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
iv
Backing up WebSphere Application Server with Tivoli Storage Management
Related publications . . . . . . . . . . . . . . . . . . . . . . IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . Other resources . . . . . . . . . . . . . . . . . . . . . . . . Referenced Web sites . . . . . . . . . . . . . . . . . . . . . . How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . IBM Redbooks collections . . . . . . . . . . . . . . . . .
...... ...... ...... ...... ...... ......
....... ....... ....... ....... ....... .......
...... ...... ...... ...... ...... ......
.. .. .. .. .. ..
81 81 81 81 82 82
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
Contents
vi
Notices
This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrates programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy, modify, and distribute these sample programs in any form without payment to IBM for the purposes of developing, using, marketing, or distributing application programs conforming to IBM's application programming interfaces.
vii
Trademarks
The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both: AIX Database 2 DB2 IBM Magstar MQSeries OS/390 OS/400 Redbooks(logo) Redbooks RISC System/6000 RS/6000 S/390 SP Tivoli WebSphere z/OS
The following terms are trademarks of other companies: Microsoft, Windows, Windows NT, Windows 2000 and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. Other company, product, and service names may be trademarks or service marks of others.
viii
Preface
This Redpaper describes how to back up and restore two different versions of WebSphere Application Server using Tivoli Storage Management products. WebSphere Application Server Version 3.5 and Version 4.0 are considered separately. WebSphere Application Server V3.5 can be backed up using Tivoli Data Protection for WebSphere Application Server. WebSphere Application Server V4.0 can be backed up using the Tivoli Storage Manager backup/archive client. This Redpaper presents an overview of WebSphere Application Server V3.5 and Tivoli Storage Management products, then shows you how to install, configure and run Tivoli Data Protection for WebSphere Application Server. For WebSphere Application Server V4.0, a script is created which will perform a backup of the administrative database and data files using operating system utilities and the Tivoli Storage Manager Backup/Archive client. We assume a basic knowledge of WebSphere Application Server and Tivoli Storage Management.
The team that wrote this Redpaper

This Redpaper was produced by a team of specialists from around the world working at the International Technical Support Organization, San Jose Center. Leos Stehlik is an IT Specialist for TSM and SAN solutions at the IBM Global Services in the Czech Republic. He has 5 years of experience in fields of UNIX, Windows NT/2000 and storage management. His previous publications include the IBM Redbook, Using Tivoli Storage Manager in a SAN Environment, SG24-6132 and he worked on the development of workshop material for Tivoli Storage Network Manager. Charlotte Brooks is a Project Leader for Open Tape and Storage Management Solutions at the International Technical Support Organization, San Jose Center. She has 11 years of experience with IBM in the fields of RISC System/6000 and storage management. She has written eight redbooks, and has developed and taught IBM classes worldwide on all areas of storage management. Before joining the ITSO in 2000, she was the Technical Support Manager for Tivoli Storage Management in the Asia Pacific region.
ix
Thanks to the following people for their contributions to this project: Mark Endrei International Technical Support Organization, Raleigh Center Ed Barton, Avishai Hochberg IBM Tivoli Storage Manager Development, San Jose Chris Zaremba, IBM Tivoli Storage Manager Development, Endicott Matthias Kubik IBM Tivoli Storage Manager Development, Boeblingen Yvonne Lyon, technical editor International Technical Support Organization, San Jose Center
Comments welcome
Your comments are important to us! We want our papers to be as helpful as possible. Send us your comments about this Redpaper or other Redbooks in one of the following ways: Use the online Contact us review redbook form found at:
ibm.com/redbooks
Send your comments in an Internet note to:

redbook@us.ibm.com
Mail your comments to the address on page ii.
Chapter 1.
Overview of WebSphere Application Server V3.5 and backup

In this chapter we provide an overview of the structure of WebSphere Application Server V3.5, plus the components needed to back it up using Tivoli Data Protection for WebSphere. We also explain why it is important to back up the WebSphere Application Server environment and introduce various backup strategies.
1.1 WebSphere Application Server overview

WebSphere Application Server is a core part of of IBMs WebSphere software a set of middleware products which enable the building, deployment and integration of high-performance Web sites with advanced e-business features using open standards. The general Web site for IBM WebSphere products is:
http://www.ibm.com/software/websphere
IBM WebSphere Application Server (WAS) provides a scalable, industrial strength deployment platform for e-business applications. The Standard Edition supports the standard Java APIs for developing dynamic Web content: Servlets, JavaServer Pages (JSP) and eXtensible Markup Language (XML). The Advanced Edition adds support for presenting business logic as Enterprise Java Beans (EJB) components. It also provides the capability to scale an application by distributing it across multiple physical machines, and the administrative tools needed to manage a distributed site. WebSphere Application Server and its supported technologies provide the ability to rapidly build sophisticated applications that are well structured and hence maintainable and extensible at e-business space. Note: Please note that for the purpose of this Redpaper we only discuss WebSphere architecture in a high-level overview. For additional detailed information on WebSphere Application Server, refer to these Redbooks:
WebSphere V3.5 Handbook, SG24-6161 IBM WebSphere V4.0 Advanced Edition Handbook, SG24-6161
1.1.1 WebSphere Application Server architecture overview

After installing and running either Standard or Advanced editions of WebSphere Application Server on a single machine, certain key processes will be running. This section gives a brief introduction to these processes and their purpose. Figure 1-1 gives a high-level overview of the major components that comprise a WebSphere Application Server instance.
application Application server(s) Server(s)
administrative console
administration server
administrative database
Figure 1-1 WebSphere Application Server components
The following sections describe the components shown in this figure.
1.1.2 Administrative server

The administrative server is the systems management runtime component of WebSphere. The administrative server is responsible for runtime management, security, transaction coordination, and workload management. In most cases (exceptions will be outlined later), the administrative server runs on all nodes in a WebSphere administrative domain and controls the interaction between each node and application server process in the domain.
1.1.3 Application server

Application code, servlets, JSPs, EJBs and their supporting classes run in an application server. Multiple application servers can be defined, each of which has its own Java Virtual Machine (JVM). The distribution of servlets, JSPs and EJBs among the application servers is user configurable.
Chapter 1. Overview of WebSphere Application Server V3.5 and backup
1.1.4 Administrative database

WebSphere stores all runtime configuration information for a domain in a single persistent repository. In Standard Edition this repository can be stored in InstantDB (which ships with the Standard Edition), DB2 or Oracle. Advanced Edition supports DB2, Oracle and Sybase. Different database versions are supported in different releases of WebSphere Application Server consult the release notes to ensure compatibility and support. In our diagram we show a single node running all processes, and this is common in small-scale development situations. It is entirely reasonable to configure the database on a remote server, and in production environments this is typically the case.
1.1.5 Administrative console

The administrative console is the graphical user interface used for administration of a WebSphere administrative domain. The administrative console can run on one of the nodes that the administrative server is running on, or it can be a remote node that attaches to a running administrative server.
1.1.6 Standard Edition

WebSphere Application Server Standard Edition is a single system, extremely easy-to-use, but complete solution for building an active Web site and basic Web applications that integrate with databases. Standard Edition can be used for applications producing both static and dynamic Web pages containing: Static HTML (HTML, .gif, .wav, etc.) HTML with imbedded client-side scripts, for example JavaScript Applications producing dynamic content with servlets and JSPs can also be developed. WebSphere Standard Edition does not provide the workload management (WLM) functionality that is available in WebSphere Advanced Edition, but does allow for multiple JVMs on a single physical server. WebSphere Standard Edition is also limited to a single node/machine unlike WebSphere Advanced Edition. These JVMs can be mapped to multiple virtual hosts on a single HTTP server to provide support for hosting multiple Web sites on a single application server.
1.1.7 Advanced Edition

WebSphere Advanced Edition extends the WebSphere Standard Editions functions across multiple machines to provide complete support for developing new high-performance, scalable and available, transactional Web-driven applications. WebSphere Advanced Edition focuses on new applications (JSPs and EJBs) that access relational databases for persistent state data. WebSphere Advanced Edition also supports distributed system management across the nodes in a distributed WebSphere Advanced Edition systems. The set of nodes that are administered collectively comprise a WebSphere administrative domain. An entire WebSphere domain can be administered from a single administrative console. The distributed WebSphere Advanced Edition architecture also requires other fundamental services. These are briefly outlined in the following sections.
1.1.8 Naming
In an object-oriented distributed computing environment, clients must have a mechanism to locate and identify the objects as if the clients and objects were all on the same machine. A naming service provides this mechanism. WebSphere uses the Java Naming and Directory Interface (JNDI) and the Lightweight Directory Access Protocol (LDAP) to provide a common front end to the naming service.
1.1.9 Security
WebSphere Advanced Edition allows access control to Web resources such as HTML pages and JSPs, and also to EJBs and the business methods they provide. Authorization to access a resource is permission-based. Access permissions can be granted to users or groups in order to control which users or groups can access the resource.
1.1.10 Transactions
A transaction is a set of operations that transforms data from one consistent state to another. Any realistic business application will have operations that require several updates to be made to a database, and for which, either all these operations should complete, or none should complete. For example, if a money transfer will debit one bank account and credit another, it would be a serious error if only one of the two updates were to occur.
Traditional implementations of such business process would require the programmer to place BEGIN and COMMIT transaction statements in the application code. One benefit of the EJB programming model is that transactional requirements are specified when configuring the EJB, not in the code. This makes the code much simpler to write. WebSphere Advanced Edition, in supporting EJBs, provides full transactional capabilities. These are implemented using the mechanism defined in the Java Transaction API (JTA).
1.1.11 Workload management

The workload management (WLM) functionality in WebSphere Advanced Edition introduces the notion of modelling of application server processes. Clones, which are instances of a model, can be created either on a single machine or across multiple machines in a cluster. In either case the WebSphere Advanced Edition WLM provides workload distribution and failover.
1.1.12 Open standards

Both WebSphere Standard and Advanced are based on and support key open-industry standards such as HyperText Transfer Protocol (HTTP), HyperText Markup Language (HTML), eXtensible Markup Language (XML), Secure Sockets Layer (SSL), Java, JavaBeans, Common Object Request Broker Architecture (CORBA), Lightweight Directory Access Protocol (LDAP), and most importantly the following Enterprise Java APIs: Enterprise JavaBeans (EJBs) are a reusable Java component for connectivity and transactions (EJB support is provided only in WebSphere Application Server Advanced Edition). JavaServer Pages (JSPs) represent inline Java code scripted within Web pages. Java Servlets are used in building and deploying server-side Java applications. Java Interface Definition Language (JIDL) supports objects whose interfaces are defined in CORBA IDL. JDBC is for connections to relational databases. WebSphere supports JDBC within its connection manager and within EJBs, for distributed database interactions and transactions. Java Messaging Service (JMS) is to be supported via MQSeries for asynchronous messaging and queuing and for providing an interface. Java Transaction Service (JTS) and Java Transaction API (JTA) are low-level APIs for interacting with transaction-capable resources such as relational databases. WebSphere uses these within EJBs for supporting distributed transactions.
Java Naming and Directory Interface (JNDI) is for communicating with directories and naming systems and is used in WebSphere Application Server to look up existing EJBs and interact with directories. Java Remote Method Invocation over Internet Inter-ORB Protocol (RMI/IIOP) is for communicating with Java objects in remote application servers.
1.2 Tivoli Storage Manager overview

Tivoli Storage Manager is part of the Tivoli Storage Management product set an enterprise-wide solution integrating automated network backup, archive and restore, storage management and disaster recovery. Tivoli Storage Manager is ideal for heterogeneous, data-intensive environments; supporting over 35 platforms and over 250 storage devices across LANs, WANs and SANs plus providing protection for leading database, collaborative and middleware applications. See Figure 1-2.
APPLE Macintosh
AUSPEX**
DATA GENERAL DG/UX
DIGITAL OpenVMS (SSSI)*** UNIX Tru64 UNIX
IBM*
HEWLETTPACKARD HP-UX AIX NUMA-Q AS/400 OS/2 OS/2 Lan Server OS/2 Warp OS/390 UNIX System Services
FUJITSU***
DB2 UDB
MICROSOFT -Windows XP Windows 95 Windows 98 Windows NT Windows NT DEC Alpha Windows 2000 Linux for Intel
INFORMIX
LOTUS DOMINO
Tivoli Storage Manager client platforms

Linux zSeries and OS/390
LOTUS NOTES NDMP filer (NetApps) MICROSOFT Exchange Server SQL Server ORACLE Oracle7 EBU SAP Oracle8 RMAN R/3 EMC Symmetrix SYBASE NOVELL NETWARE SEQUENT PTX NUMA-Q SILICON TANDEM GRAPHICS SUN GUARDIAN IRIX (ETI)*** MICROSYSTEMS Solaris SIEMENS NIXDORF SINIX SINIX Reliant UNIX SINIX 386/486
NSM VM OS/400 MVS AIX Solaris HP-UX
Tivoli Data Protection for application Family:

Lotus Domino and Notes Oracle Microsoft SQL Server Microsoft Exchange Informix R/3 NDMP WebSphere Application Server
Tivoli Storage Manager also supports:

IBM DB2 UDB Sybase
Tivoli Storage Manager server platforms
Windows NT Windows 2000
Disk
Optical
Storage Hierarchy
Tape
Figure 1-2 Tivoli Storage Manager supported platforms
Tivoli Storage Manager (TSM) allows users to confidently protect and manage information; it integrates unattended network backup and archive capabilities with centralized storage management and powerful disaster recovery functions. Tivoli Storage Manager is intended for companies with homogeneous or heterogeneous platforms and complex environments that include both traditional LANs as well as SANs. It is a best-of-breed, scalable storage management solution that helps provide consistent and reliable protection and management of mission-critical data that is spread across your company's enterprise. It protects a broad range of data across the enterprise from the laptop to the data center. Tivoli Storage Manager is an industrial-strength centralized storage management product for your enterprise, and can protect the following backup-archive clients: Windows 98/NT/2000, NetWare, Macintosh, as well as AIX, Sun Solaris, HP-UX, Linux and other UNIX variants as reflected in Figure 1-2. A Tivoli Storage Manager server is provided for OS/390, z/OS, Windows NT/2000, AIX, Solaris, HP-UX, and OS/400. This breadth of platform coverage affords you the choice in selecting the storage management platform that suits your environment and leverages your hardware and software investments. Tivoli Storage Manager can help control the cost of distributed storage management by leveraging storage resources, helping to reduce the cost of downtime and lost data, and helping to increase the productivity of storage administrators and end users. Tivoli Storage Manager exploits the numerous advantages of SANs with its LAN-Free and Library Sharing functions. These help to remove traffic from the LAN, allow for multiple Tivoli Storage Manger servers to share a library, and off load backup processing from mission-critical servers. Tivoli Storage Manager also supports server-free backup a method for backing up and restoring large volumes of data directly between client-owned disks and storage devices in a way which reduces overhead on the server and client, and which minimizes data transfer on the LAN. For more information about Tivoli Storage Management, visit its homepage
http://www.tivoli.com/products/index/storage_mgr
1.2.1 TSM basic architecture

TSM uses a client-server architecture where the server and client are defined as follows.
Server: A server is a computer system that provides services to one or more clients, or other devices over a network. A Tivoli Storage Manager server is the repository and manager of all the backed up client data. Administrative policies defined at the server control the types of backup performed and retention policies for the data. The server also manages the physical media and devices where the backed up data is stored. The TSM server consists of software installed on any of the supported platforms, along with storage devices where the backed-up client data will be located, and a catalog or database located on disk which tracks the data and its retention policies. Client: A client is a computer system that requests a service of another computer system that is typically referred to as a server. Multiple clients may share access to a common server. In Tivoli Storage Manager terms, a client is a computer system which has data assets requiring protection by the TSM server. The client decides what data will be backed up and is subject to the servers defined administrative policies for data retention. Typically, a clients data is backed up automatically by a server scheduled operation.
There are four basic types of client: Backup/Archive, HSM (Hierarchical Space Management), API, and Tivoli Data Protection (TDP). The Backup/Archive client provides basic backup (typically on a daily incremental basis) and long-term vital record retention, or archive functions for filesystem or operating system data. The backup/archive client can also backup special parts of the Windows operating system, such as the registry. The HSM client provides automatic and transparent movement of data from the client disk to the TSM server. If the user needs to access migrated data, it is dynamically and transparently restored to the client storage. The API client is a general purpose client providing an interface for applications to TSM storage management functions. The API includes function calls that can be used in an application to perform the following operations: Start or end a session Assign management classes to objects before they are stored on a server Back up or archive objects to a server Restore or retrieve objects from a server Query the server for information about stored objects Manage filespaces
TDP clients are written using the API and provide specialized backup and restore services for selected database, collaborative and middleware applications. Because the TDP clients are aware of the internal structures and operations of their applications, they can provide on-line, and often incremental backup operations. In this way, application environments can be backed up consistently and coherently.
1.3 WebSphere Application Server backup strategy

In this section we discuss why it is important to back up WebSphere Application Server data and mention some different possibilities for taking backups.
1.3.1 Why it is important to back up WebSphere Application Server

The demand for a backup policy will vary depending on the type of applications which are maintained in the WebSphere environment. In most of todays WebSphere implementations, losing WebSphere data and configuration would cause negative business impact. A WebSphere Application Server environment consists of different kinds of data, which of course, need to be treated in the appropriate way: Central database (DB2, Oracle) WebSphere file data binaries, configuration files Application data HTTP server configuration (for example, the httpd.conf file) Operating system specific data (for example, the system registry on the Windows operating system) One of the most critical configuration files is the admin.conf file, as it contains the JAVA classpath. As developers deploy new applications, a very common and convenient way to do this is to modify the JAVA classpath, appending the currently needed path to the existing CLASSPATH. This results in a somewhat lengthy and specific CLASSPATH which is critical to being able to locate applications. Knowing what type of data needs to be backed up, we can define what backup options are available: Full offline file-based backup Full offline database backup plus file data backup Full online database backup plus file data backup Full online objects based backup using Tivoli Data Protection for WebSphere Application Server Let us consider each of these options further.
10
Full offline file-based backup

For the purposes of this Redpaper, a full offline file-based backup means that the WebSphere Application Server and its database will be stopped, all necessary files will be backed up using Tivoli Storage Manager Backup/Archive client and the applications re-started. WebSphere will not be available for users during this kind of backup.
Full offline database backup with file data backup

This backup is similar to the full offline file-based backup the only difference is that native tools are used for database backup (the backup database command when using DB2, RMAN when using Oracle.). The database backup utility will utilize TSM API in order to be able to send backups to the Tivoli Storage Manager server. After the database backup is done, TSM backup-archive client file backup is run file backup to back-up WebSphere related files.
Full online database backup with file data backup

In this case, database backup is invoked by the DB backup utility (the backup database command when using DB2, RMAN when using Oracle), which utilizes TSM resources. The database is still up and running during the backup process. Afterwards you can invoke file-based backup with the TSM Backup/Archive client. You still need to stop WebSphere before this backup to ensure a consistent backup data.
Full online backup with TDP for WebSphere Application Server

On the WebSphere Master Node, Tivoli Data Protection for WebSphere creates a point-in-time snapshot of the WebSphere database and data files. Then a database backup is automatically invoked through the database backup utility. Once the backup is complete, TDP will start to send corresponding WebSphere data objects to TSM. On all WebSphere Controlled Nodes a point-in-time snapshot of appropriate WebSphere objects will be invoked and data will be sent to TSM. This is the only backup method which can operate without taking either the database or WebSphere Application Server offline.
Restriction: Tivoli Data Protection for WebSphere Application Server (TDP for WAS) is available only for WebSphere Application Server Version 3.5.
We will discuss the TDP for WebSphere architecture in more detail in Chapter 2, Overview of TDP for WebSphere Application Server on page 13.
11
1.3.2 What else needs to be backed up?

The TDP for WAS client presented in this Redpaper is designed to back up the WebSphere Application Server configuration data and the environment. Therefore TDP for WAS is a part of a total backup solution. It is important to be using a regular file-based backup for all WebSphere Application Server servers (for example, Tivoli Storage Manager Backup/Archive client) to ensure that all files in the filesystem, plus registry are backed up regularly. Application databases also require backup you can use the appropriate database backup tool for this. Typically the WebSphere Application Server environment does not change very often a PTF install, or deployment of a new application are examples. Therefore, TDP for WAS need not be run every night. However, the other system components should be backed up every night (using incremental backup techniques to reduce the total amount of data sent).
1.3.3 Considering the right strategy for your environment

Any IT infrastructure creates its own unique environment. Thats why choosing the right backup strategy will never be a straighforward process. When planning for backup/restore procedures, there are many aspects which need to be considered. Here are just a few examples: Backup window Bandwidth (network, storage, backup server) Amount of data Backup policy (number of versions, expiration period) Users demands for data availability Costs Disaster recovery requirements Security requirements For smaller to mid-scale WebSphere Application Server environments, you might consider taking online daily point-in-time full backups of WebSphere Application Server, and depending on your requirements weekly or monthly full offline backups (essentially archives from the TSM point of view). If your WebSphere Application Server environment is secured by a firewall (which is often true), you should be sure to enable the appropriate ports for backup. These are 1500 and 1501 for communication with the TSM server (by default local installations may vary) and 57321 for TDP for WAS. For large scale distributed WebSphere Application Server environments, detailed analysis is required, in order to understand and integrate WebSphere Application Server backup with other critical business systems and the overall business strategy. This exercise is beyond the scope of this Redpaper.
12
Chapter 2.
Overview of TDP for WebSphere Application Server

In this chapter we overview the architecture of Tivoli Data Protection for WebSphere Application Server (TDP for WAS). We explain how it works, the prerequisites, what versions and platforms are supported by current release, and what steps you need to take in order to successfully deploy TDP for WebSphere.
13
2.1 Architecture
In this section we look in detail at how TDP for WAS works. We discuss its main features, and also consider some features this product does not provide, and why.
2.1.1 TDP for WAS features

TDP for WAS provides the ability to archive and retrieve the data stored in a WebSphere administrative DB2 database (including servlets and Enterprise Java Beans as long as they are registered within the administrative database) and maintain DB2 redo log files. Note that although WebSphere Application can use a variety of products for its administrative database, TDP for WAS only works with DB2. TDP for WAS does not provide backup/restore operations for any application databases (used for example, by Web applications or servlets) even when these databases are registered in the WebSphere administrative database. You need to back up this data separately using the appropriate database backup utilities. Also, WebSphere binaries are not backed up with the TDP for WAS application. You can backup these files with the regular Tivoli Storage Manager Backup/Archive client.
2.1.2 Design overview

Let us now take a closer look at the TDP for WebSphere design and its components. TDP for WebSphere has the following components: TDP for WAS Main Module (TDPWS) Prole service Datamover DB2 User Exit DB2 Shared Vendor Library TDPPASSWD command line utility Figure 2-1 shows the dependencies and process flow between these TDP WebSphere components. The diagram shows two WebSphere Application Server servers and a database server. The TDPWS user command line interface communicates with the main Java application module. The main module sends requests to the Prole daemon, which then dispatches those requests to the Datamover for file-based objects and to the DB2 User Exit for the WebSphere administrative database backup. The Datamover then communicates with TSM through the Shared Vendor Library component using the TSM API.
14
TDP for WebSphere - design overview

Server A (Master Node)
Pro Prole
datamover Datamover
DB Server
WAS Admin WAS Admin Server Server TDP for WAS TDP for WAS Main Main Module
File File system system
Pro Prole
Log File Manager
datamover Datamover
Shared Vendor Library
DB2
Server B (Controlled Node)
TSM TSM Server Server
datamover Datamover
Pro Prole
datamover Datamover
File
File system system
All additional nodes are considered to be controlled nodes
Figure 2-1 TDP for WebSphere architecture overview
TDPWS
This component is a script which communicates with the main Java application. The TDPWS utility is responsible for the entire backup/restore process of all your WebSphere nodes. Please note that TDPWS is available (installed) on the master node only. Theres only one master node for the whole WebSphere domain. Before running the TDPWS command, you need to set the WAS_HOME environment variable. This process is described in Setting up environment variables on page 31.
Prole daemon
The Prole daemon receives tasks from TDPWS and forwards them to the other TDP WebSphere components, like Datamover and/or DB2 services (userexit, shared vendor library). The Prole daemon is actually responsible for executing backup and/or restore processes. Prole runs on all the WebSphere nodes where TDP for WAS has been installed.
Chapter 2. Overview of TDP for WebSphere Application Server
15
Datamover
As implied by the name of this component, the Datamover is responsible for all data transfers. It is directly controlled by the Prole daemon. The Prole daemon tells the Datamover where to move the particular data; and Datamover performs this operation, sending back a return code to the Prole daemon indicating if the operation succeeded or not.
DB2 User Exit

The DB2 User Exit program is more properly a DB2 component rather than a TDP for WAS component. However, the particular DB2UEXT binary shipped with the TDP for WAS product can only be used in conjunction with the WebSphere administrative database. Its primary responsibility is to read and transfer DB2 redo log files from the DB2 log path to the TSM client (using TSM API). The User Exit interfaces with Datamover in order to transfer log files to and from the TSM server.
DB2 Shared Vendor Library

The DB2 Shared Vendor Library implements the interface between the DB2 User Exit and the Datamover.
TDPPASSWD command line utility

Although this command line utility doesnt control any data transfer between TDP for WAS, DB2 and the TSM server, it is important for generating a TSM password file for the TSM API client. This is necessary because DB2 cannot use the PASSWORDACCESS GENERATE option for automated password handling when communicating with TSM server.
2.1.3 How backup/restore really works

In this section we explain how the TDP for WAS backup and restore process works.
Backup
When a backup is started through the tdpws command, a request for a full DB2 backup of the WebSphere administrative database is sent (through the Prole daemon) to the DB2 database. One of two possible DB2 backup methods may be used: Full database offline backup Full database online backup
16
Full database offline backup is used anytime when the WebSphere Application Server database has its BACKUP_PENDING flag set to on (for example, after a partial or failed backup, an update configuration for the database, and so forth). Otherwise a full online backup will be used. Note that the WebSphere Application Server server needs to be running when initiating a TDP for WAS backup. If it is not, only a full offline backup of the administrative database will be performed no application or configuration data will be backed up. Once the DB2 backup is complete, TDPWS starts to examine the WebSphere domain configuration, saving the configuration file in XML format. Now the backup process reaches its second stage, by parsing a list of objects intended for backup. Figure 2-2 shows the data flow among TDP WebSphere components.
TDP WebSphere data flow diagram

WAS Admin DB
DB2 instance DB2 Server Process DB2 Vendor API
TDP for WAS

TSM API
Prole daemon
Profile
Config file
TSM Server Storage Library
Figure 2-2 TDP WebSphere data flow diagram
17
Restore
When restore is started through the tdpws command, TDP WebSphere will query TSM server for list of available backups through TSM API. Then it displays this list; you need to choose the desired backup ID. Remember that backup ID names are case sensitive. TDPWS sends a request to the Prole daemon for a database restore in order to fully restore and recover the administrative database. After that, TDP for WAS will start to restore file data objects.
2.2 Prerequisites and supported environments

Here are the prerequisites and supported environments for TDP for WAS. Since support requirements do change, check the Web site for the most up to date information:
http://www.tivoli.com/support/storage_mgr/tdp_websphere.html
Supported operating systems:

AIX 4.3.2 or later with the latest maintenance level installed Windows NT (service pack 5 or higher) or Windows 2000 with the latest service pack
Prerequisite software:
WebSphere Application Server version 3.5 Standard or Advanced Edition with latest fixpack Tivoli Data Protection for WebSphere with latest available fixes (release 1.1.1.) DB2 UDB 7.1 or later recommended with latest fixpack supported with WebSphere Application Server v 3.5 TSM Server V4.1 or later on any supported platform TSM client API V4.1 or later on each node where TDP WebSphere will be installed
Restriction: Please note that WebSphere Application Server V4.0 is not supported by the current version of TDP for WAS. For details on how to manually back up a WebSphere V4.0 environment using Tivoli Storage Management, please refer to Chapter 4, Backing up WebSphere V4.0 on page 49.
18
2.2.1 Known limitations

Here is a list of some known limitations in the current release of TDP for WAS. Only one WebSphere administrative database is supported per DB2 instance. TDP for WAS can not be activated for any other DB2 database within the instance. TDP for WAS can not be activated for any other DB2 database than the WebSphere Application Server administrative database. If using the original release version of TDP for WAS (V1.1) and a backup of the administrative database fails, the instance must be restarted (refer to 3.3.2, Recovering from failed TDP for WAS backup on page 46). This is not necessary for versions of TDP for WAS at 1.1.1 and above. The delete function of TDP for WAS (see 3.2.5, Deleting unwanted backups on page 41) does not actually physically delete backup versions of the WebSphere backups in TSM. It will mark them as inactive you need to use the TSM management class parameters to set the retention period and number of versions to keep for expired objects to ensure that old backups will actually be deleted from the TSM database. DB2 on AIX requires an existing instance for a database restore. You need to create an empty database instance before starting a restore (create db <dbname>). DB2 on Windows cannot load the vendor library from a directory path with embedded spaces (for example D:\Program Files). Do not use the default installation directory, follow exactly the installation instructions we describe in 3.1.5, Installing TDP for WAS on page 25.
2.3 Introducing TDP for WAS to your infrastructure

For the purpose of this section, we assume that you already have your WebSphere infrastructure installed and running and that a TSM server is also available. We will focus only on things you need to consider before implementing TDP for WAS into the WebSphere infrastructure. The following components will be affected in some way during TDP for WAS implementation: WebSphere Application Server DB2 database Tivoli Storage Manager
19
If you plan to install TDP for WAS into a production environment, remember that you need to schedule a shutdown for the WebSphere nodes and database servers. Thats because some configuration changes are required in the DB2 database environment in order to use TDP for WAS properly. These changes require DB2 to be offline, which will of course affect production WebSphere systems. For more detailed information on the changes necessary to set up the environment for TDP for WAS installation, see 3.1, Deploying TDP for WAS on page 22. How long of an outage for will be required for TDP for WAS implementation? It depends primarily on the scale of your environment (number of WebSphere nodes, size of the WebSphere Application Server administrative databases), throughput of your TSM backup system and so on. This is because the DB2 configuration change needed for TDP for WAS requires an immediate full offline backup of that database depending on the size of the database. If the database is large this may take some time. From the TSM perspective, few changes are necessary, and an outage may not be required. You need to define a policy for your WebSphere backups, register the client nodes, create storage pools, and define media into them. This can all be done online. You may then need to consider changing the time-out parameters for client sessions in the main TSM configuration file dsmserv.opt as described in Modify server options file on page 23. This change does require the TSM server to be restarted.
20
Chapter 3.
Installing TDP for WAS

In this chapter we describe how to install Tivoli Data Protection for WebSphere Application Server (TDP for WAS). These topics are covered: Setup necessary on the Tivoli Storage Manager server side before you install TDP for WAS DB2 setup Description of the configuration files, location, environment variables, password handling, tdppasswd usage Basic backup and restore operations Troubleshooting section error messages, tracing, advanced diagnostics
21
3.1 Deploying TDP for WAS

In this section we describe the installation process and all the necessary pre-installation and post-installation steps.
3.1.1 WebSphere Application Server V3.5 setup

For the purpose of this Redpaper, we assume that WebSphere Application Server V3.5 is installed in the default location, it uses a local DB2 database as its database engine, and the IBM HTTP server is up and running. WebSphere Application Server uses a database called WAS as the administrative database. If you need any further information on how to install WebSphere application server, please refer to the product manuals or to the Redbook WebSphere V3.5 Handbook, SG24-6161.
3.1.2 Preparing the TSM server for TDP for WAS

In this section we provide a brief overview of how to configure the TSM server in order to use it with TDP for WAS. We assume that your TSM server is already installed with a storage device configured. Please note that we cover only those TSM topics, which are necessary for the purpose of this Redpaper. If you need any further information on how to install and configure a TSM server environment, please refer to the product manuals for your TSM platform. These are available at:
http://www.tivoli.com/support/public/Prodman/public_manuals/td/TD_PROD_LIST.html
Creating storage pool and policy definitions

You need two storage pools one for the WebSphere Application Server administrative database backup, and one for data files from your WebSphere Application Server nodes. Create those storage pools as shown in Example 3-1. We use the storage pool names was_db for the database backup and was_nodes for the data files.
Example 3-1 Create storage pools
define stgpool was_db 3570 maxscratch=5 define stgpool was_nodes 3570 maxscratch=5
22
Specify the correct device_class_name for your environment. Now we need to create and activate a policy definition for our WebSphere data, associating our storage pools with the management class. Our policy domain is called dom_was, with a policy set dom_was and management classes mdata, mdb and mlog. The DB2 database backups and redo logs will use the mdb and mlog management classes respectively, and the WebSphere Application Server data files will use the mdata management class. The class mdata is associated with the storage pool was_nodes and the management classes mdb and mlog use the storage pool was_db. See Example 3-2.
Example 3-2 Creating and activating backup policy
define domain dom_was define policyset dom_was dom_was define mgmtclass dom_was dom_was mdata define mgmtclass dom_was dom_was mdb define mgmtclass dom_was dom_was mlog define copygroup dom_was dom_was mdata standard destination=was_nodes define copygroup dom_was dom_was mdb standard destination=was_db define copygroup dom_was dom_was mlog standard destination=was_db define copygroup dom_was dom_was mdata standard destination=was_nodes t=a define copygroup dom_was dom_was mdb standard destination=was_db t=a define copygroup dom_was dom_was mlog standard destination=was_db t=a assign defmgmgtclass dom_was dom_was mdb activate policyset dom_was dom_was
It is not necessary to specify versioning information in the TSM backup copy groups, because TDP for WAS uses the TSM archiving function for operation and therefore all data is processed according to the archive copy group definition. Finally, you need to register a client node in the TSM server for the WAS node defining it to the policy domain set up for WebSphere (dom_was). You will log in under this account to the TSM server from TDP for WAS. See Example 3-3.
Example 3-3 Register node for backing up Webpshere in TSM server
register node was <password> domain=dom_was maxnummp=2
Set the MAXNUMMP value (how many mount points this TSM client node can allocate) according to your environment.
Modify server options file

For most database related backups it is necessary to increase the default values for COMMTIMEOUT and IDLETIMEOUT in the server options file dsmserv.opt.
Chapter 3. Installing TDP for WAS
23
If you forget to change these parameters, and you start a backup session (or any other backup which uses TSM API as the communication interface), the TSM server might cancel that session if one of the timeout values is exceeded. If that happens, your backup will fail since even if the client is able to restart its backup session, it will be using a different session ID; however, the backup utility is still expecting the original session ID. We recommend that you set these values as shown in Example 3-4.
Example 3-4 Modify timeout options in DSMSERV.OPT configuration file
COMMTIMEOUT 600 IDLETIMEOUT 45
A TSM server restart is required to activate changes to the server options file.
3.1.3 Installing TSM client API on WAS nodes

In this step, install TSM Backup/Archive client and the client API on all your WebSphere nodes which you plan to back up with TDP for WAS. Theres no additional configuration necessary, you only need to install it.
3.1.4 Preparing DB2 for using with TDP for WAS

TDP for WAS is fully integrated with the native DB2 commands for database backups. In order to take online DB2 backups, your WebSphere Application Server administrative database must be set to logretain mode and must be enabled to use the userexit program shipped with TDP for WAS.
Attention: After applying the changes described below, you wont be able to start the WebSphere server until you run a full offline DB2 backup of your WebSphere administrative database. This is because setting the logretain and userexit parameters switches the database to a Backup pending mode. However, before running a DB2 full backup, you need to set additional TSM environment variables. You can find information on how to prepare your environment in Setting up environment variables on page 31.
Example 3-5 shows how to apply these changes from the DB2 Command Line Processor.
Example 3-5 Update DB2 WebSphere administrative database settings
db2> update db cfg for was using logretain on db2> update db cfg for was using userexit on
24
Now we need to disable database parallel recovery. Quit the DB2 Command Line Processor and issue the command shown in Example 3-6 from the regular operating system command line:
Example 3-6 Disabling database recovery parallelism
db2set DB2_USE_PARALLEL_RECOVERY=FALSE
More information on backing up DB2 using TSM (including full installation details) is available in the Redbook Backing up DB2 Using Tivoli Storage Manager, SG24-6247. We are now ready to install TDP for WAS.
3.1.5 Installing TDP for WAS

In this section, we will show the installation process on Windows 2000 and AIX platform.
Windows 2000 installation

In order to install TDP for WAS on a WebSphere node, run setup.exe from your installation directory. An Installation Wizard will start (Figure 3-1) to guide you through the installation.
Figure 3-1 Install Wizard startup
25
Click the Next button to continue with the installation. You will select the installation type as shown in Figure 3-2.
Figure 3-2 Select type of installation
Choose Master Node installation if youre installing TDP for WAS on the master WebSphere node in your domain. This type of installation will install all the files necessary to perform backup/restores of your complete distributed WebSphere environment. If you choose Controlled Node installation, setup will install only those files which are necessary to back up/restore a local WebSphere node. Backup/restore operations are controlled (invoked) by the master node.
Custom installation gives full control over what packages will be installed and is not recommended.
Select the destination directory where the TDP for WAS binaries and configuration files will be installed as shown in Figure 3-3. Do not use the default Program Files folder. Many applications do not correctly handle folder names containing spaces, and we observed DB2 backup errors when using the default installation path.
26
Figure 3-3 Choose installation folder
Attention: Do not use the default ?:\Program Files\tdpws directory. Choose an alternative folder without spaces (we used C:\TDPWS).
27
Figure 3-4 shows the next screen with a valid installation directory specified. Click Next to continue.
Figure 3-4 Choose install destination other than Program files folder
You will be asked to confirm the installation options before the install process starts to copy the files. Once installation program finishes copying files, a new service is added to the systems registry as shown in Figure 3-5. This gives you the ability to control the Prole service.
Figure 3-5 Creates entries for system services
Tip: If you need to understand the role of the Prole service or other components of the TDP for WAS, please read 2.1, Architecture on page 14.
28
Figure 3-6 shows the Windows Services panel with the Prole service installed and running after TDP for WAS installation.
Figure 3-6 Prole installed as a Windows system service
As shown in Figure 3-7, the installation process now asks you if you wish to set your environment variables. Do not do this now. Setup will set the DSMI_CONFIG variable as a Windows User Variable, which will not work with the DB2 database. DB2 requires to have the DSMI_CONFIG variable set as System variable, therefore click No here.
Figure 3-7 Setting environment variables
The installation is now complete.
29
AIX installation
To install TDP for WAS on your AIX machine, perform the following: 1. Mount the TDP for WAS installation CD, for example: mount /cdrom 2. Switch to the installation directory /cdrom/usr/sys/inst.images/ : cd /cdrom/usr/sys/inst.images 3. Start SMIT based installation: smitty install_latest 4. Specify the installation directory as a installation directory. 5. Choose all_latestunder Software to install. Your SMIT screen should look like Example 3-7. 6. Press Enter to start the installation.
Example 3-7 SMIT screen for installing TDP for WAS
Install and Update from LATEST Available Software Type or select values in entry fields. Press Enter AFTER making all desired changes. [Entry Fields] /cdrom/usr/sys/inst.im> [_all_latest] no yes no yes yes no no yes no yes
* INPUT device / directory for software * SOFTWARE to install PREVIEW only? (install operation will NOT occur) COMMIT software updates? SAVE replaced files? AUTOMATICALLY install requisite software? EXTEND file systems if space needed? OVERWRITE same or newer versions? VERIFY install and check file sizes? Include corresponding LANGUAGE filesets? DETAILED output? Process multiple volumes?
TDP for WAS will be installed in the directory /usr/tivoli/tsm/tdpws. Now follow the steps described in 3.1.6, Post-installation steps on page 31. Please note that you need to adapt the procedure (setting environment variables and so on) according to the specifics of your operating system environment.
30
3.1.6 Post-installation steps

In this section we cover the necessary post-installation steps in order to get TDP for WAS working properly.
Setting up environment variables

After installation, we need to set up the environment variables properly, otherwise TDP for WAS will not work. To avoid possible problems, we recommend that you set up all variables as System variables. For example, DB2 requires to have the DSMI_CONFIG variable set as a System variable because, if this is set as a User variable, DB2 wont be able to perform backup/restore operations. In Table 3-1, we show the list of variables which you need to set up. We assume that TDP for WAS has been installed in the directory c:\tdpws, and that the TSM Backup/Archive and API clients are installed in the default directories of c:\Program Files\Tivoli\tsm\baclient and c:\Program Files\Tivoli\tsm\API.
Table 3-1 Setting up environment variables
Variable Name DB2_DIAG_PATH DB2_UEXT_PROFILE DB2_VENDOR_INI XINT_PROFILE DSM_CONFIG DSMI_CONFIG DSM_DIR DSMI_DIR DSM_LOG DSMI_LOG WAS_HOME Value (example) c:\db2\db2dump c:\db2\sqllib\db2uext2.utl c:\db2\sqllib\vendor.env c:\db2\sqllib\initwas.utl c:\tdpws\dsm.opt c:\tdpws\dsm.opt c:\progra~1\tivoli\tsm\baclient c:\progra~1\tivoli\tsm\api c:\tdpws\ c:\tdpws\ c:\websphere\appserver
In order to set system variables, right-click the MyComputer icon on your desktop, choose Properties -> Advanced and then click Environment Variables as shown in Figure 3-8. Add the new variables in the System variables heading.
31
Figure 3-8 Managing environment variables
Re-start DB2
After setting up the environment variables, you need to restart the DB2 database. This is the only way for the DB2 database to re-read the changed environment on the Windows platform.
Add DB2 configuration data

Now you need to create a folder to store all the configuration data necessary for TDP for WAS to operate with the DB2 database. We chose the name C:\DB2\SQLLIB you should not include spaces in the folder name. Now copy the following files from the TDP for WAS install directory into this new folder: agent.lic initWAS.utl db2uext2.utl db2uext2.exe db2tadsm.dll Now we need to edit the profile files.
32
Editing db2uext2.utl file

In this file, you need to change the following values according to your environment: LOG_DB_NAME This contains the name of the WebSphere Application Server Administrative database (WAS by default). LOG_USEREXIT_LOGPATH This contains the path for the Log File Manager's logs and control files. This is a subdirectory of the DB2 configuration data folder created in Add DB2 configuration data on page 32. LOG_DB_NODE This contains the WebSphere DB node name. Example 3-8 shows how we set those values in our environment.
Example 3-8 Modify db2uext2.utl file
LOG_DB_NAME WAS LOG_USEREXIT_LOGPATH"c:\db2\sqllib\db2dump LOG_DB_NODE NODE0000
Please note that the LOG_USEREXIT_LOGPATH uses the directory we created in Setting up environment variables on page 31. The full db2uext2.utl file listing is shown in Appendix A, TDP for WAS config files on page 65.
Editing initWAS.utl file

You need to set the following values in your initWAS.utl file: CONFIG_FILE This points to the TDP for WAS configuration file (initWAS.bki in the TDP for WAS installation directory). SERVER This points to the same SERVERNAME set in the dsm.opt file (see Create dsm.opt file on page 35). SESSIONS This specifies how many sessions will be opened to TSM server. This is limited by the value specified by MAXNUMMP in the nodes TSM definition (using TSM command REGISTER NODE or UPDATE NODE) if the backups are to be sent straight to a tape storage pool. PASSWORDREQUIRED This is a password handling option; if set to YES, then the PASSWORD option must be defined in dsm.opt file. ADSMNODE This is the TSM client node name which TDP for WAS will use for backup/restore operations we defined this as was in Example 3-3 on page 23.
33
BACKUPMGTCLASS This specifies the TSM management class for storing all data, as specified in Creating storage pool and policy definitions on page 22. ARCHIVEMGTCLASS This specifies the TSM management class for storing the DB2 offline log files, as specified in Creating storage pool and policy definitions on page 22. MAXSESSIONS This specifies the number of total parallel sessions which will be established by Tivoli Data Protection for WebSphere Application Server. This number should correspond to the number of simultaneously available tape drives specified for the Tivoli Storage Manager server.
Note: Please be aware that here we only mention those parameters you need to change in order to set up TDP for WebSphere correctly. There are many other parameters you can set up if required for your environment or diagnostic purposes. Refer to the product documentation or the sample configuration file provided for a detailed description of all parameters.
Example 3-9 shows how we have modified our initWAS.utl file for our lab environment. The full initWAS.utl file listing is shown in Appendix A, TDP for WAS config files on page 65.
Example 3-9 Modify initWAS.utl file
CONFIG_FILE c:\tdpws\initWAS.bki" LOG_SERVER server_a DETAIL SERVER server_a SESSIONS 2 PASSWORDREQUIRED YES ADSMNODE was BACKUPMGTCLASS MDB ARCHIVEMGTCLASS MLOG MAXSESSIONS 2
Editing vendor.env file

Now we will modify values in the vendor.env file, which supplies additional environment variables for DB2. Your vendor.env file is located according to the environment variable DB2_VENDOR_INI (see Setting up environment variables on page 31), which in our case points to c:\db2\sqllib\vendor.env. Example 3-10 shows the vendor.env file modified for our environment.
34
Example 3-10 Example of vendor.env file

XINT_PROFILE=c:\db2\sqllib\initWAS.utl DB2_DIAGPATH=c:\db2\sqllib\db2dump DB2_UEXT2_PROFILE=c:\db2\sqllib\db2uext2.utl DSMI_DIR=c:\progra~1\tivoli\tsm\api DSMI_CONFIG=c:\tdpws\dsm.opt
XINT_PROFILE This points to the TDP for WAS profile weve copied to the c:\db2\sqllib directory in Add DB2 configuration data on page 32. DB2_DIAGPATH This points to the standard DB2 location for storing log files and traces. DB2_UEXT2_PROFILE This points to the DB2 userexit profile file weve copied to the c:\db2\sqllib directory in Add DB2 configuration data on page 32. DSMI_DIR This points to the TSM API directory. Its value must correspond with the value of the DSMI_DIR variable defined in Setting up environment variables on page 31. DSMI_CONFIG This points to the DSM.OPT configuration file for TSM API and must correspond with the value of the DSMI_CONFIG variable defined in Setting up environment variables on page 31. The full vendor.env file listing is shown in Appendix A, TDP for WAS config files on page 65. The next step is to create dsm.opt file.
Create dsm.opt file

Create a file, dsm.opt, in the TDP for WAS installation directory. This file contains the TSM client node configuration definitions. See Example 3-11.
Example 3-11 Create dsm.opt file
servername server_a commm tcpip tcps brazil tcpp 1500 nodename was passwordaccess prompt
SERVERNAME This must match the SERVER definition in the initWAS.utl file (see Editing initWAS.utl file on page 33). TCPS This is the hostname or TCP/IP address of the TSM server.
35
NODENAME This is the TSM nodename, and must match the value specified in ADSMNODE in the initWAS.utl file. Now copy the dsm.opt file in the TDP for WAS directory into a new file called <SERVERNAME>.opt, where <SERVERNAME> is the value of the SERVERNAME parameter in your dsm.opt file. Our file is copied to server_a.opt.
Run tdppasswd command utility

Now in the last step, you need to run the tdppasswd command line utility in order to generate an encrypted TDP password file. This command also serves as a check to see if your TDP for WAS configuration is correctly set up. See Example 3-12.
Example 3-12 Output of the tdppasswd utility
C:\tdpws>tdppasswd -p "c:\tdpws\initwas.utl" Tivoli Data Protection for WebSphere Application Server - Version 1, Release 1, Level 0 (c) Copyright IBM Corporation, 2002, All Rights Reserved. BKI2000I: Successfully connected to ProLE on port 57321. BKI0049I: Please enter password for node WAS on server SERVER_A: *** BKI0051I: Password successfully verified for node WAS on server SERVER_A.
You should be able to connect to the TSM server there will be an entry in the TSM activity log to show that the session for your WebSphere node started successfully as shown in Example 3-13. Note that the session shows up as type TDP R3.
Example 3-13 Output of the TSM activity log console
ANR0406I Session 11670 started for node WAS (TDP R3 WINNT) (Tcp/Ip 9.1.38.187(3041)). ANR0403I Session 11670 ended for node WAS (TDP R3 WINNT).
At this point we recommend that you reboot the machine.
3.2 Backup/restore with TDP for WAS

In this section we show how to test the basic TDP for WAS backup/restore commands.
36
3.2.1 Our TDP for WebSphere lab scenario

Figure 3-9 shows our TDP for WAS test environment.
POLONIUM Windows 2000
BRAZIL AIX 4.3.3
LAN
WebSphere Appl. Server DB2 TSM B/A Client TDP for WAS
SCSI Tape Library
TSM Server
M IB
Figure 3-9 Simple test scenario for TDP for WAS
We have a TSM V4.2 server called BRAZIL running on AIX with a Magstar SCSI 3570 library connected to it. Server POLONIUM is a Windows 2000 Intel machine with WebSphere Application Server (Advanced Edition) V3.5 installed The administrative database uses DB2 7.1 with Fixpack 3. We also have the TSM API & Backup/Archive client version 4.1.2 installed plus TDP for WebSphere Application Server at V1.1.1.
3.2.2 Backing up WebSphere using TDP for WAS

Before using TDP for WAS backup/restore functionality, make sure that your DB2 database backup utility is properly configured and working with the TSM server. You can check the DB2 to TSM connectivity by issuing the db2adutl query command from the command line (to open the DB2 command line, click Start -> IBM DB2 -> Command Line Processor). Your output should look similar to Example 3-14:
Example 3-14 Verifying that DB2 can connect to TSM
C:\PROGRA~1\SQLLIB\BIN>db2adutl query Warning: No DB2 backup images found in ADSM for any alias.
37
After running this command, the TSM console will show that a session for your backup node was started and ended on the TSM server. Confirm that the WebSphere Application Server is up and running. TDP for WAS provides a command line utility for backup/restore/query operations which allows operations to be automated through scripts. To back up the WebSphere Application Server, change to the TDP for WAS installation directory and enter the tdpws command as shown. You should see output similar to Example 3-15:
Example 3-15 Sample output of the tdpws command
C:\tdpws>tdpws -p "c:\tdpws\initwas.utl" -f backup Tivoli Data Protection for WebSphere Application Server - Version 1, Release 1, Level 0 (c) Copyright IBM Corporation, 2001, All Rights Reserved. DKP0005I: Start of program at: Apr 23, 2002 7:05:00 PM . DKP8000I: Host "localhost": start backup of database "was".
At this point you should be able to see that TDP for WAS started a client session with the TSM server and is waiting for media mount (if you are using a tape storage pool). See Example 3-16.
Example 3-16 TSM console TDP session started and appropriate tape mounted
tsm: SERVER1>q ses Session established with server SERVER1: AIX-RS/6000 Server Version 4, Release 2, Level 0.0 Server date/time: 04/23/02 17:13:18 Last access: 04/23/02 Sess Number -----11,669 11,674 11,675 Comm. Method -----Tcp/Ip Tcp/Ip Tcp/Ip Sess Wait Bytes Bytes Sess Platform State Time Sent Recvd Type ------ ------ ------- ------- ----- -------Run 0 S 3.1 K 140 Admin WinNT RecvW 0 S 1.9 K 7.8 M Node TDP WAS Run 0 S 120 114 Admin AIX
12:49:43
Client Name -------------------ADMIN WAS ADMIN
tsm: SERVER1>q mount ANR8330I 3570 volume 083964 is mounted R/W in drive DR0 (/dev/rmt0), status: IN USE. ANR8334I 1 volumes found.
38
After a while you should be able to see a message in the tdpws command window that the database backup has been completed successfully and that TDP will begin backing up file data. See Example 3-17.
Example 3-17 TDP for WAS database backup completed and file data backup started
C:\tdpws>tdpws -p "c:\tdpws\initwas.utl" -f backup Tivoli Data Protection for WebSphere Application Server - Version 1, Release 1, Level 0 (c) Copyright IBM Corporation, 2001, All Rights Reserved. DKP0005I: Start of program at: Apr 23, 2002 7:05:00 PM . DKP8000I: Host "localhost": start backup of database "was". DKP8002I: Host "localhost": backup of database "was" with ID 20020423190501 successfully finished. DKP8015I: Examining configuration. DKP8003I: Host "polonium": start backup.
Now TDP for WAS will create a list of file data objects to be backed up (as was indicated in Example 3-17 with the message DKP8015I). Then it will start to send those files to the TSM server. The list of objects to be backed up will be processed as a batch. TDP for WAS starts as many parallel sessions as specified with MAXSESSIONS in the profile file initWAS.utl (see Editing initWAS.utl file on page 33). These sessions remain open until there are no more objects to back up/restore see Example 3-18.
Example 3-18 TSM Console individual sessions for file data backup ANR0406I Session 12261 started for node WAS (TDP WAS WIN32) (Tcp/Ip 9.1.38.187(3685)). ANR0406I Session 12262 started for node WAS (TDP WAS WIN32) (Tcp/Ip 9.1.38.187(3686)).
Eventually, the backup processing will end with return code 0 as shown in Example 3-19. This indicates that the backup process was successful.
Example 3-19 Successful backup of WebSphere node
C:\tdpws>tdpws -p "c:\tdpws\initWAS.utl" -f backup Tivoli Data Protection for WebSphere Application Server - Version 1, Release 1, Level 0 (c) Copyright IBM Corporation, 2001, All Rights Reserved. DKP0005I: Start of program at: Apr 24, 2002 2:20:22 PM .
39
DKP8000I: Host "localhost": start backup of database "was". DKP8002I: Host "localhost": backup of database "was" with ID 20020424142023 succesfully finished. DKP8015I: Examining configuration. DKP8003I: Host "polonium": start backup. DKP8004I: Host "polonium": backup with ID WAS___9C04241421 successfully finished. DKP0020I: End of program at: Apr 24, 2002 2:45:13 PM . DKP0024I: Return code is: 0.
3.2.3 Querying the backup

The TDP for WAS tdpws command includes an inquiry option. TDP will establish a connection to the TSM server and issue a query for all backup IDs which belong to that particular master node. You can invoke the inquire function by issuing the tdpws command from the TDP WebSphere directory as shown. This command returns the number of TDP for WAS backups found in the TSM server and the backup ID for each one. Example 3-20 shows the output of the tdpws inquire command.
Example 3-20 Output of the TDP WebSphere inquire command
C:\tdpws>tdpws -p "c:\tdpws\initWAS.utl" -f inquire Tivoli Data Protection for WebSphere Application Server - Version 1, Release 1, Level 0 (c) Copyright IBM Corporation, 2001, All Rights Reserved. DKP0005I: Start of program at: Apr 24, 2002 2:48:52 PM . DKP8018I: Found 1 backup ID's: WAS___9C04241421 DKP0020I: End of program at: Apr 24, 2002 2:49:00 PM . DKP0024I: Return code is: 0
3.2.4 Restoring WebSphere using TDP for WAS

Before beginning a restore of WebSphere Application Server, you need to stop the WAS administration server. You can do that from Windows 2000 Services window.
40
The WebSphere Application Server restore process also utilizes the tdpws command. This command will query the TSM server for available backups. Select the desired backup ID and enter it at the command prompt. Please note that backup ID names are case sensitive. Example 3-21 shows a successful restore session.
Example 3-21 Restoring WebSphere Application Server with TDP for WAS
C:\tdpws>tdpws -p "c:\tdpws\initwas.utl" -f restore Tivoli Data Protection for WebSphere Application Server - Version 1, Release 1, Level 0 (c) Copyright IBM Corporation, 2001, All Rights Reserved. DKP0005I: Start of program at: Apr 25, 2002 12:47:26 PM . DKP8018I: Found 5 backup ID's: WAS___9C04181203 WAS___9C04181121 WAS___9C04172021 WAS___9C04171851 WAS___9C04171822 DKP8016I: Please enter the backup ID to restore and press ENTER: WAS___9C04181121 DKP8011I: Querying for files for backup with ID WAS___9C04181121. DKP8005I: Now restoring 1270 files for backup with ID WAS___9C04181121. DKP8006I: Host "localhost": start restore of database "was". DKP8007I: Host "localhost": database "was" succesfully restored. DKP0020I: End of program at: Apr 258, 2002 1:03:29 PM . DKP0024I: Return code is: 0.
3.2.5 Deleting unwanted backups

The tdpws command includes a delete option which deletes unneeded backups from the TSM server. To do this, use the delete option on the tdpws command. This command will query the TSM server for available backups, and displays a list. To delete a particular backup, enter the backup ID. Remember that backup IDs are case sensitive.
Restriction: Note that this delete function does not physically delete the backups from TSM it marks them as inactive. Normal management class parameters apply for the number of versions to keep and retention period for expired objects as mentioned in 2.2.1, Known limitations on page 19.
41
Example 3-22 shows an example of deleting a TDP for WAS backup from the TSM server.
Example 3-22 Deleting unwanted TDP WebSphere backups
C:\tdpws>tdpws -p "c:\tdpws\initwas.utl" -f delete Tivoli Data Protection for WebSphere Application Server - Version 1, Release 1, Level 0 (c) Copyright IBM Corporation, 2001, All Rights Reserved. DKP0005I: Start of program at: Apr 25, 2002 1:27:23 PM . DKP8017I: Please enter the backup ID's to delete and press ENTER twice: WAS___9C04172021 Do you really want to delete BackupsID's: WAS___9C04172021 (y/n) ?y DKP8010I: DKP8012I: DKP0020I: DKP0024I:
Preparing to delete all files for backup ID WAS___9C04172021. Host "localhost": BID 20020417201857 succesfully deleted. End of program at: Apr 25, 2002 1:29:18 PM . Return code is: 0.
3.3 Troubleshooting
In this section we will describe basic recovery procedures for failed or aborted operations. We will also discuss the TDP for WAS trace facility and give you advice on what files to look at when you encounter problems with your operation.
3.3.1 What to do when things go wrong

Often, a user may report a problem simply as My backup didnt work! Since this is a very general problem description, you need to try to narrow down the source of the problem. Is there a problem with the DB2 configuration? Does the TSM API work correctly? Was there a problem with the TSM server? Keep in mind that TDP for WAS depends on many components, and if any of these components fail, TDP for WAS wont operate properly either.
42
Here we define a list of components which can prevent TDP for WAS from operating properly: DB2 configuration for database backup to the TSM server Other DB2 database related problems (userexit, pending backup jobs, manual recovery needed after aborted jobs) Password handling problems between DB2, the TSM API and the TSM server TSM server related problems (communication timeout for client sessions, defective media, server out of storage space). Operating system problems (filesystem out of free space, not only for DB2 transaction logging, but also space for diagnostic logs, traces) TDP for WAS related problems (Prole service not running, not correctly installed, does not communicate correctly with other components). WebSphere related problems (WebSphere not running, version incompatibility). In our experience of setting up TDP for WAS in our lab environment, most of the initial problems were related to correctly setting up the environment variables necessary for DB2 to be able to back up its data to a TSM server. Carefully follow the configuration procedure described in 3.1.6, Post-installation steps on page 31 to ensure the variables are correctly set. Another source of problems was TSM server related especially communication and idle timeout settings for a TSM client session. Refer back to Modify server options file on page 23 for information on this.
Basic troubleshooting procedure

We can differentiate three basic component groups: DB2 TSM TDP for WAS In the following sections we focus on each group and give advice on logs you may have to look at, and explain how to obtain additional traces, if needed.
DB2 related problems

Before going any further with DB2 diagnostics, make sure you have updated the DB2 configuration for maintaining redo logs (logretain set to on), using the db2 userexit, and ensure that parallel recovery is disabled. Also make sure that you have restarted the DB2 instance after these changes.
43
DB2 related problems are usually indicated by the following symptoms:
Case 1: The db2 backup db command hangs nothing happens, with no session opened to TSM server. Case 2: The db2 backup db command fails with an error message similar to the following:
SQL2062N an error occurred while accessing media ?:\SQLLIB\bin\db2tadsm.dll Reason Code 2041
In the first case, the problem is most likely caused by a failed or incorrectly aborted database backup. The only solution of this problem is to issue restart db <dbname> command. Note that db2stop/db2start will not help in this case. The second case can have three different causes: TSM password handling: Check that your dsm.opt or dsm.sys file does not use the PASSWORDACCESS GENERATE option. If so, the DB2 backup command will fail. Check that you have set the right password when initializing the connection through the dsmapipw and tdppasswd commands. Incorrect password problems are easily detected in the dsmierror.log file Environment variable problem: This problem is the most common one in the Windows environment. The TDP installation program by default sets all necessary variables as USER variables in Windows NT or Windows 2000. In order for the DB2 backup to work properly, all DSM and DSMI variables (especially DSMI_COFNIG which points to the dsm.opt file) must be set as SYSTEM variables. If they are not, the DB2 backup db command returns reason code 406, which indicates that the dsm.opt file was not found. Invalid option problem with dsm.opt and/or dsm.sys: There is an invalid option in one of those files, or dsm.opt/sys was not found. Check the path to those files. Also check if your DSM and DSMI environment variables point to the proper location.
Additional DB2 logs

There are additional log files on the DB2 server, db2recovery.log and db2diag.log. These are placed in the directory pointed to in the DB2_DIAGPATH. These may provide more diagnostic information.
44
TSM related problems

On the TSM server side, check that your server is operating properly, including: Library and drives definitions Device class definitions Scratch media available in library Correctly defined and activated backup policy Client node registered to the proper domain One of the most common problems appearing generally when backing up large databases is client session timeout. There are two timeout parameters which you can set up in the dsmserv.opt configuration file: COMMTIMEOUT (default value 60 seconds): If the communication delay between TSM client and server is longer than 60 seconds, the session is cancelled by the TSM server. This 60 second timeout is usually not long enough when processing large databases. Increase the value up to 600 seconds. IDLETIMEOUT (default value 15 minutes): For heavy active databases, a client session may need to wait for longer than 15 minutes in order to be able to access a particular database table. Increase the value up to 45 minutes. It is necessary to restart the TSM server after modifying the dsmserv.opt file.
TDP for WAS related problems

We didnt encounter any serious problems on the TDP for WAS side during our tests. However, we will briefly discuss the TDP for WAS trace facility, which can be helpful in the overall problem determination process. Before doing any further problem determination on the TDP for WAS side, check that the WebSphere Administration server is up and running and that it is able to restart after a shutdown. If not, there is most likely a problem in the DB2 database refer to DB2 related problems.
TDP for WebSphere trace facility

There are three different components to trace within TDP for WAS: 1. TDPWS utility trace: You can enable this trace by supplying the undocumented -t option at the end of the tdpws command. Example 3-23 shows how to trace the behavior of the tdpws utility.
45
2. WebSphere file-data related trace: To enable this tracing, uncomment the TRACE and TRACEFILE options in the initWAS.utl file. Location of the generated trace file corresponds with the TRACEFILE options. 3. Log File Manager trace (DB2 userexit related trace): You can activate tracing by adding a LOG_TRACE and LOG_USEREXIT_LOGPATH to the db2uext2.utl file. Trace files will be stored in the directory specified with the LOG_USEREXIT_LOGPATH under the name db2uext2.trace.T<timestamp>.
Example 3-23 TDPWS trace option example
tdpws -p "c:\tdpws\initwas.utl" -f backup -t
Example 3-24 shows an extract from the db2uext2.utl file with the options necessary to enable Log File Manager tracing.
Example 3-24 Enable log file manager tracing in the db2uext2.utl file
LOG_TRACE detail LOG_USEREXIT_LOGPATH c:\tdpws\traces\
Please note that if there are insufficient privileges for writing into the location specified by LOG_USEREXIT_LOGPATH and/or TRACEFILE variables, a fall-back path will be used. The default fall-back path is depending on the operating system platform /tmp or TMP directory.
3.3.2 Recovering from failed TDP for WAS backup

If any failure occurs during the TDP for WAS backup operation, you may need to go through the following procedure in order to re-start it. The most common error message found when you try to restart your backup after a previous backup operation has failed is shown in Example 3-25.
Example 3-25 Error message for TDP for WAS failed backup
C:\tdpws>tdpws -p "c:\tdpws\initwas.utl" -f backup Tivoli Data Protection for WebSphere Application Server - Version 1, Release 1, Level 0 (c) Copyright IBM Corporation, 2001, All Rights Reserved. DKP0005I: Start of program at: Apr 18, 2002 11:03:10 AM . DKP8000I: Host "localhost": start backup of database "was".
46
DKP8001E: Error: SQL1015N The database must be restarted because the previous session did not conclude normally. SQLSTATE=55025 DKP8001E: Error: database backup failed DKP0020I: End of program at: Apr 18, 2002 11:03:22 AM . DKP0024I: Return code is: 2.
To recover from this error, run the restart database <database name> command form the DB2 console. Under some circumstances you still may not be able to invoke a backup operation with TDP for WAS. In this case, use the following recovery procedure: 1. Stop WebSphere Administrative server. 2. Stop TDP for WAS Prole process (service). 3. Cancel TDP for WAS client sessions on your TSM server. 4. Start DB2 database. 5. Start DB2 offline database backup in order to recover from DB2 backup pending condition. 6. Start WebSphere Admin server. 7. Start Prole. 8. Re-run the tdpws backup command. In some cases you wont be able to stop the Prole service (especially on Windows platform). You also may not be able to terminate the Prole service from Windows Task Manager. If you encounter this situation, the only workaround is to reboot your machine. Figure 3-10 shows an unsuccessful attempt to terminate the Prole service from the Windows 2000 Task Manager.
Figure 3-10 Unable to stop and/or terminate TDP for WAS Prole service
47
Important: Please make sure you have the latest version of the Tivoli Data Protection for WebSphere installed (Version 1.1.1 at the time of writing this Redpaper). This version fixes the problem of needing to restart the database after a failed backup.
However, in some situations, you may still need to go through the recovery procedure described above.
48
Chapter 4.
Backing up WebSphere V4.0

In this chapter we focus on how to protect a WebSphere Application Server V4.0 environment using Tivoli Storage Manager. As already mentioned, using Tivoli Data Protection for WebSphere Application Server (TDP for WAS) is only appropriate for a WebSphere Application Server V3.5 environment. Therefore, here we will document a method for backing up WebSphere Application Server V4.0.
49
4.1 WebSphere 4.0 considerations

The WebSphere 4.0 architecture has changed from V3.5, hence at the time of writing this Redpaper, TDP for WAS does not support the WebSphere 4.0 environment. For more detailed information on what is new in WebSphere 4.0 and its architecture, please refer to the IBM Redbook IBM WebSphere V4.0 Advanced Edition Handbook, SG24-6176.
4.1.1 WebSphere Application Server 4.0 backup/restore strategy

Our goal in this chapter is to create a generic procedure which will back up all necessary WebSphere data files and the administrative database on a particular node. In order to back up all your WebSphere nodes in an administrative domain, you will need to run this procedure on all WebSphere nodes belonging to that domain.
Backup strategy
When backing up WebSphere, you need to make sure that your administrative database backup is consistent with the backed-up data files. Consistency in this case means that we need to keep associated data file and database backups in synchronization. In order to achieve data consistency, we will use the following method for a manual WebSphere backup: Create a backup directory in the local filesystem for storing backup snapshot (referred to as the WAS snapshot directory). Stop WebSphere Administrative server. Backup data files to snapshot directory. Backup WebSphere administrative database, offline, to a local file in the WAS snapshot directory. Archive all files in the WAS snapshot directory. Please note that we focus only on the most reliable method of backing up the WebSphere node configuration and administrative database that means our goal is similar to what TDP for WAS offers. We do not focus on how to back up your application databases and data files. However we will just briefly point out what additional steps are necessary for application data backup.
50
Our main backup/restore concept is taken from a procedure described in the Redbook, IBM WebSphere 4.0 Advanced Edition Handbook, SG24-6176. We have modified it to use the TSM Backup/Archive client to archive the data collected the WAS snapshot directory. We recommend using the archiving function of TSM in order to achieve a kind of versioning function when backing up WebSphere manually, and because it enables keeping all the data in one package. This eases the restore process and guarantees consistency among file data and administrative database backup. In order to place all data for a particular backup into one archive directory, it is not possible to back up the WebSphere Administrative database online/offline directly to TSM using the DB2 backup database with use tsm option. This would create a separate data object in TSM and therefore the DB2 backup would not be a part of the archive package. You would then have to manually keep track of which DB2 backup belongs to which corresponding file data archive package. Therefore we will back up the database to a temporary file in the WAS snapshot directory and include it in the TSM archive package.
Restore strategy
The restore strategy consists of three main steps: Retrieve data files from the archive to temporary location on your WebSphere nodes. Copy data files from temporary location to live WebSphere directory structure. Restore and recover DB2 database.
4.2 Planning for backup and recovery

In this section we introduce our testing scenario and describe the detailed backup and restore procedures for WebSphere 4.0.
4.2.1 Introducing our testing environment

Our environment consists of two WebSphere V4.0 nodes, RADON and LEAD, installed on Windows 2000. RADON is the master node which has the DB2 administrative database locally installed. The node LEAD has an application server (also called LEAD), with a clone defined on node RADON. Two Enterprise Applications, including the sample Web application are running on the nodes. LEAD accesses the WAS repository (administrative database) remotely on RADON. The configuration is shown in Figure 4-1.
Chapter 4. Backing up WebSphere V4.0
51
LAN
RADON W2K
BRAZIL AIX 4.3.3
LEAD W2K
WebSphere 4.0 "master" node DB2 7.2 TSM BA Client 4.2.1.
TSM Server
SCSI Tape Library
M IB
WebSphere 4.0 node TSM BA Client 4.2.1. DB2 client, acces WAS repository on server RADON remotely
Figure 4-1 ITSO WebSphere lab environment
The repository (administrative) database name is WAS. In order for LEAD to access this database, a remote TCP/IP alias has been defined and cataloged. This alias name is REMOTEWAS. For information on how to set up remote access between DB2 clients and server, refer to Part 3 of the IBM Redbook, IBM WebSphere 4.0 Advanced Edition Handbook, SG24-6176. As shown in Figure 4-2, we have a server group defined in our WebSphere environment consisting of node LEAD and its clone on RADON. On both LEAD and its clone we have our sample Enterprise Application installed and running.
52
Figure 4-2 WebSphere 4.0 configuration in our test lab
As a backup server we use Tivoli Storage Manager Server version 4.2 for AIX with latest available fixes and a tape library connected to it. We have used the same TSM server configuration as weve been using during our testing of TDP for WAS. Refer to 3.1.2, Preparing the TSM server for TDP for WAS on page 22 for TSM server configuration. On both WebSphere nodes we have the TSM Backup/Archive client installed with default configuration for file backup. Note that the Backup/Archive client is the only TSM software required for this configuration the API client is not needed, as we will be backing up DB2 as a regular file. In Example 4-1 we show the output of our dsm.opt configuration file it can be the same on each WebSphere node for reasons that is, the node name will be the same. We will explain this in Backing up the WAS Distributed Multinode environment on page 55.
Example 4-1 TSM client configuration dsm.opt example
COMMM TCPIP TCPS brazil TCPP 1500 NODEname was PASSWORDACCESS generate
53
4.2.2 Defining which objects we need to back up

First we identify the file data which we need to back up for a complete recovery: The admin.config file from <WASHOME>/bin directory All properties files from <WASHOME>/properties directory All files from <WASHOME>/etc directory In order to back up your application data, you also need to back up these directories: <WASHOME>/installedApps <WASHOME>/installableApps <WASHOME>/installedConnectors We assume that all applications are installed in the default directories as shown above. If any application file (including data files, configuration files) is installed in different directories, you should add these also for backup. The script developed in this chapter can be easily adapted for this. Finally, we need to back up the WebSphere administrative database. You may also want to back up your application databases using appropriate database backup utilities.
4.2.3 Backup procedure

Backup procedures are slightly different depending on the WebSphere infrastructure. For the purpose of this Redpaper, we will differentiate between the WAS Standalone Node and the WAS Distributed Multinode environment.
WAS Standalone Node Both the WebSphere Application Server and DB2 (or any other) database engine are installed and running on a single machine. WAS Distributed Multinode environment There are at least two nodes belonging to one WebSphere domain using the same database repository (DB2 database). The database engine runs either on one of the WebSphere nodes or standalone on another non-WebSphere system.
Backing up WAS standalone nodes

The procedure shown here uses Windows syntax we provide a sample script for both Windows and UNIX in Appendix B, WebSphere Application Server V4.0 backup scripts on page 73. In order to back up a WAS Standalone Node, do the following: 1. Create these directories (Example 4-2) for storing WebSphere backup data (WAS snapshot directory).
54
Example 4-2 Create WAS snapshot directory

mkdir mkdir mkdir mkdir mkdir mkdir mkdir mkdir C:\WASbackup C:\WASbackup\bin C:\WASbackup\properties C:\WASbackup\etc C:\WASbackup\db C:\WASbackup\installedApps C:\WASbackup\installableApps C:\WASbackup\installedConnectors
2. Stop WebSphere Administrative Server (from either the Windows Services panel or the WebSphere Administrative Console). 3. Copy WebSphere data files to the WAS snapshot directory (Example 4-3).
Example 4-3 Save WebSphere Application Server data files in WAS snapshot directory
copy C:\WebSphere\AppServer\bin\admin.config C:\WASbackup\bin copy C:\WebSphere\AppServer\properties\*.prop* C:\WASbackup\properties xcopy C:\WebSphere\AppServer\etc C:\WASbackup\etc xcopy /e C:\WebSphere\AppServer\installedApps C:\WASbackup\installedApps xcopy /e C:\WebSphere\AppServer\installableApps C:\WASbackup\installableApps xcopy /e C:\WebSphere\AppServer\installedConnectors C:\WASbackup\installedConnectors
Open the DB2 command line processor and back up the DB2 administrative database: backup database <WAS db name> to C:\WASbackup\db Re-start WAS Run TSM Backup/Archive client in order to archive backup all the saved WebSphere data to TSM: dsmc archive -subdir=yes C:\WASbackup\ Repeat this procedure (or run the script) on all your standalone WebSphere nodes.
Backing up the WAS Distributed Multinode environment

As mentioned earlier in this section, a distributed multinode environment consists of two or more nodes belonging to the same WebSphere domain.
Tip: More information on WebSphere domains and multiple WebSphere node configurations is provided in the IBM Redbook, IBM WebSphere 4.0 Advanced Edition Handbook, SG24-6176.
55
You can see our sample lab environment in Figure 4-1 on page 52. Because all nodes of the particular domain are accessing the same database repository (WebSphere administrative database) you need to run database backup only once from that node which has that database locally installed. We will refer to this node as the master node. All other nodes will be called secondary nodes. The backup procedure consists of the following stages: Back up the WAS master node. Back up all other WAS nodes belonging to the same domain as the master node. To back up the WebSphere master node, follow the steps in Backing up WAS standalone nodes on page 54 or run the sample script provided in Appendix B, WebSphere Application Server V4.0 backup scripts on page 73. Backup of WAS secondary nodes is similar to backing up the master node the only difference is that you skip the DB2 database backup, since that backup has already been done. You can also use our sample script provided in Appendix B, WebSphere Application Server V4.0 backup scripts on page 73, but you need to remove the line which calls the DB2 database backup.
Archive package directory structure

For WebSphere Application Server on Windows, your backup script can still use the same directory structure for the WAS snapshot directories on all WebSphere nodes. In Windows, TSM uses network (UNC) names for identifying TSM archive objects, so even if the directory structure is the same on all nodes, TSM will store each nodes data in its own subtree as shown in Figure 4-3 on page 62. We can see that two separate nodes, RADON and LEAD, have each archived files in their respective WASbackup directories to the same named archive package. If you are backing up WAS on a UNIX system, we need to identify each nodes files separately. We do this by adding the system hostname to the WAS snapshot directory so that each node will have a separate directory structure. This is shown in the script in Appendix B-2, WAS4.0 sample backup script for AIX environment on page 75, via the parameter HOST. We have used the format hostname/WASbackup to store each nodes files. You can use this as a virtualmountpoint (defined in the dsm.sys file) for easier archiving and viewing.
56
Sharing TSM node name

We also suggest that you use the same TSM client nodename on all WebSphere nodes in the domain. If you do this, all the files for all your WebSphere nodes will be stored in the same archive package. This ensures that the backups are synchronized and it makes restore easier, as you dont need to search what file data to restore. You only need to restore either the complete archive package or only those files from the package which belonging to the particular WebSphere node. Note that if the same TSM node name is used on all WAS nodes in the domain, you cannot use TSM server prompted scheduling to automate the backups, because only one physical TCP/IP address can be associated with a node name in this case. Client polling scheduling however can be used. Another thing to be aware of when using the same TSM client node name on multiple physical WAS nodes is password expiry. If you use automatic password expiry set on the TSM server, by default only one node would have its password reset when it expires. To avoid this, you can either disable password expiry, or set up a command script to change each WAS nodes TSM client password before the expiry date. This will ensure all the WAS nodes TSM client passwords are synchronized. When we use the same TSM node name among all WAS nodes, we want to be sure that all objects within the package are bound to the same management class. This will ensure that all objects expire on the same date since they are all associated with the one logical backup. To do this, either always use the same default management class, or specify the same explicit management class on every backup to the package using the -ARCHMC option on the DSMC ARCHIVE command.
Archive package description

By default, archive packages use a package name in the format Archive Date: <date the archive has been taken>, for example, Archive Date:05/07/2002. If you have used the same nodename in the dsm.opt file for each WAS V4.0 node (as recommended), and have used appropriate scheduling to ensure that the backups run separately, each new WAS node backup will add to the same original archive package. This will work provided that you run only one complete backup per day and that all backups are run within the same day (from midnight to midnight).
57
However, it is more elegant to specify an explicit archive description, using the option -description=<description of the archive package> with the DSMC ARCHIVE command. This will allow you to easily identify the different packages. Our script uses a simple timestamp description, however you can use adjust to include other information such as nodes, type of backup (daily or weekly), and so on, as required.
Backup of standalone DB2

Up to now, our backup strategy assumes that the database engine runs on the same machine as one of the WebSphere Application servers. If you are running the database server on a standalone machine, accessing the WebSphere database repository remotely, you can still use our procedure as described in 4.2.3, Backup procedure on page 54. The only difference is that you will run a separate operation on the remote standalone DB2 system to back up the database to a file. You can simply modify the script to include only the DB2 backup and archive operation. However, if the DB2 server is running on an OS which is different from the WAS systems, you cannot add this file to the archive package as the directory structure is different. You can still run a script to back up the DB2 database to a file. You will then need to use a suitable method to transfer this file to one of the WebSphere nodes backup directory in order to have this file included in the archive package. You might use FTP, or a file system mount to do this be careful of any file naming conventions.
Backup versioning
First, please note that we use the term backup here in generic terms, although from the TSM server point of view we in fact do archives. We explained the choice of archiving rather than backup in 4.1.1, WebSphere Application Server 4.0 backup/restore strategy on page 50. Versioning is determined by the retention period defined in the Archive Copy Group which the TSM node uses for archiving. Please note that you need to change the retention period according to the interval in which you plan to run your WebSphere backups. For example, suppose you want to keep at least 2 versions of WebSphere snapshots, and you plan to run your backup once a week Your retention period for the archive copygroup should be set to at least 15 (two weeks + one day reserve). If you need additional information on how TSM backup policies work, please refer to the product documentation at the Tivoli Web site:
http://www.tivoli.com/support/public/Prodman/public_manuals/td/TD_PROD_LIST.html
58
4.2.4 Restore procedure

We assume that you have a fresh installation of the WebSphere Application Server 4.0 environment. Before you begin the restore as described below, make sure that your environment complies with the following requirements: The node name of the host system must be the same as it was during backup. The version number of WebSphere including fixpack levels must be exactly the same. The <WAS_HOME> directory must be the same. Database containers must be created with the DB2 create database command, and all names for the repository (administrative databases) must be the same as they were during backup. Next we describe a procedure which restores the WebSphere master node first and then you can use this procedure to restore your secondary nodes. The only difference is that for a secondary node, the database restore step is not required. 1. Stop WebSphere Administrative Server (either from the Windows Services panel or the WebSphere Administrative Console) all WebSphere nodes. 2. Consider your WAS snapshot directory: a. If your WAS snapshot directory still exists in the local filesystem (and contains the latest version of the backup) on all of your nodes, continue with step 3. Check this on all WebSphere nodes. Otherwise you need to retrieve the archived WAS snapshot directories on all nodes as described in the substep below: b. If the WAS snapshot directory no longer exists on the WebSphere nodes local filesystem, or does not contain the most recent backup data set, retrieve that directory and its contents using the TSM Backup/Archive client. Start the TSM client on that node and click on Retrieve. Select the desired archive package and select the node you want to retrieve files for. An example of how to select an individual WAS snapshot directory for retrieval for a node is shown in Figure 4-3 on page 62. We have selected just the files for the node LEAD. You will repeat this process on each node in the domain, selecting only that nodes directory and files in turn. 3. Copy back all the configuration, properties, key files, installed and installable application code and connectors all WebSphere nodes (Example 4-4).
59
Example 4-4 Copy WebSphere Application Server files from temporary to real location
copy C:\WASbackup\bin\* C:\WebSphere\AppServer\bin copy C:\WASbackup\properties\*.prop* C:\WebSphere\AppServer\properties\ xcopy C:\WASbackup\etc C:\WebSphere\AppServer\etc xcopy /e C:\WASbackup\installedApps C:\WebSphere\AppServer\installedApps xcopy /e C:\WASbackup\installableApps C:\WebSphere\AppServer\installableApps xcopy /e C:\WASbackup\installedConnectors C:\WebSphere\AppServer\installedConnectors
4. Now restore WAS database repository using DB2 restore database command WAS Master Node only: a. Make sure DB2 is running. WAS Administrative server must be stopped. b. Edit <WAS_HOME>\bin\admin.config file and set the following properties (Example 4-5):
Example 4-5 Properties for WAS administrative database restore
install.initial.config=false com.ibm.ejs.sm.adminServer.createTables=false
c. Log in to DB2 with SYSADMIN privileges and do the following: Check the administrative database container is created in DB2, start your DB2 command line interface and issue: list database directory You can see sample output from this command in Example 4-6. Please note that both remote and local entries for the WebSphere Application Server database must exist, as shown in the example. If the database you want to restore is not created, run the following commands: create database <database name> update db cfg for <database name> using applheapsz 256 Restore the database using: restore database <database name> user <admin user name> using <password> from C:\WASbackup\db 5. Restart WAS Admin database repository using the DB2 restart db command WAS Master Node only: db2 restart database <database name> 6. Restore your application databases if necessary. 7. Start all your WebSphere nodes.
60
Check that WebSphere works properly, for example, by starting the WebSphere Administrative client. At this point, you have successfully restored your WebSphere environment (Example 4-6).
Example 4-6 Sample output of the DB2 list database directory command
Database 1 entry: Database alias Database name Node name Database release level Comment Directory entry type Catalog node number Database 2 entry: Database alias Database name Database drive Database release level Comment Directory entry type Catalog node number = = = = = = = RADON RADON C:\DB2 9.00 Indirect 0 = = = = = = = RADONWAS RADON LEAD 9.00 Remote -1
You can use the TSM Backup/Archive client interface for retrieving archive packages from the TSM server. Figure 4-3 shows the TSM client view of an archived WebSphere package created with our sample script.
61
Figure 4-3 TSM backup archive client WAS archive package view
4.2.5 WebSphere Application Server V4.0 complete recovery

If a complete bare metal restore of WebSphere Application Server is necessary, there are two possibilities for doing this:
Method 1: Restore your system from a full backup including operating system, all binaries, configuration files, system registry entries (in case your environment runs on Windows). Information on doing this with Windows 2000 in contained in the IBM Redbook Deploying the Tivoli Storage Manager Client in a Windows 2000 Environment, SG24-6141. Method 2: Install WebSphere from distribution CDs, apply patches and fixpacks to the same level as it was during the last time of backup, then restore the configuration files, applications, databases using the methods described in this chapter.
62
We do not recommend the first method, mainly because of the issue of backup synchronization. Using a regular file-based backup, you can never be sure if your backup is really consistent. For example is your registry backup consistent with your current WebSphere file-data backup? Is it also consistent with the DB2 database repository backup? Are there any configuration files stored out of the <WAS_HOME> directory, and if so, are all these files backed up? Therefore, we recommend using the second method for WebSphere full recovery even though it may appear to involve more work. If you are really starting from scratch, you can make sure that your WebSphere environment is up and running properly and then restore the previously backed up configuration. This will prevent integrity problems between the various components. In any case, full recovery should be carefully tested and documented in each individual environment to make sure the procedure works and is understood.
4.2.6 Backing up and restoring a non-DB2 WAS environment

As described, WebSphere can use other database products for the repository, apart from DB2. This section will briefly cover backup of non-DB2 WebSphere environments. When considering the suggested backup and restore procedures described in this chapter, it is obvious that they could be easily adapted to other database environments. You only need to change the method to invoke the database backup, since the file data structure is the same in all cases. The main reason why this backup/restore procedure is so adaptable is that we in export the WebSphere Administrative database to a file, instead of taking online backups provided directly through the specific database API to Tivoli Storage Manager. This gives us the possibility to theoretically backup any WebSphere environment regardless of the database engine being used. You only need to substitute the appropriate database backup command for the DB2-specific command in the script.
63
64
Appendix A.
TDP for WAS config files

This appendix contains the full listings of these important configuration files, which are necessary for a successful Tivoli Data Protection for WebSphere Application Server (TDP for WAS) installation: initWAS.utl db2uext.utl vendor.env Samples of all these files are provided when TDP for WAS is installed, but you need to modify them according to your environment. If you need a more detailed description of the definitions in these files, please refer to the product manual, Tivoli Data Protection for WebSphere Application Server Installation & Users Guide, SC33-6399.
65
InitWAS.utl
#--------------------------------------------------------------------------# # Sample profile for Tivoli Data Protection for WebSphere Application Server # #--------------------------------------------------------------------------# # See the 'Tivoli Data Protection for WebSphere Application Server # Installation & User's Guide' for a full description. # # For comment symbol the character '#' can be used. # Everything following this character will be interpreted as comment. # # Tivoli Data Protection for WebSphere Application Server accesses its # profile in "read only" mode only. All variable parameters like passwords, # date of last password change, current version number will be written into # the file specified with the CONFIG_FILE parameter. # The passwords will be encrypted.
#-------------------------------------------------------------------------# Prefix of the 'Backup ID' which will be stored in the description field # of the Tivoli Storage Manager archive function. # Maximum 6 characters. # Default: none. #-------------------------------------------------------------------------BACKUPIDPREFIXWAS___
#-------------------------------------------------------------------------# Number of total parallel sessions which will be established by Tivoli Data # Protection for WebSphere Application Server. Note: this number should # correspond with the number of simultaneously available tape drives # specified for the Tivoli Storage Manager server. # Default: none. #-------------------------------------------------------------------------MAX_SESSIONS 2# 1 Tivoli Storage Manager client session is default
#-------------------------------------------------------------------------# Number of backup copies of the archived redo logs. # Default: 1. #-------------------------------------------------------------------------#REDOLOG_COPIES2 # 1 is default
#--------------------------------------------------------------------------
66
# Specifies how many files are multiplexed into one data stream # to a Tivoli Storage Manager server. Multiplexing is usefull when the data # rate to a Tivoli Storage Manager server is higher (fast tapes, fast network) # than the I/O rate from a single disk or when backing up a lot of small files. # The valid range of MULTIPLEXING is from 1 to 8. # Default: 1 (meaning no multiplexing) #-------------------------------------------------------------------------#MULTIPLEXING3 # 1 is default
#-------------------------------------------------------------------------# Specifies the block size for disk I/O (in bytes). The valid range is # from 4 KB to 256 KB. # A large blocksize may increase the performance when transfering large # files. For a lot of small files we strongly recommend 4 KB. #-------------------------------------------------------------------------BUFFSIZE4096 # block size in bytes
#-------------------------------------------------------------------------# Name of a program that is called by Tivoli Data Protection for WebSphere # Application Server before the backup task is started. # Default: none. #-------------------------------------------------------------------------#FRONTEND pgmname parameterlist
#-------------------------------------------------------------------------# Name of a program that is called by Tivoli Data Protection for WebSphere # Application Server after the backup task is completed. # Default: none. #-------------------------------------------------------------------------#BACKEND pgmname parameterlist
#-------------------------------------------------------------------------# Maximum number of backup versions to be kept. # Default: 0. (means do not delete any version) #-------------------------------------------------------------------------#MAX_VERSIONS4
#-------------------------------------------------------------------------# Batch processing # Default: # The default for the BATCH parameter is YES for the backup run and NO for # the restore run if the BATCH parameter is COMMENTED OUT in this profile. # It's recommended to set BATCH to YES on all the controlled nodes. #--------------------------------------------------------------------------
Appendix A. TDP for WAS config files
67
BATCH #BATCH
YES NO
# unattended automated operation # manual operation
#-------------------------------------------------------------------------# Controls generation of a trace file. # Note: we recommend using the trace function only in cooperation with # the hotline. # Default: NO. #-------------------------------------------------------------------------#TRACE 100
#-------------------------------------------------------------------------# Specify the trace file for Tivoli Data Protection for WebSphere Application # Server to store all trace information (if TRACE ON), full path and name of file. # Note: for an actual trace the string '%BID' will be replaced by # the current backupid. # (.../tdpwas_%BID.trace changes to .../tdpwas_WAS___9B09182300.trace). # Default: stdout. #-------------------------------------------------------------------------#TRACEFILE/tmp/tdpwas.trace #TRACEFILE C:\tdpws\tdpws_%BID.trace
#-------------------------------------------------------------------------# Specify the configuration file for Tivoli Data Protection for WebSphere # Application Server to store all variable parameters, full path and # name of the file. # Default: none. #-------------------------------------------------------------------------CONFIG_FILE "C:\tdpws\initWAS.bki"
#-------------------------------------------------------------------------# Number of times Tivoli Data Protection for WebSphere Application Server # retries to save/restore a file in case an error occurs. # Default: 3. #-------------------------------------------------------------------------#FILE_RETRIES3
#-------------------------------------------------------------------------# Shall Tivoli Data Protection for WebSphere Application Server send # error/status information to a Tivoli Storage Manager server. The statement # for servername must match one of the servers listed in a SERVER statement. # Statements for verbosity can be ERROR, WARNING, or DETAIL. # Default: none.
68
#-------------------------------------------------------------------------#LOG_SERVER servername [verbosity] LOG_SERVER server_a DETAIL #-------------------------------------------------------------------------# Shall Tivoli Data Protection for WebSphere Application Server send # error/status information to a network management program via SNMP traps? # Default: none. #-------------------------------------------------------------------------#SNMPTRAP Hostname community level #SNMPTRAPserver_a publicdetail
#************************************************************************** # Server Statement #************************************************************************** SERVER server_a SESSIONS 2 PASSWORDREQUIRED YES ADSMNODE WAS BACKUPMGTCLASS MDB ARCHIVEMGTCLASS MLOG # Servername # Max sessions # Use a password # Tivoli Storage Manager Nodename # Mgmt-Classes # Mgmt-Classes
#-------------------------------------------------------------------------# End of profile
69
db2uext.utl
#--------------------------------------------------------------------------# # Sample profile for Tivoli Data Protection for WebSphere Application Server # -- Log File Manager -# #--------------------------------------------------------------------------# # See the 'Tivoli Data Protection for WebSphere Application Server # Installation & User's Guide' for a full description. # # As a comment symbol, the character '#' can be used. # Everything following this character will be interpreted as a comment. # #--------------------------------------------------------------------------
#-------------------------------------------------------------------------# # Section for DB2 Database WAS1, Node NODE0000 # Do not change the order of LOG_DB_NAME and LOG_DB_NODE. # LOG_DB_NAME and LOG_DB_NODE must precede all other settings for this # database. # #-------------------------------------------------------------------------#-------------------------------------------------------------------------# Database name #-------------------------------------------------------------------------LOG_DB_NAME WAS #-------------------------------------------------------------------------# Database node number #-------------------------------------------------------------------------LOG_DB_NODE NODE0000 #-------------------------------------------------------------------------# Path for the Log File Manager's own logs and control files # This parameter is required. # The path must not contain blanks. #-------------------------------------------------------------------------LOG_USEREXIT_LOGPATH"c:\db2\sqllib\db2dump" #-------------------------------------------------------------------------# Path where DB2 redo log files are retrieved # If not set, DB2 redo logs are retrieved to the db2 log path specified # in the call.
70
# The path must not contain blanks. #-------------------------------------------------------------------------#LOG_RETRIEVE_PATHc:\DB2\NODE0000\WAS\SQLLOGDIR\
#-------------------------------------------------------------------------# End of profile
71
vendor.env
XINT_PROFILE=c:\tdpws\initWAS.utl DB2_DIAGPATH=c:\db2\sqllib\db2dump DB2_UEXT2_PROFILE=c:\db2\sqllib\db2uext2.utl DSMI_DIR=c:\progra~1\tivoli\tsm\api DSMI_CONFIG=c:\tdpws\dsm.opt
72
Appendix B.
WebSphere Application Server V4.0 backup scripts

In this Appendix we introduce the sample scripts which we used in our test environment to back up WebSphere Application Server V4.0. These scripts provide only the basic functionality to show you how to automate a backup procedure. You will probably need to modify this sample script according to your environment you might want to add versioning functionality to keep multiple backup versions in a different subdirectory tree and so on. You might choose to use the -d option on the TSM Backup/Archive client archive command in order to delete the temporary WAS backup snapshot directory after the archive command is complete. This saves space on the WebSphere Application Servers. On the other hand, leaving this directory in the local filesystem gives you the ability to restore single backup objects directly from this directory without having to retrieve them from TSM. Both Windows and UNIX versions are given. The UNIX version uses the Bourne shell it could easily be modified for the C shell if required. Note that in the UNIX version, we have used the WAS nodename to identify the different WAS snapshot directories. This will avoid failover issues if you are running WebSphere Application Server in a clustered environment. These scripts are available for download for more information on how to do this, see Appendix C, Additional material on page 77.
73
B.1 Windows environment

Example B-1 shows the Windows version of the script we have used to back up our WebSphere Version 4.0 environment.
Example: B-1 WAS4.0 sample backup script for Windows environment
set WAS_HOME=c:\WebSphere\AppServer set WAS_SNAPDIR=c:\WASbackup set DSMC_LOCATION=c:\progra~1\tivoli\tsm\baclient for /f "tokens=1,2,3,4 delims=/ " %i in ('date /T') do (set DD=%j)&(set MM=% k)&(set YY=%l) set TIMESTAMP=%DD%%MM%%YY% set WAS_DB=WAS echo off @echo "removing old backup tree.." rmdir /S /Q %WAS_SNAPDIR% @echo "creating backup directories.." mkdir %WAS_SNAPDIR% mkdir %WAS_SNAPDIR%\bin mkdir %WAS_SNAPDIR%\properties mkdir %WAS_SNAPDIR%\etc mkdir %WAS_SNAPDIR%\db mkdir %WAS_SNAPDIR%\installedApps mkdir %WAS_SNAPDIR%\installableApps mkdir %WAS_SNAPDIR%\installedConnectors @echo "stopping WAS node.." net stop "IBM WS AdminServer 4.0" @echo "copying WAS data files.." copy %WAS_HOME%\bin\admin.config %WAS_SNAPDIR%\bin copy %WAS_HOME%\properties\*.prop* %WAS_SNAPDIR%\properties xcopy %WAS_HOME%\etc %WAS_SNAPDIR%\etc xcopy /e %WAS_HOME%\installedApps %WAS_SNAPDIR%\installedApps xcopy /e %WAS_HOME%\installableApps %WAS_SNAPDIR%\installableApps xcopy /e %WAS_HOME%\installedConnectors %WAS_SNAPDIR%\installedConnectors @echo "backup db2 database.." "C:\Program Files\SQLLIB\bin\db2cmd.exe" db2 backup database %WAS_DB% user db2inst1 using admin to %WAS_SNAPDIR%\db @echo "starting WAS node.." net start "IBM WS AdminServer 4.0" @echo "starting TSM archive session.." cd c:\progra~1\tivoli\tsm\baclient dsmc archive -subdir=yes %WAS_SNAPDIR%\ -description=%TIMESTAMP%
74
B.2 AIX environment

Example B-2 shows the AIX version of the script we have used to back up our WebSphere V4.0 environment.
Example: B-2 WAS4.0 sample backup script for AIX environment
# We have hardcopied the value of WAS_NODENAME to our hostname. This represents # the WAS nodename. This is not done using a parameter because it is # possible to run multiple WAS nodes on a single system - in this case they # would have different names # #The variable WAS_DB represents the WAS administrative database name export WAS_HOME=/usr/WebSphere/AppServer export DSMC_LOCATION=/usr/tivoli/tsm/client/ba/bin export WAS_NODENAME=wewak export WAS_SNAPDIR=/$WAS_NODENAME/WASbackup export TIMESTAMP=`date +"%a %d %h %y"` export WAS_DB=WAS echo "removing old snapshot tree.." rm -rf $WAS_SNAPDIR echo "creating snapshot directory structure.." mkdir $WAS_SNAPDIR mkdir $WAS_SNAPDIR/bin mkdir $WAS_SNAPDIR/properties mkdir $WAS_SNAPDIR/etc mkdir $WAS_SNAPDIR/db mkdir $WAS_SNAPDIR/installedApps mkdir $WAS_SNAPDIR/installableApps mkdir $WAS_SNAPDIR/installedConnectors chmod a+w $WAS_SNAPDIR/db echo "stopping WAS node" $WAS_HOME/bin/wscp.sh -c "Node stop /Node:$WAS_NODENAME/" echo "copying WAS data files to snapshot dirs.." cp $WAS_HOME/bin/admin.config $WAS_SNAPDIR/bin cp $WAS_HOME/properties/*.prop* $WAS_SNAPDIR/properties cp -R $WAS_HOME/etc $WAS_SNAPDIR/ cp -R $WAS_HOME/installedApps $WAS_SNAPDIR/ cp -R $WAS_HOME/installableApps $WAS_SNAPDIR/ cp -R $WAS_HOME/installedConnectors $WAS_SNAPDIR/ echo "backup db2 database.." su - db2inst1 "-c db2 backup database $WAS_DB user db2inst1 using admin to $WAS_SNAPDIR/db" echo "starting WAS node" $WAS_HOME/bin/wscp.sh -c "Node start /Node:$WAS_NODENAME/" echo "starting TSM archive session.." cd $DSMC_LOCATION dsmc archive -subdir=yes $WAS_SNAPDIR/ description=$TIMESTAMP
Appendix B. WebSphere Application Server V4.0 backup scripts
75
76
Appendix C.
Additional material
This Redpaper refers to additional material that can be downloaded from the Internet as described below.
Locating the Web material

The Web material associated with this Redpaper is available in softcopy on the Internet from the IBM Redbooks Web server. Point your Web browser to:
ftp://www.redbooks.ibm.com/redbooks/REDP0149
Alternatively, you can go to the IBM Redbooks Web site at:

ibm.com/redbooks
Select the Additional materials and open the directory that corresponds with the redbook form number, REDP0149.
Using the Web material

The additional Web material that accompanies this Redpaper includes the following files:
File name
Description
77
was_archive.zip
Zipped Command Scripts for UNIX and Windows. See the README.TXT file for details
System requirements for downloading the Web material

The following system configuration is recommended:
Hard disk space: Operating System: Processor : Memory:
1MB Windows/UNIX Any As required by the Operating System
How to use the Web material

Create a subdirectory (folder) on your workstation, and unzip the contents of the Web material zip file into this folder. Refer to the file README.TXT and the comments in the scripts themselves for more information. The scripts are listed in full in Appendix B, WebSphere Application Server V4.0 backup scripts on page 73.
78
Abbreviations and acronyms

API CORBA EJB GUI HSM HTML HTTP IBM IDL ITSO JDBC JIDL JMS JNDI JRMI/IIOP Application Programming Interface Common Object Request Broker Architecture Enterprise Java Bean Graphical User Interface Hierarchical Space Management HyperText Markup Language HyperText Translation Protocol International Business Machines Corporation WAS XML WebSphere Application Server eXtensible Markup Language
Interface Definition Language

International Technical Support Organization Java DataBase Connectivity
Java Interface Definition Language Java Messaging Service Java Naming and Directory Interface Java Remote Method Invocation over Internet Inter-ORB Protocol
Java Server Page
JSP JTA JTS LDAP SAN SSL TDP TSM
Java Transaction API Java Transaction Service

Lightweight Directory Access Protocol Storage Area Network Secure Sockets Layer Tivoli Data Protection Tivoli Storage Manager
Copyright IBM Corp. 2002. All rights reserved.
79
80
Related publications
The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this Redpaper.
IBM Redbooks
For information on ordering these publications, see How to get IBM Redbooks on page 82.
WebSphere V3.5 Handbook, SG24-6161 IBM WebSphere V4.0 Advanced Edition Handbook, SG24-6176 Tivoli Storage Management Concepts, SG24-4877 Getting Started with Tivoli Storage Manager: Implementation Guide, SG24-5416 Backing up DB2 Using Tivoli Storage Manager, SG24-6247
Other resources
These publications are also relevant as further information sources:
Tivoli Data Protection for WebSphere Application Server Installation and Users Guide,SC33-6399
Referenced Web sites

These Web sites are also relevant as further information sources: WebSphere Overview
http://www.ibm.com/software/websphere/
TDP for WebSphere Application Server

http://www.tivoli.com/support/storage_mgr/tdp_websphere.html
Tivoli Storage Manager

http://www.tivoli.com/tsm
81
How to get IBM Redbooks

You can order hardcopy Redbooks, as well as view, download, or search for Redbooks at the following Web site:
ibm.com/redbooks
You can also download additional materials (code samples or diskette/CD-ROM images) from that site.
IBM Redbooks collections

Redbooks are also available on CD-ROMs. Click the CD-ROMs button on the Redbooks Web site for information about all the CD-ROMs offered, as well as updates and formats.
82
Index
A
administrative database 4 administrative domain 3 administrative server 3 administrative tools 2 application server process 3 archive package description 57 authorization 5 dynamic Web pages 4
E
e-business 2 EJB 23, 6, 14 programming model 6 Enterprise Java APIs 6 Enterprise Java Beans 2, 6 environment variables 31 eXtensible Markup Language 2, 6
B
BEGIN statement 6 business logic 2 business methods 5 business process 6
F
failover 6
G C
clients 5 client-side scripts 4 clone 6 cluster 6 COMMIT statement 6 Common Object Request Broker Architecture 6 connection manager 6 CORBA 6 graphical user interface 4 GUI 4
H
HTML 4, 6 HTTP 6 HTTP server 4, 22 HyperText Markup Language 6 HyperText Transfer Protocol 6
D
DB2 4, 47 create db 19 DB2UEXT 16 logs 44 remote database backup 58 Shared Vendor Library 14 tracing 46 User Exit 14 DB2UEXT 16 db2uext2.utl 33, 46 distributed database 6 distributed system management 5 domain 3 dsm.opt 33, 35, 53 dsmierror.log 44 dynamic content 4
I
initWAS.bki 33 initWAS.utl 33, 46 InstantDB 4
J
Java 7 Interface Definition Language 6 Servlets 6 Java APIs 2 Java Messaging Service 6 Java Naming and Directory Interface 5, 7, 79 Java Transaction API 6 Java Transaction Service 6 Java Virtual Machine 3 JavaScript 4
83
JavaServer Pages 2, 6 JDBC 6 JIDL 6 JMS 6 JNDI 5 JSP 23, 6 JTA 6 JTS 6 JVM 3
storage management 8 Sybase 4 systems management 3
T
TDP for WAS 14 backing up WAS 38 backup 16 Datamover 14 DB2 Shared Vendor Library 14 DB2 User Exit 14 DB2UEXT 16 db2uext2.utl 33, 46 deleting WAS backups 41 dsm.opt 35 environment variables 31 failed backup recovery 46 initWAS.bki 33 initWAS.utl 33, 46 limitations 19 Main Module 14 prerequisites 18 Prole service 14 querying WAS backup 40 restore 18 restoring WAS 41 tdppasswd 14, 36 tdpws 1416, 38, 4041 tracing 45 troubleshooting 42 vendor.env 34 tdppasswd 14, 36, 44 tdpws 1415, 38, 4041 Tivoli Data Protection 9 Tivoli Storage Manager 7 activity log 36 API 16 API client 9 archive description 57 backup versioning 58 Backup/Archive client 9, 51, 53 client 9 client polling scheduling 57 client scheduling 57 database 9 dsm.opt 33, 53 dsm.sys 56 dsmierror.log 44
L
LDAP 5 Lightweight Directory Access Protocol 5
M
model 6 MQSeries 6
N
naming service 5
O
objects 5 OO distributed computing environment 5 Oracle 4
P
password 33, 44
R
Redbooks Web site 82 Contact us x repository 4 restart database 47 RMI/IIOP 7 runtime management 3
S
SAN 7 Secure Sockets Layer 6 security 3 servlet 2, 14 single machine 2 snapshot directory 50 SSL 6 static Web pages 4
84
HSM client 9 LAN-Free 8 nodename 57 password 33, 44 password expiry 57 server 9 server prompted scheduling 57 TDP client 9 tdppasswd 44 virtualmountpoint 56 transaction 5 coordination 3
U
UNC 56
V
vendor.env 34 virtual hosts 4
W
WebSphere 2 administrative database 50 snapshot directory 50 WebSphere Application Server 4 administrative console 4 administrative database 4, 14 backup strategy 10 clone 6 Controlled Node 11 Master Node 11 naming service 5 setup for TDP for WAS 22 WebSphere V4.0 applications 54 backup strategy 50 backup versioning 58 complete recovery 62 distributed multinode 54 restore strategy 51 standalone node 54 WebSphere V4.0 restore 59 WLM 4, 6 workload management 3
X
XML 2, 17
Index
85
86
Back cover
Backing up WebSphere Application Server

with Tivoli Storage Management
WebSphere Application Server V3.5: backup/restore using TDP WebSphere Application Server V4.0: backup/restore using TSM Installation and configuration
This Redpaper describes how to back up and restore two different versions of WebSphere Application Server using Tivoli Storage Management products. WebSphere Application Server Version 3.5 and Version 4.0 are considered separately. WebSphere Application Server V3.5 can be backed up using Tivoli Data Protection for WebSphere Application Server. WebSphere Application Server V4.0 can be backed up using the Tivoli Storage Manager backup/archive client. The Redpaper presents an overview of WebSphere Application Server V3.5 and Tivoli Storage Management products, then shows you how to install, configure and run Tivoli Data Protection for WebSphere Application Server. For WebSphere Application Server V4.0, a script is created which will perform a backup of the administrative database and data files using operating system utilities and the Tivoli Storage Manager Backup/Archive client. We assume a basic knowledge of WebSphere Application Server and Tivoli Storage Management. product set.
Redpaper
INTERNATIONAL TECHNICAL SUPPORT ORGANIZATION
BUILDING TECHNICAL INFORMATION BASED ON PRACTICAL EXPERIENCE IBM Redbooks are developed by the IBM International Technical Support Organization. Experts from IBM, Customers and Partners from around the world create timely technical information based on realistic scenarios. Specific recommendations are provided to help you implement IT solutions more effectively in your environment.
For more information: ibm.com/redbooks

Backing Up WebSphere Application Server With Tivoli Storage Management Redp0149

Enviado por

Dados do documento

Descrição original:

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Backing Up WebSphere Application Server With Tivoli Storage Management Redp0149

Enviado por

Direitos autorais:

Formatos disponíveis

Front cover

Backing up WebSphere Application Server

Leos Stehlik Charlotte Brooks

Copyright IBM Corp. 2002

Backing up WebSphere Application Server with Tivoli Storage Management

...... ...... ...... ...... ...... ......

....... ....... ....... ....... ....... .......

...... ...... ...... ...... ...... ......

Backing up WebSphere Application Server with Tivoli Storage Management

Copyright IBM Corp. 2002

Backing up WebSphere Application Server with Tivoli Storage Management

The team that wrote this Redpaper

Copyright IBM Corp. 2002

Send your comments in an Internet note to:

Mail your comments to the address on page ii.

Backing up WebSphere Application Server with Tivoli Storage Management

Overview of WebSphere Application Server V3.5 and backup

Copyright IBM Corp. 2002

1.1 WebSphere Application Server overview

1.1.1 WebSphere Application Server architecture overview

Backing up WebSphere Application Server with Tivoli Storage Management

application Application server(s) Server(s)

Figure 1-1 WebSphere Application Server components

The following sections describe the components shown in this figure.

1.1.2 Administrative server

1.1.3 Application server

Chapter 1. Overview of WebSphere Application Server V3.5 and backup

1.1.4 Administrative database

1.1.5 Administrative console

1.1.6 Standard Edition

Backing up WebSphere Application Server with Tivoli Storage Management

1.1.7 Advanced Edition

Chapter 1. Overview of WebSphere Application Server V3.5 and backup

1.1.11 Workload management

1.1.12 Open standards

Backing up WebSphere Application Server with Tivoli Storage Management

1.2 Tivoli Storage Manager overview

DATA GENERAL DG/UX

DIGITAL OpenVMS (SSSI)*** UNIX Tru64 UNIX

Tivoli Storage Manager client platforms

NSM VM OS/400 MVS AIX Solaris HP-UX

Tivoli Data Protection for application Family:

Tivoli Storage Manager also supports:

Tivoli Storage Manager server platforms

Windows NT Windows 2000

Figure 1-2 Tivoli Storage Manager supported platforms

Chapter 1. Overview of WebSphere Application Server V3.5 and backup

1.2.1 TSM basic architecture

Backing up WebSphere Application Server with Tivoli Storage Management

Chapter 1. Overview of WebSphere Application Server V3.5 and backup

1.3 WebSphere Application Server backup strategy

1.3.1 Why it is important to back up WebSphere Application Server

Backing up WebSphere Application Server with Tivoli Storage Management

Full offline file-based backup

Full offline database backup with file data backup

Full online database backup with file data backup

Full online backup with TDP for WebSphere Application Server

Chapter 1. Overview of WebSphere Application Server V3.5 and backup

1.3.2 What else needs to be backed up?

1.3.3 Considering the right strategy for your environment

Backing up WebSphere Application Server with Tivoli Storage Management

Overview of TDP for WebSphere Application Server

Copyright IBM Corp. 2002