Benchmark Factory User Guide 6-8-1

Benchmark Factory for
Databases
User Guide
6.8.1
Copyright
2012 Quest Software, Inc.
ALL RIGHTS RESERVED.
This guide contains proprietary information protected by copyright. The software described in
this guide is furnished under a software license or nondisclosure agreement. This software may be
used or copied only in accordance with the terms of the applicable agreement. No part of this
guide may be reproduced or transmitted in any form or by any means, electronic or mechanical,
including photocopying and recording for any purpose other than the purchasers personal use
without the written permission of Quest Software, Inc.
The information in this document is provided in connection with Quest products. No license,
express or implied, by estoppel or otherwise, to any intellectual property right is granted by this
document or in connection with the sale of Quest products. EXCEPT AS SET FORTH IN
QUEST'S TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FOR
THIS PRODUCT, QUEST ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS
ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTS
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT. IN NO EVENT SHALL QUEST BE LIABLE FOR ANY DIRECT,
INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES
(INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESS
INTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE OR
INABILITY TO USE THIS DOCUMENT, EVEN IF QUEST HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES. Quest makes no representations or warranties with respect
to the accuracy or completeness of the contents of this document and reserves the right to make
changes to specifications and product descriptions at any time without notice. Quest does not
make any commitment to update the information contained in this document.
If you have any questions regarding your potential use of this material, contact:
Quest Software World Headquarters
LEGAL Dept
5 Polaris Way
Aliso Viejo, CA 92656
email: legal@quest.com
Refer to our Web site (www.quest.com) for regional and international office information.
Patents
Toad for Cloud Databases contains patent pending technology.
Trademarks
Benchmark Factory for Databases User Guide
Copyright
Quest, Quest Software, the Quest Software logo, Benchmark Factory, Spotlight, LiteSpeed,
Simplicity at Work, Toad, TOAD, T.O.A.D., and TOADWORLD and vToadare trademarks and
registered trademarks of Quest Software, Inc. For a complete list of Quest Softwares trademarks,
see http://www.quest.com/legal/trademark-information.aspx. Other trademarks and registered
trademarks used in this guide are property of their respective owners.
Benchmark Factory for Databases

User Guide
September 2012
Table of Contents
Copyright
Getting Started
17
Getting Started
17
Benchmark Factory Community
17
17
New in this Release
17
New for Release 6.8.1
17
New for Release 6.8
18
New For Release 6.7
18
Understanding Benchmark Factory
20
About Benchmark Factory
20
The Benchmark Factory Environment
20
Integration with Other Quest Software Products
21
Understanding the Benchmark Factory Environment
21
Understanding Benchmark Factory Terminology
22
How Benchmark Factory Works
25
Create a Script With Benchmark Factory
26
1. Set Up Benchmark Factory
27
Benchmark Factory Components
35
BenchmarkFactory Console
35
Agent(s)
36
Run Reports
37
Cutting and Pasting of Run Reports
38
Testing Server Throughput
38
Testing SQL Scalability
39
Testing Real Application Clusters
40
Table of Contents
Repository Manager
41
Views
44
Script View
44
Jobs Queue View
45
Properties View
47
Agents View
47
Output View
47
Properties
47
How Do I...
91
Overview
91
How do I...
91
User Scenario Wizard
92
Profile Creation Wizard
92
Overview
92
Edit a Profile
94
Submit Job Dialog
94
Run Reports
96
97
97
98
99
Advanced Creation Objects (Object Mapping)
100
Benchmark Objects Wizard
101
Changing Scaling Factors
103
To Change Scale Factors
103
Execute File Wizard
105
BFScript Wizard
105
Table of Contents
Script Wizard
105
Script Assist
107
SQL Scalability Wizard
107
Overview
107
Overview Tab
109
109
Transactions
109
Real-Time
110
Summary
110
Messages
111
Copy a Load Scenario to a Replay, Mix, Goal, or Scalability Test
112
Running SQL Preview
113
Replace Child Latencies
113
Goal Testing Quick Start
115
Overview
115
Actions Perfomred in This Quick Start
115
Use Global Search and Replace
124
Creating an ODBC Trace File
125
Real Application Clustering (RAC) Quick Start
125
Overview
125
Purpose of this Quick Start
126
Actions Performed in This Quick Start
126
Save the Selected Job to the Script View
136
Schedule A Job
136
About Latency
137
Set the Number of Users on an Agent
138
Stagger User Startup
140
Table of Contents
Running Benchmark Factory with Multiple Agents
140
Benchmark Factory Console
140
Benchmark Factory Agent
141
Running a Job With Multiple Agents
141
Creating Load Scenarios
142
Load Scenario Wizard
142
Overview
Settings
142
144
About Settings
144
General Settings
144
Benchmarks Settings
146
Timing Settings
147
User Load (Measurement Intervals)
148
LatencySettings
150
Repository Settings
152
Agent Settings
154
Profiles
156
Overview
156
Edit Profiles
156
Drivers
158
Profile Properties
163
Overview
163
General Tab
163
Options Tab
165
Profile Properties Real-Time Counters Tab
166
Clustering Tab
169
Table of Contents
Wizards
171
Overview
171
171
Data Repository Migration Wizard

171
173
173
Execute File Wizard
174
Execute File Wizard
174
175
175
Capture and Replay Oracle Workload
176
Industry Standard Benchmark (TPC-C, TPC-H...)
192
Validate Oracle RAC Cluster Scalability
193
Test Database Scalability
195
Test SQL For Scalability
196
Replay Load from an Oracle Trace File
197
Test Stored Procedures for Scalability
200
Replay Load from SQL Server Trace Table
201
Import from Foglight Performance Analysis Repository
202
Import from a Delimited Text File
203
Replay Load From an ODBC Trace File
205
Import from Oracle Dynamic Performance View
206
Import from Quest Tools Export Files
207
Create a Custom Load Scenario (Scalability, Goal...)
208
Mix Test
208
Replay Test
209
Goal Test
210
Scalability Test
212
Table of Contents
213
213
BFScript
BFScript Wizard
214
214
217
217
Submit Job Dialog
222
Submit Job Dialog
222
Transaction Wizard
224
Transaction Wizard
224
225
Create SQL Statement Transaction
226
Create Prepared SQL Statement Transaction(s)
227
Create Execute Prepared SQL Statement Transaction
231
Import Transactions
232
233
233
Import and Replay Load Scenarios From an Oracle Trace File
233
Replay Load from a SQL Server Trace Table
234
235
Replay Load from an ODBC Trace File
235
Create a Custom User Scenario
236
Testing Results (Run Reports)
237
Benchmark Factory TestingResults
237
Benchmark Factory Testing Results Terminology
237
Exporting Test Results to Excel Using the Command Line
238
Real-Time Load Testing Results
239
Overview-Viewing Real-time Statistics Workloads
239
10
Table of Contents
Change Graph Views
239
Summary Tab
242
Realtime Statistics Tab
242
Run vs.Userload Tab
244
Transaction Node
246
Benchmark Factory Run Reports
248
Run Reports
248
Change Graph Views
251
Run Reports Viewer
254
View Load Testing Results
256
Using Benchmark Factory Run Reports
257
View Multiple Run Results
270
Export BenchmarkFactory Test Results to Excel
274
Benchmarks
276
Overview
276
Realistic Expectations When Using Benchmarks
276
What Benchmarks Measure
276
Provided Benchmarks
276
AS3AP Benchmark
Best Practices
Scalable Hardware Benchmark
277
277
278
How the Scalable Hardware Benchmark Works
278
Scaling Factor
278
Best Practices
278
TPC-B Benchmark
279
Overview
279
Certification of Transaction Processing Council (TPC) Testing Results
279
Best Practices
279
11
Table of Contents
Scaling Factor
280
TPC-C Benchmark
280
Overview
281
TPC-C Tables
281
282
Best Practices
282
TPC-D Benchmark
283
Overview
283
283
Best Practices
283
Scaling Factor
284
TPC-E Benchmark
284
Overview
284
285
Best Practices
285
Scaling Factor
285
TPC-H Benchmark
285
Overview
285
Certification of Transaction Processing Council Testing Results
286
Best Practices
286
Scaling Factor
286
BFScripts
288
About Scripts
288
Using Scripts in Username and Password Fields
290
Dates and Numbers
290
$BFCreditCardExp
290
$BFCurrentDate
290
BFCurrentDateTime
291
12
Table of Contents
$BFDate
File Access
$BFFileArray
Global Variable
292
293
293
294
$BFGetGlobalVar
294
$BFSetGlobalVar
294
$BFSetGlobalVarRtn
295
Name and Address
295
$BFAddress
295
$BFAddress2
295
$BFCity
296
$BFCompany
296
$BFCountry
297
$BFEmail
297
$BFFirstName
297
$BFFullName
298
$BFLastName
298
$BFMiddleInitial
298
$BFPhone
299
$BFState
299
$BFZipCode
300
Numerical Manipulation
300
$BFFormat
300
$BFProd
301
$BFSum
301
Random Numbers
302
$BFRand
302
$BFRandRange
302
13
Table of Contents
$BFURandRange
303
$BFURand
303
Random String
304
$BFCreditCard
304
$BFRandList
304
$BFRandMultiList
305
$BFRandStr
306
$BFURandList
307
$BFList
307
String Manipulation
308
$BFAsc
308
$BFChr
308
$BFConcat
309
$BFLeft
309
$BFLen
310
$BFLower
310
$BFMid
311
$BFRight
311
$BFTrim
311
$BFTrimLeft
312
$BFTrimRight
312
$BFUpper
313
Test Info
313
$BFGetVar
313
$BFMaxNode
314
$BFNode
314
$BFNumberOfIterations
314
$BFProfile
315
14
Table of Contents
$BFRunID
315
$BFSetVar
316
$BFSetVarRtn
316
$BFUserCounter
317
$BFUserID
318
$BFUserLoad
318
$BFNextUserload
318
$BFPrevUserload
319
Quick Starts
320
About the Quickstarts
320
320
Overview
320
321
321
Upgrading Your Database Server Quick Start

Oracle Trace Import-Database Replay Quick Start
331
331
339
Overview
339
339
Standard Benchmark Quick Start
346
Overview
346
Actions Performed Duirng this Quick Start
346
353
Overview
353
354
Troubleshooting
363
Agent Connection
363
Use Benchmark Factory with SQL Server 2005 Client
363
15
Table of Contents
MySQL Initialization Settings

Appendix
Change Graph Views
363
364
364
Show Data/Show Graph
364
Graph Legend
365
Toolbar
366
Print
366
Copy Data to Clipboard
366
Copy Graph to Clipboard
366
Load Configuration
366
Save Configuration
366
Set as Default
366
Clear Configuration
367
Customize List Controls
367
Create a SQL 7 Trace Table Using the SQL Server Profiler
367
368
369
Oracle Instant Client Installation
370
Migrating Repository Data Using the DOS Command Line
371
Store Procedure Examples
372
Oracle Trace File Activation
373
Support Bundle
375
System Requirements/Upgrade Requirements/Supported Databases
376
System Requirements
Shortcut Keys
Copyright, Licensing, and Licensing Agreement
Copyright
376
378
380
380
16
Table of Contents
Licensing and Virtual User Packs
381
Modify the Benchmark Factory License
382
Modify the Benchmark Factory Virtual User Pack
383
Legal Notice
383
384
384
Appendix:Contact Quest
Contact Quest/About Quest Software
396
396
About Quest
396
Contacting Quest Software
396
ContactingQuest Support
396
Index
398
1
Getting Started
Getting Started
Benchmark Factory is an intuitive, easy to use database load testing tool. The following links
provide you with information to quickly start the testing process and gather results.
l
Video: Take a Tour of Benchmark Factory
About Benchmark Factory (page 20)
Understanding the Benchmark Factory Environment (page 21)
Understanding Benchmark Factory Terminology (page 22)
How Benchmark Factory Works (page 25)
Create a Script With Benchmark Factory (page 26)
Benchmark Factory TestingResults (page 237)
Quick Starts
Real Application Clustering (RAC) Quick Start (page 320)
Upgrading Your Database Server Quick Start (page 331)
Oracle Trace Import-Database Replay Quick Start (page 339)
Standard Benchmark Quick Start (page 346)
Goal Testing Quick Start (page 353)
Quest Benchmark Factory Web Site (contains additional videos and white papers)
Benchmark Factory Community

Get the latest product information, find helpful resources, and join a discussion with the
Benchmark Factory for Databases team and other community members. Join the Benchmark
Factory for Databases community at http://www.benchmarkfactory.inside.quest.com.
New in this Release

New for Release 6.8.1
This release of Benchmark Factory for Databases provides bug fixes.
18
Getting Started
New for Release 6.8

New features in this release:
l
New Oracle Workload Capture Method-This release of Benchmark Factory for

Databases provides a new method to accomplish Oracle Capture/Replay using Oracle's
Fine Grained Auditing (FGA) objects.
Customer Improvement Program -Benchmark Factory now tracks feature utilization in

order to provide feedback to improve Benchmark Factory for Databases.
Automatic Partitioning-Automatic Partitioning of TPC-H tables and indexes for Oracle

testing has been added.
User Interface for SQLStatements-The user interface for SQLStatements has been
enhanced.
Reports- A performance comparison report has been added to Run History to compare
Oracle runs.
Wizards-The Oracle and SQL Server Capture wizards user interfaces have been streamline
for easier use.
New For Release 6.7

New features in this release:
l
Improved Run Reports User Interface-Benchmark Factory Run Reports has been
improved to provide easier access to test results tables and graphs.
19
Getting Started
Import and Export of Test Results-You can now import and export test results form
Run Reports.
Increased Benchmark Loading Speed-Benchmarks used in the database testing process

now load faster to reduce database testing time.
2
About Benchmark Factory
Benchmark Factory is a database performance and code scalability testing tool that simulates
users and transactions on the database and replays production workload in non-production
environments. This enables developers, DBAs, and QA teams to validate that their databases will
scale as user load increases, application changes are made, and platform changes are
implemented. Benchmark Factory is available for Oracle, SQL Server, DB2, Sybase, MySQL, and
other databases via ODBC connectivity. The following topics provides an overview of the
Benchmark Factory Environment:
l
The Benchmark Factory Environment (page 20)
Understanding the Benchmark Factory Environment (page 21)
Understanding Benchmark Factory Terminology (page 22)
How Benchmark Factory Works (page 25)
Create a Script With Benchmark Factory (page 26)
The Benchmark Factory Environment

Benchmark Factory places enormous stress on a database system, which is typically hard to
achieve in a standard testing environment. A system typically breaks under extreme load. By
identifying system capacity and performance bottlenecks before they occur, Benchmark Factory
facilitates proactive testing, which in turn reduces downtime, development costs, and potential
loss of revenue. Benchmark Factory allows you to:
l
Determine system throughput and capacity for database systems
Simulate thousands of concurrent users with a minimal amount of hardware
Find applications that do not scale well with an increase in the number of users
Find breaking points, weak links, or bottlenecks of a system
Quantify application or server performance
All test results are collected and stored in the repository for data analysis and reporting.
Benchmark Factory collects a vast amount of statistics, including overall server throughput
(measured in transactions per second, bytes transferred, etc.) and detailed transaction statistics by
21
individual workstation producing a load. You use these statistics to measure, analyze, and
predict the capacity of a system.
Integration with Other Quest Software Products

Benchmark Factory integrates with other Quest Software products to allow IT departments to
quickly measure the capacity and performance of their systems to ensure that users will
experience fast response times. For instance, Benchmark Factory can run multiple workloads in
conjunction with Spotlight products. This provides the ability to detect and diagnose issues,
allowing users to resolve bottlenecks, slow performance, and application flaws before an
application is entered into production.
Understanding the Benchmark Factory Environment

Note: The Benchmark Factory Quick Starts provide procedures that introduce you to the
Benchmark Factory environment. See "About the Quickstarts" (page 320) for more information.
The following provides a narrative on the terminology, application, and mechanics of using
Benchmark Factory.
Automated Testing, Virtual Users, and Agents
When you load test with Benchmark Factory, you are performing automated testing. Automated
testing is the process of using software to test computer hardware or software implying that the
software is used instead of actual users. The software acts as a "virtual" user. For example, say
you would like to test your database with two hundred virtual users over a given period of time.
Benchmark Factory lets you select two hundred virtual users and the length of time you wish to
perform the test.
Benchmark Factory uses Agents to deploy the virtual users. An Agent is a software routine that
waits in the background and performs an action when a specified event occurs. One Agent can
simulate thousands of virtual users (limited by hardware and workload characteristics) at a time.
Each virtual user has their own connection to the system under test.
Understanding Benchmarks
A benchmark is a performance test of hardware and/or software on a system-under-test.
Benchmark Factory provides the option of using industry standard benchmarks during the load
testing process. Benchmarks measure system peak performance when performing typical
operations.
Benchmark Factory comes equipped with the following industry standard benchmarks:
l
AS3AP Benchmark (page 277)
Scalable Hardware Benchmark (page 278)
22
TPC-B Benchmark (page 279)
TPC-C Benchmark (page 280)
TPC-D Benchmark (page 283)
TPC-E Benchmark (page 284)
TPC-H Benchmark (page 285)
Understanding Benchmark Factory Terminology

The following provides a list of terminology required to understand the Benchmark Factory load
testing process.
l
A script is the user's workspace that allows you to create and save jobs.
l
Scripts are created in the Script View. See "Script View" (page 44) for more
information.
When ready to run, jobs are submitted to the Jobs Queue View. See "Jobs Queue
View" (page 45) for more information.
The Load Scenario Wizard is the starting point for the creation of scripts. See
"Load Scenario Wizard" (page 175) for more information.
A job contains load scenario, execution of a file, creation/deletion of benchmark objects,

or a combination of each.
A load scenario is an assembled test comprised of user scenarios and/or transactions.

These tests can be run with multiple virtual users. A load scenario can be one of the
following:
l
Mixed Workload: A Mixed Workload test runs for a specified time at each
predetermined user load level. Each user will run a transaction mix based upon
the weights defined on the transactions tab. For example, if a test has two
transactions, A and B, with A having a weight of one and B having a weight of
four, on average B will run four times for every time A is run once. The run order
will be randomly generated for each user so they are not all running the same
transaction simultaneously. That run order is used for that user each time the test is
performed to ensure reproducible results.
Goal Test: A goal test is used to find maximum throughput or response time
goals. A transaction mix is executed at user load levels, determined by setting a
beginning, ending, and interval value. When run, the specified goal criterion is
evaluated at the end of each iteration and the test ends once the goal or maximum
user load has been reached.
Replay Test: A Replay Test runs multiple transactions with each one running
independently on a specified number of users. The test will run until the defined
number of executions for each transaction or a specified time limit is reached.
23
SQL Scalability Test: A SQL Scalability test executes each transaction

individually for each userload and timing or execution period. For example, a
test could have two transactions, A and B, and two userloads of 10 and 20, with
an iteration length of one minute. Transaction A would execute continually for
one minute at userload 10, then B would do the same. Next A will run at userload
20, followed again by test B, for a total time of 4 minutes.
Click here for load scenario creation procedures.

l
A user scenario is a series of one or more transactions to be executed in order to make a

single transaction, normally associated with user behavior simulated against the systemunder-test. An Execute SQL Statement from a file User Scenario will execute SQL
statements as defined in an XML file. See "User Scenario Wizard " (page 233) for more
information.
A transaction is a single unit of work in the testing process, used in load scenarios and
user scenarios. For example, a SQL statement. Click here for procedures to create a
transaction.
Execute a file allows you to execute a file during the running of a job. See "Execute File
Wizard" (page 174) for more information.
Create or delete benchmark objects configures a system-under-test for industry standard

benchmark tests to measure system performance. Each standard benchmark has been
developed with specific system configuration requirements, that include tables, indexes,
data, etc. See "Benchmark Objects Wizard" (page 171) for more information.

24
25
How Benchmark Factory Works

The following steps provide an overview of how Benchmark Factory components interact during
the load testing process.
1. The BenchmarkFactory Console (page 35) implements the workload testing process and
controls one or more distributed agent machines. Each agent machine can simulate
thousands of users. Each simulated user executes transactions and records statistics.
2. Benchmark Factory Agent(s) (page 36)machines simulate virtual users. The Agents send
transactions to the system-under-test. The Agents record statistics that includes how much
data the resulting transaction contained and how long it took to get the results. At the
end of an iteration, each agent machine reports its findings back to Benchmark Factory.
3. A server (system-under-test) is the database Benchmark Factory connects to. Benchmark
Factory is server neutral and network protocol independent. Benchmark Factory uses
vendor client libraries of the system-under-test. Any system that the client software can
support, Benchmark Factory can support. Its only requirement is that Agent machines
must be able to connect to a server through an appropriate manner and it supports the
test system.
4. The Repository Manager (page 41) stores all testing data.
5. Results are viewed from Run Reports (page 248).

Create a Script With Benchmark Factory

The following provides a narrative discussion on how to create a script using Benchmark
Factory. This process consists of following steps:
1. Set up Benchmark Factory
2. Create a Script
3. Create a Profile
4. Create a Job
5. Run the Script
6. Analyze Test Results
26
27
Note: The Benchmark Factory Quick Starts provide procedures that introduce you to the
Benchmark Factory environment. See "About the Quickstarts" (page 320) for more information.
1. Set Up Benchmark Factory

The Settings window displays default values used when creating new scripts. Changes to these
settings affect only new load scenarios, not existing load scenarios.
To Display Benchmark Factory Settings
l
Select Edit | Settings. The settings dialog displays.
The Settings window provides the following tabs:

l
General Settings (page 144)
Benchmarks Settings (page 146)
Timing Settings (page 147)
LatencySettings (page 150)
Repository Settings (page 152)
Agent Settings
28
2. Create a Script
Using the Benchmark Factory graphical user interface, you can create scripts in the Script View.
You open one script at a time when using Benchmark Factory. See "BenchmarkFactory Console"
(page 35) for more information.
All tasks required to complete a Benchmark Factory script can be accomplished using wizards.
The first wizard you will use is the Load Scenario Wizard. The Load Scenario Wizard takes you
through the steps required to create a script and complete a database load test. See "Load
Scenario Wizard" (page 175) for more information.
29
The first step the Load Scenario wizard asks you to perform is to select a load scenario type from
the load scenario dialog. Load Scenarios are an assembled load test comprised of:
l
User scenarios: A series of one or more transactions to be executed in order normally

associated with a behavior to be tested on the system-under-test. These are the
components used to build a load scenario.
Transaction: A single unit of work in the testing process, such as retrieving a Web page,
executing a SQL statement, writing a file, or sending an email.
30
After selecting the desired task type, the Load Scenario Wizard will guide you through the steps
for script creation, depending on which type of task you select.
3. Create a Profile
Profiles are sets of information that Benchmark Factory uses to communicate with the systemunder-test . This information includes, server name, IP address, and user name. You create
profiles using the:
l
Profile Creation Wizard: (See "Profile Creation Wizard" (page 213) for more information.
Edit Profiles (See "Edit a Profile" (page 94) for more information.
31
4. Create a Job
A job holds a load scenario, execution of a file, creation/deletion of benchmark objects, or a
combination of each. Jobs are created in Benchmark Factory when a load scenario is added to a
script. A script can hold as many jobs as you chose to create.
32
Change Script Properties

Benchmark Factory script properties can be changed or adjusted to meet load testing requirement.
You change properties by clicking on the node of the property you wish to change. For example,
if you wish to change the user load test you click on the Load Scenario node. See About the
Script Node for more information.
33
To change the User Load properties, select the User Load tab and select the desired
number of users.
5. Run the Script

After completing the wizards and setting in the desired properties, you run the job by selecting
the Job Node and clicking the Submit Job icon
, or right-clicking and selecting Submit Job.
See "Submit Job Dialog " (page 222) for more information.
34
6. Analyze Test Results

Benchmark Factory provides a number of metrics to analyze a system-under-test. These include:
l
Response, retrieval, and transactions times
Transaction throughput in transactions per second (TPS)
Amounts of data transferred in bites per second (BPS)
Errors incurred
Transaction Distribution
Any user defined counters (Windows NT/2000 Performance Monitor) added using the
Benchmark Factory Console.
Benchmark Factory provides the following testing results:

l
Real time statistics (See "Realtime Statistics Tab " (page 242) for more information.)
Run Reports (See "Run Reports " (page 248) for more information.)
Testing results exported to Excel (See "Export BenchmarkFactory Test Results to Excel"
(page 274) for more information.)
3
BenchmarkFactory Console
The Benchmark Factory console implements the database workload testing process. This
interface is where:
l
Scripts are created and initiated. A script is the user workspace that allows you to design
and save jobs.
Jobs are created and submitted to the Jobs Queue View. See "Jobs Queue View" (page
45) for more information.
All script properties are set. See About the Script Node for more information.
Settings can be made. See "About Settings" (page 144) for more information.About
Scripts (page 288)
Users can access test results. See "Benchmark Factory TestingResults" (page 237) for
more information.
Agents are managed. See "Agent(s)" (page 36) for more information.
The Benchmark Factory Console contains the following views:

l
Script View (page 44)
Jobs Queue View (page 45)
Properties View (page 47)
Agents View (page 47)
Output View (page 47)
36
Agent(s)
Benchmark Factory Agent(s) reside on each physical client machine and spawn multiple virtual-user sessions. Each Agent
generates an information screen with tabs that contain a variety of Agent information. Benchmark Factory can control
hundreds of Agent machines. Each Agent can simulate up to 1500 virtual users, depending upon hardware and test
configuration. Virtual users simulate the load against the system-under-test. Each virtual user is a separate thread, acting
independently of the other virtual users, with its own connection to the system-under-test. Each virtual user tracks its own
statistics that includes transaction times and the number of times a transaction executes.
The Agent provides the follow tabs:

l
Agent Test Status Tab
Agent Virtual Users Tab
Agent Transaction Tab
Agent Output Tab
37
Run Reports
Benchmark Factory Run Reports is a separate executable that opens outside of Benchmark
Factory and provides a comprehensive and detailed collection of database load testing results.
With Benchmark Factory you can drill down into a database to view a wide array of information
and statistics that gives you accurate insight into database performance. Run Reports Viewer
allows you to access Benchmark Factory load testing results. See "Run Reports Viewer" (page
Note: Three instances of Run Reports can be viewed at one time.
38

From Benchmark Factory run reports you can cut and paste test results into the Clipboard. See
Cut and Pasting Reports for more information.

Server throughput is the measurement of database capacity. Benchmark Factory allows you to
view a variety of server throughput statistics that include:
l
Transaction/Second (TPS)
Transaction Time
Bytes/Second (BPS)
Rows/Second (RPS)
Total Bytes
Total Errors
Total Rows
Response Time
39

The Benchmark Factory transaction comparison graph provides users with a comparison of the
individual transaction performance as they compare to each other. This is most useful when
analyzing a SQL Scalability test.
40

Benchmark Factory allows you to load test Oracle Real Application Clusters (RAC) to determine
system capacity. The cluster performance graph allows users to view the performance of
individual nodes.
41
Repository Manager
Note: If you create a new Benchmark Factory 5.5 repository, earlier versions of Benchmark
Factory will not work against this repository.
The Repository is a database where all of the test results are stored. Benchmark Factory inserts
test results into the repository and provides an easy way to access the data. By default, the
Repository is a MySQL database that resides on the same machine as Benchmark Factory for
Databases. The Repository can reside on another database server if required. Tto change the
database select the Data Source Name of the ODBC connection for the new database.
Note: If the database structure does not exist on the selected database, a prompt to create the
structure will appear when OK is clicked.
The supported repository database types are:
l
Sybase Adaptive Server Anywhere
Microsoft SQL Server
MySQL
Oracle
42
To launch the Repository Manager

1. Select Start | All Programs | Quest Software | Benchmark Factory for Databases |
Repository | Repository Manager. The Repository Manager dialog displays.
Section
Connection
Parameters
Field/Icon
Description
Data Source
Name
Data Source name of the ODBC

connection used to connect to the
repository database.
ODBC Driver
Current ODBC driver
User Name
The User Name used to log into the

selected database.
Password
The Password associated with the user

name used to log into the database.
43
Section
Field/Icon
Description
Displays the ODBC connection

information dialog for the selected data
source.
Displays the ODBC Data Source

Administrator dialog, used to add and
edit ODBC connections.
Tests the connection of the currently

selected ODBC Data Source.
Maintenance
Creates the repository objects on the

selected database.
Deletes the repository objects on the

selected database.
Warning: This will delete all test
results stored in the Repository.
Launches the Data Migration Wizard.
See "Data Repository Migration
Wizard" (page 173) for more
information.
44
Views
Script View
The Script View displays the currently open script and provides user workspace that allows you
to design and save jobs. When ready to run, jobs are submitted to the Jobs Queue. The Script
View Toolbar provides user functionality when creating or editing load testing scripts.
Note: You must be in the script view to create jobs and edit jobs.
Script View Toolbar

The Script View toolbar provides user functionality when creating or editing load testing scripts.
Creates a blank job. You must click on the Script Node to active the icon.
Displays the Load Scenario Wizard (page 175)

Displays the User Scenario Wizard (page 233)
45
Displays the Transaction Wizard (page 224)

Displays the Benchmark Objects Wizard (page 171)
Displays the Execute File Wizard (page 174)

Displays the Submit Job Dialog (page 222)
Moves an item up in the Script View
Moves an item down in the Script View
Jobs Queue View

From the Jobs Queue, jobs are run, status can be viewed, and real-time testing results can be
viewed. The Jobs Queue Toolbar (page 46) provides user functionality to save or delete jobs.
The following job statues are provided:
l
Scheduled: All jobs currently waiting to run or are scheduled to run a future time.
Running: Job currently running.
Completed: All completed jobs.
46
Jobs Queue Toolbar

Jobs Queue icons provides user functionality when running jobs.
Deletes the selected job from the Jobs Queue.
Saves the selected job to the Script View. See "Save the Selected Job to the Script View"

Properties View
The Properties View allows you to monitor and managed scripts. Properties can be
viewed for the:
l
Script Node Properties (page 48)
Jobs Node
Agents View
The Agent view displays the status of connected agents.
Output View
The Output window displays errors and messages that occur outside of running a job. For
example, a .dll error or re-registering a benchmark.
Properties
About the Properties View
Benchmark Factory properties allow you to:
l
Adjust script parameters
View script properties
47
48
Access Benchmark Factory Run Reports See "Run Reports " (page 248) for more
information.
View realtime statistics (See "Realtime Statistics Tab " (page 242) for more information.
Set and view latency See "About Latency " (page 137) for more information.
The following provides Benchmark Factory properties definitions:

l
Script (See About the Script Node for more information.)
Job (See Jobs Node for more information.)
Latency (See "About Latency " (page 137) for more information.)
Benchmark Properties See "Benchmark Properties" (page 87) for more information.)
Script Node Properties

Script Node
The Script node displays the Benchmark Factory Home page and Help options.
Jobs Node
Note: In the Jobs Queue view, not all properties are editable.
The Jobs node provides the following tabs:
49
General
Alerts
Job Schedule
Real-Time Counters
Global Variables Node (page 72)
Load Scenario Node

Note: In the Jobs Queue view, no properties are editable.
The Load Scenario node provides the following tabs:
l
Transaction Mix Tab (page 49)
Transaction Tab
Goal Test (page 210)
User Load Tab (MeasJobs Nodeurement Intervals)
Timing Tab
Options Tab-Mixed Workload, Goal, and Scalability Test (page 54)
Transaction Mix Tab

Note: The Transaction Mix tab is only editable in the Script View.
This tab allows you to run a transaction mix based on the assigned weight. If a test has
two transactions, A and B, with A having the weight of one and B having the weight of
six, transaction B will run six for every time A runs once. The run order is randomly
generated for users.

Settings
Description
Name
Name assigned to the transaction.
Weight
Weight assigned to a transaction in a mix.
Percentage
Percentage assigned to the transaction.
Total
Weight
Total weight of all the combined transactions.
Deletes a selected transaction from the mix.
User Scenario Node

The User Scenario node properties allows you to:
l
Delete transactions
Adjust transaction order
Adjust latency (See "About Latency " (page 137) for more information.)
50

User Scenario From File

The User Scenario From File node properties allows you to:
l
Executing a SQL Statements from a File (page 51)
Executing a SQL Statements from a File

When large Oracle trace files are imported into Benchmark Factory, they are converted to
editable XML files.
To Edit SQL Statements from a file
1. Click Edit.
51

2. Click File | Open.
The Open dialog displays.

3. Navigate to and select the required file XML file.
4. Click Open.
5. Under Session Details, click on the SQL statement you wish to edit. That statement
displays in the SQL Statement view.
52

6. From the Prepare SQL tab, you can click in the upper right-hand corner to:
l
Run SQL Preview See "Running SQL Preview" (page 113) for more
information.
Launch the BFScript Wizard See "BFScript Wizard" (page 214) for more
information.
7. After you finish making the required edits, click File| Save to save your edits.
53

Transaction Node
The transaction node displays properties based on transaction type.
To view the Transaction node
Click the Transactions node in the Jobs Queue or Script View.
Options Tab-Mixed Workload, Goal, and Scalability Test

The Options tab specifies Repository, error handling, file execution, and database
checkpoint settings.
54
55
Section
Field/Icon
Description
Repository Settings
Save Real-Time Counter

Information
Saves counters and

performance monitor
counters to the Repository
during test. Sample rate is
determined in job settings.
Note: To change real-time

counters please refer to Profile
Properties Real-Time Counter
Tab.
Save User Statistics
Saves user level statistics
Save Response Distribution
Saves transaction per

second and response time
per user distribution.
Log Transaction Results to disk
Logs the transaction results

of a job to disk.
Note: Checking this option
imposes performance
penalties.
Error Handling
Stop test after first error
When checked, the test is

stopped when an error is
56
Section
Field/Icon
Description
reported.
Execute the following

program at the
beginning of each
iteration
Execute the following

program at the end of
each iteration
Database Checkpoints
File Name
Note: This field accepts
BFScripts.
Name of the program to

execute at the beginning of
an iteration.
Enforce Timeout
Enforces a timeout on the

file being executed.
File Name
Name of the program to

execute at the end of an
iteration.
Note: This field accepts

BFScripts.
Enforce Timeout
Enforces a timeout on the

file being executed so that
if the program does not
complete in the specified
time, it will be stopped, and
the load scenario will
continue.
Perform checkpoint at start of

each test iteration
Initiates a database
checkpoint at the beginning
of a test iteration.
Perform checkpoints during each

iteration
Initiates a database
checkpoint during a test
iteration.
Number of checkpoints
Specifies the number of

checkpoints to initiate.
Options Tab-Replay Test

The Repository Properties tab controls what information is saved in the Repository during the
running of a test, and allows you to set error handling properties. The database where all
Benchmark Factory information is saved. It contains information about the transactions stored in
each benchmark DLL and is where all of the statistical information is stored about benchmark
executions.
57
To change Repository properties

1. Click the Load Scenario node.
2. Click the Options tab.
Setting
Description
Repository Settings
Save Real-Time Counter

Information
Saves Benchmark Factory real time counters and

performance monitor counters to the repository during
test.
Save User Statistics
Saves user level statistics.
Save Distribution
Saves transactions per second and response time

distribution.
Error Handling
Stops the test when an error is returned.
Run Reports
58

l
Transaction Time
Bytes/Second (BPS)
Rows/Second (RPS)
Total Bytes
Total Errors
Total Rows
Response Time
59

60

individual nodes.
61
SQL Statement Node

The SQL statement node displays individual SQL statements and allows you edit a SQL
statement or use the BFScript Wizard. See "BFScript Wizard" (page 214) for more information.
62
Create Objects Node

This node allows you to either create the benchmark objects or delete the benchmark objects.
Benchmark objects are used by standard benchmark load scenarios. Benchmarks can be added to
a job using the Benchmark Objects Wizard. See "Benchmark Objects Wizard" (page 171) for
more information.
Field
Description
Benchmark
Name of the benchmark whose benchmark properties are being

edited.
Scale
The factor used to increase the amount of data added to the

database.
Valid benchmark scales are dependent upon the benchmark selected.
For example, the AS3AP scales by a factor of 10 with valid scales of
"1, 10, 20, 30..." and the TPC-C benchmark scales by a factory of 1,
with valid scales of "1, 2, 3, 4..."
63
Field
Description
Displays the Benchmark Properties page and allows you to edit the
benchmark scale and map database tables and indexes. See
"Benchmark Properties" (page 87) for more information.
Always delete
objects before
create
Checking this box deletes benchmark objects before creating new

ones.
When unchecked, Benchmark Factory checks for the benchmark
tables. If they exist, then the creation will be skipped.
Note: The scale factor is not checked.
Distribute load
using Benchmark
Factory Agents
Distributes data generation across multiple agent machines to speed

up the standard benchmark data load.
Number of virtual
users to perform
creation/load
Number of virtual users used to perform the test.
Execute File Node

The Execute File node allows you to place a file execution in a job as a sibling to a load
scenario. The file executes during the running of a job.
64
Field
Description
Execute File
Type in or Select the File Name to execute:

Name of the file you wish to execute.
Note: This field accepts BFScripts. See "About Scripts" (page
Click
to open a file.
Enforce Timeout
Places a time constraint on the number of seconds a file has to

execute.
Enter a number
between 30 and 10000
[n] Secs.
The number of seconds a file has to execute.
Jobs Queue Node Properties

Using the Jobs Node and Tabs
Note: Some of the nodes in the Jobs Queue are the same as the Script view. Not all nodes in the
Jobs Queue are editable and provide property viewing only.
The Jobs Node and tabs allow you to edit or view job parameters. These tabs can be used
when jobs are:
l
Scheduled (page 64)
Running (page 65)
Completed (page 66)
Scheduled
When a job is scheduled to run, it is placed in the Scheduled Jobs Queue.

Note: Not all of the tabs are editable in the schedule section.
The Job Queue provides the following tabs:
l
Job Setup Tab (page 66)
Create Objects Node (page 67)
Run Job Tab (page 68)
See Result Tab (page 69)
Running
As a job runs, clicking on the Run Job tab allows you to view realtime testing results.
The following provides details on realtime testing viewing:

l
Summary Tab (page 242)
Realtime Statistics Tab (page 242)
Run vs.Userload Tab (page 244)
Transaction vs. Load Tab (page 82)
65
66
Completed
Jobs in the Completed Jobs Queue section allow you to perform limited test edits and completed
testing results using the See Result Tab (page 69).
Note: Not all of the tabs are editable in the schedule section.
The following provides tab details in the Completed tests section:
l
Job Setup Tab (page 66)
Create Objects Node (page 67)
Run Job Tab (page 68)
See Result Tab (page 69)
Job Setup Tab

This tab allows to view or edit job parameters and provides the following tabs:
l
General
Jobs Alerts
Real-Time Counters
Global Variables Node (page 72)
67
Create Objects Node

This node allows you to either create the benchmark objects or delete the benchmark objects.
Benchmark objects are used by standard benchmark load scenarios. Benchmarks can be added to
a job using the Benchmark Objects Wizard. See "Benchmark Objects Wizard" (page 171) for
more information.
68
Field
Description
Benchmark
Name of the benchmark whose benchmark properties are being

edited.
Scale

database.
Valid benchmark scales are dependent upon the benchmark selected.
For example, the AS3AP scales by a factor of 10 with valid scales of
"1, 10, 20, 30..." and the TPC-C benchmark scales by a factory of 1,
with valid scales of "1, 2, 3, 4..."
Displays the Benchmark Properties page and allows you to edit the
benchmark scale and map database tables and indexes. See
"Benchmark Properties" (page 87) for more information.
Always delete
objects before
create
Checking this box deletes benchmark objects before creating new

ones.
When unchecked, Benchmark Factory checks for the benchmark
tables. If they exist, then the creation will be skipped.
Note: The scale factor is not checked.
Distribute load
using Benchmark
Factory Agents
Distributes data generation across multiple agent machines to speed

up the standard benchmark data load.
Number of virtual
users to perform
creation/load
Number of virtual users used to perform the test.
Run Job Tab

This RunJob tab allows you to schedule jobs and view job results. It provides the
following tabs:
l
Schedule
Summary Tab (page 242)

Realtime Statistics Tab (page 242)
Run vs.Userload Tab (page 244)
Transaction vs. Load Tab (page 82)
See Result Tab

The see results tab allows you to:
l
Select results of tests based on profile selection
Sort test results by successful completion or those that completed with errors.
View graphs of completed tests.
To use the See Results Tab:

1. From the Profiles drop-down, select the All Profiles or the required profile.
69
70
2. From the Status drop-down, select the required run type.
3. Click on the required requied test to view graphs.
4. To delete a test, select the test and click Delete in the upper right section of the dialog.
71
5. To view Run Reports, select the required test and click the Show Test Results button in
the upper right section of the dialog.
See "Run Reports " (page 248) for more information.
Load Scenario Node

The Load Scenario node provides the following tabs:
l
Transaction Mix Tab (page 49)
Transaction Tab
Goal Test (page 210)
User Load Tab (MeasJobs Nodeurement Intervals)
Timing Tab
Options Tab-Mixed Workload, Goal, and Scalability Test (page 54)
User Scenario Node

The User Scenario node properties allows you to:

Delete transactions
Adjust transaction order
Transaction Node
The transaction node displays properties based on transaction type.
To view the Transaction node
Click the Transactions node in the Jobs Queue or Script View.
Global Variables Node

From the global variable node, you can add or delete the required variables.
72

To add a global variable:

1. Right-click to display the global variable dialog.
2. Enter the name and value.
3. Click Add.
The new global variable dislays.
73
74
Run Reports

l
Transaction Time
Bytes/Second (BPS)
Rows/Second (RPS)
Total Bytes
75
Total Errors
Total Rows
Response Time

76

individual nodes.
77
Summary Tab
The Summary Tab properties displays information on the job when it is running. You can
monitor the status of the test, which userload its running, transactions per second (TPS), and a
variety of other datapoints that give you insight into the actual performance your database
provides under the userload being tested.
Iteration Overruns
Iteration overruns occur at the end of an iteration to allow time for all transactions submitted
within the test iteration cycle to complete, so that all transaction statistics can be collected. For
example, an agent may execute a transaction during the last five seconds of test iteration, if this
transaction takes 15 seconds to complete, an iteration overrun of 10 seconds will occur.
78

The Real Time Statistics node provides you with real-time graphs and raw data. This data allows
you to spot system-under-test issues that may be affecting server performance. The Real Time
Statistics node displays real-time counter data. Right-clicking inside the graph displays a dropdown that allows you to change graph settings and view.
To view Real-Time Statistic
1. In the Jobs Queue, click the job running.

2. Select Click here to Add Datapoints. The Add Datapoints dialog displays.
3. Select the desired datapoints to view.
79
80
3. Click OK. The graph displays with the selected datapoints. (To change graph views,
right-click. See "Change Graph Views" (page 364) for more information.)
Run vs.Userload Tab

The Run vs. Load tab compares the user load transaction against the selected datapoint allowing
you to view how the various datapoints are being affected by the job running.
To view Run Vs.Userload tab testing results:
1. While the job is running, click the RunJob tab, then select the Run vs.Userload tab.
2. Click the datapoints Vs. Userload drop-down and select the desired datapoint.

The graph updates with the selected datapoint.
3. To change graph views, right-click. See Change Graph Views for more information.
81
82
Transaction vs. Load Tab

The Transaction vs. Load tab compares the user load transaction against the selected datapoint
allowing you to view how the various datapoints are being affected by the job running.
To view Transaction vs.Load tab testing results:
2. Click the datapoints vs. Userload drop-down and select the required datapoint.

3. To change graph views, right-click. See "Change Graph Views" (page 364) for more
information.
Messages Node
The Messages node displays:
l
Job status
Any errors or warnings
83
84
Latency
About Latency
Latency is the speed in which transactions are sent to a server or the period between receiving
results of one transaction and sending the next transaction to a serve. Benchmark Factory allows
you to select the following latency models and distribution models:
Latency Types
Distribution Models
No Delay (page 84)
Absolute Distribution (page 86)
Interarrival Time (page

85)
Uniform Distribution (page 86)
Keying Time (page 85)
Negative Exponential Distribution (page

86)
Think Time (page 85)
Normal Distribution (page 86)
Poisson Distribution (page 87)
No Delay
No Delay means that transactions execute as fast as possible. As soon as one transaction is
processed, the next transaction is issued against the server. In the case of a mixed workload test,
each virtual user issues transactions as fast as possible. The No Delay option is used when the
goal of the test is to stress the system to its limits, without concern for accurately simulating
85
users. With No Delay specified, a relatively low number of users can stress the system to its
limits. As such, there is no easy way to correlate N virtual users running with no delay to some
number of real users.
Interarrival Time
Interarrival Time derives its name from achieving a specific transaction rate as seen by a server
(the rate at which transactions are arriving at a server.) When specifying Interarrival Time,
Benchmark Factory is instructed, that regardless of how long a transaction actually takes to
execute, to ensure that the transactions arrive at a server at a specific interval.
Keying Time
Keying Time is used to simulate an amount of time performing data entry, before executing a
transaction (entering information). In many cases, Keying Time is used with Think Time to
provide a delay before and after a transaction execute. Selecting Keying Time inserts a fixed or
variable delay before executing a transaction.
Think Time
Think Time is used to simulate an amount of time "to think" about the results of a previous
transaction. This could be time spent performing analysis on the results of a database query.
Selecting Think Time inserts a fixed or variable delay after executing a transaction.
Latency Distribution Models

About
Benchmark Factory provides the following distribution models
l
Negative Exponential Distribution (page 86)
86
Absolute Distribution
Absolute is a fixed length delay. If a 2000 ms delay is specified for Keying Time (page 85) and a
3000 ms delay is specified forThink Time (page 85), each time the transaction should execute,
Benchmark Factory waits 2000 ms, then fires the transaction, then waits an additional 3000 ms
before deciding which transaction to execute next. If an Inter-Arrival model is chosen with a
delay type of Absolute set to 2000 ms, Benchmark Factory marks the time, executes the
transaction, and assuming the transaction finishes in less than two seconds, waits until two
seconds from the marked time has elapsed before determining which transaction to execute next.
Uniform Distribution
Selecting a Uniform delay instructs that random delay should be chosen, with an equal
probability of being the minimum value, the maximum value, or any value in between. Uniform
delays are chosen when it is suspected that the delay is highly random within a range or a
minimal amount of statistical analysis has been performed to determine how the actual users
react. Suppose a uniform Keying Time (page 85) is selected with a minimum value of 1000 ms
and a maximum value of 1500 ms. If the transaction is executed more than 500 times, there is a
high probability that each possible delay has been selected at least once. With the other delay
types, this is not the case. If the same 2000 ms to 2500 ms uniform delay is set for Interarrival
time, the tester essentially is setting the test so that a server sees the transaction every 2 to 2.5
seconds, instead of exactly two seconds as in the Absolute delay time.
Negative Exponential Distribution

Similar to the Normal distribution, Negative Exponential inserts a random delay based on a
mathematical model. A Negative Exponential distribution should be chosen when most users
have latency of the mean latency or less. Negative Exponential is weighted heavily toward
smaller latencies being selected relative to the mean and maximum latency.
Normal Distribution
Normal distributions differ from Uniform delays in that most of the delays chosen by Benchmark
Factory will be close to the average, but can vary by as much as 10% of the mean. While a
Uniform delay is used when users have latencies within equal likelihood of being anywhere
between two values, Normal distributions are chosen when all users fall within a range, but most
of the modeled users have a latency close to the average latency.
87
Poisson Distribution
A Poisson distribution is very similar to the Normal distribution and can be used most places
where a Normal distribution delay could be used. The biggest difference between a Normal
distribution and a Poisson distribution is that Poisson selects discreet values.
Benchmark Properties
Benchmark Properties
Benchmark scale factors are used to increase the size of the database testing tables during the
creation of benchmark objects, and to correctly execute the benchmark transactions as needed to
fully utilize the database objects created.
The Benchmark Properties page displays information on:
l
Scales (page 87)Scales (page 87)
Advanced (page 89)
History Tables (page 89)
Scales
The scale dialog specifies the benchmark scale used to create table data
88
Field
Description
Benchmark Scale

database.
Valid benchmark scales are dependent upon the benchmark
selected. For example, the AS3AP benchmark scales by a factor of
10 with valid scales of "1, 10, 20, 30..." and the TPC-C benchmark
scales by a factory of 1, with valid scales of "1, 2, 3, 4..."
Show Empty
Tables
Displays any tables that will be created but not populated with
data.
Estimated Size
The approximate size that the tables will use in the database
without indexes. Additional space is needed for indexes.
89
Advanced
Advancer mapping provides views of the tables and indexes that allows you to determine where
there tables and indexes will be stored in the database. This allows your database to run more
efficiently. See "Advanced Creation Objects (Object Mapping)" (page 100) for more information.
History Tables
History tables allows you to set the number of history tables to create. The TPC-B benchmark is
made up of only one transaction that updates three tables and inserts a record into a history table.
Inserting one record into one history table limits testing performance. The Benchmark Factory
properties page allows the user to set the number of history tables to create during a test. The
best ratio of history tables to virtual users is based on database configuration and hardware. The
number of history tables to use is determined by the tester.

Field
Description
Number of History
tables
Allows you to set the number of history tables to create.
90
4
How Do I...
Overview
The How Do I section points you to the procedures required to perform load testing.
How do I...
l
Create a Load Scenario - This wizard walks you through the steps to create and run
scripts and load scenarios.
Create a User Scenario - This wizard walks you through the steps to create a
user scenario.
Create a Profile - This wizard walks you through the steps to hook up to the data base
you are testing.
Edit a Profile
Submit a Job - This wizard walks you through the steps to submit a job to the Jobs
Queue for testing.
Use Run Reports
Use Advanced Creation Objects
Create Benchmark Objects for Load Testing This wizard walks you through the steps to
create benchmark objects for load testing.
Execute a File - This wizard walks you through the steps to execute a file outside of
Benchmark Factory.
Change Scale Factors
Use Benchmark Factory Scripting Capabilities - This wizard walks you through the steps
to use Benchmark Factory built-in functions for use in scripts and the load testing process.
Use SQL Scalability Wizard
Copy a Load Scenario to a Replay Test, Mix Test, or Goal Test
Run SQL Preview
Perform a Goal Test
Create a Customized ODBC Trace File
Run a Real Application Cluster (RAC) Workload
92
How Do I...
Set Latency
Run Benchmark Factory with Multiple Agents

The User Scenario Wizard allows you to build user scenarios. A user scenario is a series of one
or more transactions to be executed in order, normally associated with user behavior simulated
against the system-under-test. From the User Scenario Wizard you can:
l
Replay load from SQL Server trace table
Import from a delimited text file
Replay from an ODBC trace File
Create a custom user scenario

Overview
Profiles are used by Benchmark Factory to connect to the system-under-test. The profile contains
information required to connect to the system-under-test, such as server name, IP address, and
user name. The Profile Creation Wizard walks you through steps required to create a Benchmark
Factory profile.
To create a profile
1. Select Wizards | Profile Creation.
2. Click Next.
3. From the Driver drop-down list, select the required driver type.
4. Click Next.
5. Enter the User/Schema.
93
How Do I...
6. Enter the Password.

7. From the TNSDriver drop-down, select the required database.
or
Click the Direct tab and enter the required fields.
8. From the Connect as drop, select the required connection, Normal, SYSDBA, or
SYSOPER.
9. Select the installed client if applicable.
10. Click the Verify connection before proceeding check box to verify the connection to the
specified database.
11. Click Next.

12. Enter a profile name.
13. Click Finish.
94
How Do I...
Edit a Profile
To edit a profile
1. Click the Edit Profile icon
2. Double-click on a profile to edit or click View Properties

the dialog.
in the upper right corner of
The Edit Profiles dialog displays. The following options are provided.
l
Adds a profile
Deletes a profile
Displays profile properties
allowing you to edit the profile
Submit Job Dialog

A job contains load scenario, execute file, creation/deletion of benchmark objects, or a
combination of each. The Submit Job Wizard walks you through the process of adding a job to
the jobs queue. See "Jobs Queue View" (page 45) for more information.
To submit a job to the Jobs Queue
1. Select the Job node.
2. Right-click.
3. Select Submit Job. The Submit Job dialog displays.

How Do I...
The Submit Job dialog allows you to tailor your job needs and provides the
following options:
Section
Description
Profile
Allows you to select profiles already created.
Displays the Profile Creation Wizard. See

"Profile Creation Wizard" (page 213) for more
information.
Displays the Profile setting dialog. See

"General Tab" (page 163) for more information.
Actions
Allows you to:

Run Job: Submits the job to the queue and runs the
job.
Run Job when queue becomes idle: Submits the job
to the jobs queue and puts it in a ready state so that
it will run when the currently ready or running job
completes.
95
96
How Do I...
Section
Description
Run Job per schedule: Submits the next job

scheduled.
Start Agent and Run Job: Starts the agent and runs
the job.
Start Agent and Run Job when queue becomes
idle: Starts the agent and runs the job when all
previous jobs have finished.
Start Agent and Run Job per schedule: Starts the
agent and runs the next job scheduled.
No action: Closes the dialog after you select
Submit.
Comments
Allows you to add a comment to the job.
Alerts
Allows you to send an email, pager, or net-send

operator alerts when the job completes.
Job Counters
Allows you to add Job Counters to the job.
Stop job after

first error
When checked, the job will stop if an error is

encountered.
4. After filling in the desired parameters, click Submit.
Run Reports
97
How Do I...


l
Transaction Time
Bytes/Second (BPS)
Rows/Second (RPS)
Total Bytes
Total Errors
Total Rows
Response Time
98
How Do I...

99
How Do I...

individual nodes.

How Do I...
Advanced Creation Objects (Object Mapping)

The Advanced Creation Option allows you to set advanced properties for tablespaces and
datafiles, used for the creation of standard benchmark tables and indexes.
To create advanced objects
1. From the Database Type drop-down, select the desired database type.
2. Select the Tables or Indexes radio button.
3. Select the object(s) you wish to map.
100
101
How Do I...
4. Click Edit Options. The Mapping Options dialog displays.

5. Select the desired profile from the Load Tablespaces from Profile drop-down.
6. Select the desired options.
7. Click OK.
8. To save the mapping options, click Save. The Save Options File As dialog display.
9. Enter the desired name.
10. Click Save.

The Benchmark Objects Wizard creates or clears the database objects (tables/indexes, etc.)
needed to run industry standard benchmarks. The Benchmark Objects Wizard is able to run if
you have a root node job, load scenario, Create/Delete Benchmark object, or Execute File node
selected. The Create/Delete Benchmark object node will be added just below the item selected or
as a child of the job.
The Benchmark Objects Wizard:
l
Selects the benchmark to create objects for
Determines the scale of a benchmark
Creates an instruction to create the benchmark objects
To create a Benchmark Object

Note: To create, delete, and modify database tables/indexes you must have the user login
rights required.
1. Select the Job or Load Scenario node.
2. Select Wizards | Benchmark Objects.
102
How Do I...
3. Click Next.
4. From the drop-down, select the desired benchmark.
5. Click Next.
6. Select the desired benchmark scale in Benchmark Scale. When setting up a load test,
Benchmark Factory allows you to change the scaling factor of the standard benchmarks
provided. Scales factors increase the size of a database during the testing process allowing
realistic testing. By the scale factors, the number of rows added to the tables increases,
allowing for larger userloads to be used, placing a greater stress on the system under test.
During load testing, you can run various test with numerous scaling factors to ensure your
database performs at various performance levels.
7. The Advanced Creation Option allows you to set advanced properties for tablespaces and
datafiles, used for the creation of standard benchmark tables and indexes. To add mapping
options click Advanced Creation Option or proceed to step 18.

13. Click OK.
16. Click Save.
17. Click OK.
18. Click Next. The Create/Delete benchmark objects dialog displays and provides the
following options:
l
Create Benchmark Objects: Creates and initializes benchmark objects (tables, indexes,
etc.) required to run the standard benchmark transactions.
103
How Do I...
Always delete objects before create: Deletes existing benchmark objects (tables,
indexes, etc.) before creating new ones.
Note: Checking the "Always delete objects before create" check box will delete all
benchmark objects in the selected database and then re-create the objects in the
databases. When the checkbox is unchecked, Benchmark Factory checks for the
benchmark tables. If they exist, then the creation will be skipped. The scale factor is
not checked.
Delete Benchmark Objects: Deletes existing benchmark objects (tables, indexes, etc.).
19. Select the desired option.

20. Click Next. The Benchmark Objects Wizard completion dialog displays.
21. Click Finish. The Benchmark Object node creates and displays in the Script View.
Changing Scaling Factors

When setting up a load test, Benchmark Factory allows you to change the scaling factor of the
standard benchmarks provided. Scales factors increase the size of a database during the testing
process allowing realistic testing. By changing the scale factors, the number of rows added to the
tables increases, allowing for larger userloads to be used, placing a greater stress on the systemunder-test. During load testing, you can run various test with numerous scaling factors to ensure
your database performs at varying user load levels.
Benchmark Factory provides the following scalable benchmarks:
l
AS3AP
Scalable Hardware
TPC-B
TPC-C
TPC-D
TPC-H
TPC-E Benchmark (page 284)
The Benchmark properties page allows you to change scale factors properties.
To Change Scale Factors

You can set the load test scale factor when:
l
Creating a new script
Creating/delete a benchmark object, using the Benchmark Object Wizard
In Benchmark Factory settings
From the Benchmark Object's property page
104
How Do I...
Setting the Scale Factor in Benchmark Factory Settings

Benchmark Factory settings allows you to set default scaling factors so you don't have to set it
each time you create a new load test.
To set a default scale factor
1. Select Edit | Settings. The settings dialog displays.
2. Click Benchmarks.
3. Double-click the desired benchmark. The Benchmark Properties dialog displays.
4. Set the desired scale in Benchmark Scale.
5. Click OK.
6. Click OK.
Changing the Scale Factory from the Benchmark Object Property Page
After creating the a benchmark object, you can edit that object to change the scale factor.
To edit the scale factor for a Benchmark Object
1. Click on the desired Benchmark Object in the Script view.
2. In the properties view for the Benchmark Object, select Edit. The Benchmark Properties
page displays.
4. Click OK.
105
How Do I...
Execute File Wizard

During the running of a workload, you may want to add an executable file (.bat or .exe) as part
of a job. This gives you the flexibility of setting up your database before running a workload.
For example, you could run a .bat file to set up a prepare SQL statement before running a
workload.
To execute a file:
2. Select Wizards | Execute File.
3. Click Next. The Execute File dialog displays.
4. Enter the path or navigate to the desired file.
5. Click the Enforce Timeout checkbox if desired. Enforce Timeout allows you to enter the
desired number of seconds Benchmark Factory waits for the execution to complete before
continuing the job.
6. Enter the desired number of seconds.
7. Click Next. The completion dialog displays.
8. Click Finish. The Execute File is created and displays in the Script View.
BFScript Wizard
Note: Controls with yellow background can be scripted.
Benchmark Factory scripting allows you to randomize the load testing process. Scripting can be
utilized by selecting the SQL Statement node and inserting the script in the properties view. A
field has scripting capabilities if its right-click menu contains the Script Wizard option.
The Script Wizard is a quick and easy way to use Benchmark Factory scripts. The Script Wizard
consists of a script category and a script function. Each script function has a short description
included and if applicable, its parameters.
There are two features within Benchmark Factory that assist you when using scripting
capabilities:
l
Script Wizard
Script Assist
Click here for an overview of $BFScripts.
Script Wizard
1. Right-click the edit control. A drop-down display.
106
How Do I...
2. Select BFScript Wizard from the drop-down. The Script Wizard displays.
3. Select the desired script.

4. Some scripts require further parameters settings. For a complete overview of individual
script settings, please refer to Scripts Overview.
If scripts require parameters:
1. Click Next.
2. Enter the required parameters.
107
How Do I...
5. Click Finish. The script and parameters display in the Script Wizard.
Script Assist
1. Type in the desired SQL statement. When $BF is typed the Script Assist is activated
allowing you to insert the desired script.

3. Press Enter. The script displays.
4. Type in parameters as required.

Note: To run the SQL Scalability Wizard from Toad you must have Benchmark Factory loaded.
The Benchmark Factory console does not have to be open to run the SQL Scalability Wizard.
Overview
The SQL Scalability test allows you to execute SQL statements, letting users spot potential
issues not seen with a single execution. Users can run variations of a SQL statement generated
by SQL Tuning in order to find the SQL that will perform the best under a load test.
Creating a SQL Scalability load scenario requires the following steps.
108
How Do I...
1. Creating the SQL tuning connection

2. Entering the desired SQL statement
3. Running the statement using the in Benchmark Factory SQL Scalability Wizard
Creating the SQL tuning connection
To create the SQL Turning connection
1. In Quest Central or Toad, connect to an Oracle or SQL Server database.
2. From the Quest Central main toolbar select Tools|SQL Tuning to launch.
Entering the desired SQL statement
To enter the desired SQL statement
1. Enter the desired SQL statement in SQL Text.
2. Click the Optimize Statement icon. Clicking this icon executes multiple syntax
transformation rules to produce a list of semantically equivalent SQL statements.
Running the SQL statement using the in Benchmark Factory SQL Scalability Wizard
1. Click the Benchmark Factory drop-down icon and select the desired option. Three options
are provided:
2. Test for Scalability-Tests the currently displayed SQL.
3. Test All for Scalability-Tests all SQL statements.
4. Test Selected for Scalability-Tests the selected SQL statements.
2. The Benchmark Factory SQL Scalability Wizard displays. Click Next. The Measurement
Interval dialog displays.
3. Enter the desired user load.
4. Click Next. The Iteration Length dialog displays.
5. Enter the desired number of executions per iterations or executions per iteration.
6. Click Next. The Real World Latencies dialog displays.
7. Select the desired latency.
8. Click Next. The connection information dialog displays.
9. Enter the connection parameters.
10. Click Next. The Connection Agent dialog displays. Benchmark Factory Agent(s) reside
on each physical client machine and spawn multiple virtual-user sessions. Each Agent
generates an information screen with tabs that contain a variety of Agent information.
Benchmark Factory can control hundreds of Agent machines.
11. If there are more than 20 users in your test, you will need to run an agent. Click Start
Local Agent.
12. Click Next. The Benchmark Factory SQL Wizard completion dialog displays.
109
How Do I...
13. Click Finish. The SQL Scalability dialog displays. From this dialog, you can control,
monitoring, and view SQL testing results. This dialog provides the following tabs
and buttons:
Overview Tab
The Overview tabs provides transactions per second testing results for individual user loads and
iterations.
Transactions
The Transactions view provides the following statistics on the SQL statements being run:
l
Executions
Rows
Bytes
Errors
Average Transaction Time
110
How Do I...
Minimum Transaction Time
Maximum Transaction Time
Real-Time
Real-Time Statistics provides real-time graphs and raw data. This data allows you to spot systemunder test issues that may be affecting server performance. Right-clicking inside the graph
displays a drop-down that allows you to change graph settings and view.
Summary
The Summary graph provides real-time and raw data summary graphs. The Summary graph
displays real-time counter data. Right-clicking inside the graph displays a drop-down that allows
you to change graph settings and view.

How Do I...
Messages
The Output window displays messages about the job being run.
111
112
How Do I...
Copy a Load Scenario to a Replay, Mix, Goal, or

Scalability Test
Load scenarios can be quickly converted from one type test to another by using the "copy"
command. A load scenario is an assembled test comprised of user scenarios and/or transactions.
These tests can be run with multiple virtual users. A load scenario can be one of four types: mix
test, goal test, replay test, or scalability.
In Benchmark Factory you can copy a:
l
Replay test
A Replay Test runs multiple transactions with each one running independently on a specified number of users. The
test will run until the defined number of executions for each transaction or a specified time limit is reached.
Mixed test
A Mixed Workload test runs for a specified time at each predetermined user loads level. Each user will run a
transaction mix based upon the weights defined on the transactions tab. For example, if a test has two transactions,
A and B, with A having a weight of one and B having a weight of four, on average B will run four times for every
time A is run once. The run order will be randomly generated for each user so they are not all running the same
transaction simultaneously. That run order is used for that user each time the test is performed to ensure
reproducible results.
Goal test
A goal test is used to find maximum throughput or response time goals. A transaction mix is executed at user load
levels, determined by setting a beginning, ending, and interval value. When run, the specified goal criterion is
evaluated at the end of each iteration and the test ends once the goal or maximum user load has been reached.
Scalability
A SQL Scalability test executes each transaction individually for each userload and timing period. For example, a
test has two transactions, A and B, and two userloads of 10 and 20, with an iteration length of one minute.
Transaction A would execute continually for one minute at userload 10, then B would do the same. Next A will run
at userload 20, followed again by test B, for a total time of 4 minutes.
Copying a load scenario creates a new load scenario containing all compatible settings, user
scenarios, and transactions from the original test.
To copy a Load Scenario to a Replay Test, Mix Test, or Goal test
1. Right-click the desired Load Scenario in the Script View. A drop-down displays.
2. Select the desired type of test you want to copy to.
3. The test creates and displays in the Script View.
113
How Do I...
Running SQL Preview

SQL Preview allows you to preview the results of individual SQL statements while editing a
SQL transaction.
To run SQL Preview
1. Double-click on a user transaction in the Script View.
2. Click the SQL Query icon
. The SQL statement displays.
3. Select the appropriate profile from the Profile drop-down list.
4. Type the appropriate maximum number of rows to preview into Maximum Returned, or
check the All box to preview all rows.
5. Click OK. The SQL Preview window displays.
6. Review the information.
7. Click Close.

Latencies are used to control the rate that transaction execute. Replace child latencies is used to
quickly change all of the latencies of the direct children on a node. For example, when an
Oracle trace file is imported, the latencies are automatically set to replay the transactions at
their original rate, but changing these to a set value can replay the transactions in a shorter
period of time.
To replace Child Latencies
1. Right-lick on the desired load scenario. A drop-down displays.
2. Select Replace Child Latencies. The Replace Child Latencies Dialog displays.

How Do I...
3. Select the desired latencies.
4. Click OK. The latencies replace the previously set latencies for the children of the
selected Load Scenario or User Scenario.
Note: The latencies grandchildren of the selected node will not be changed.
114
115
How Do I...

Goal tests are used to determine the maximum transaction per second (TPS) or userload of a
system. When performing a goal test, Benchmark Factory provides the following options:
Find maximum TPS: After each iteration completes, the TPS value is compared with the
previous two testing iterations. If the TPS value has dropped for the last two values, then the
maximum TPS has been discovered and the test stops. The maximum TPS is dictated by the
users system. Benchmark Factory submits the workload to the server and measures the TPS that
the server is experiencing for that workload. The users system dictates how fast the workload
will be process by the database server.
Find maximum userload where response time is less than [n] ms: This option allows you to
execute the transaction mix at each userload until the response time for an iteration is less than
the entered value.
Overview
This quick start presents a load testing methodology to run a goal test that finds the maximum
Transactions Per Second (TPS) of a database, by adding a SQL Statement Transactions to the
goal test, then viewing real-time and stored testing results to determine if your database lives up
to the required performance parameters.
A Goal Test uses a transaction mix A transaction is a single unit of work in the testing process,
used in load scenariosand user scenarios. For example, a SQL statement. based on userload
intervals. The userloads are determined by setting a beginning, ending value, and interval value.
This value specifies an interval to increase the number of users tested for each iteration of the
Goal Test. When run, the specified goal criterion is evaluated at the end of each iteration, and
the test ends once the goal or maxim user load has been reached.

l
Creating a blank goal testing script
Set script timing
Add a transaction to the goal test
Create Benchmark Objects
Submit the job to run
View Realtime Results
View Run Reports Results
Saving the Script
116
How Do I...
Creating a Blank Goal Testing Script

1. Start Benchmark Factory if required.
To start Benchmark Factory
a. Click the Start button from the Windows task bar.
b. Select Programs Quest Software Benchmark Factory for Databases
Benchmark Factory for Databases. Benchmark Factory launches. The Run Quick
Start Dialog displays. Launch the Quick Start if desired.
2. Select the Script tab in the Script View.
3. Select Wizards | Load Scenario. The Load Scenario Wizard displays. The Load Scenario
Wizard is the starting point for all Benchmark Factory scripts. The Load Scenario Wizard
walks you through the steps to build a complete load testing script.
4. Click Next. The Select Load Scenario dialog displays.
5. Click the Create a custom load scenario (Scalability, Goal...).
6. Click Next. The Test Type dialog displays.
7. Click the Goal radio button.
8. Click Next. The Goal Page displays. The Goal page allows you to:
l
Find maximum TPS: Selecting this option will show how the maximum TPS can
be reached during a load test.
Find maximum userload where response time is less than [n]ms: Selecting this
option shows how a maximum userload can be reached when response time is less
than a keyed in value.
117
How Do I...
10. Select the desired option. For this quick start, the Find maximum TPS is selected.
11. Set the desired Start, Ending, and Interval in Userloads. For this test, the defaults values of
1, 5, and 1 are used. When run, User Loads of 1, 2, 3, 4, 5 are run until the TPS of one
run is less than the TPS of the previous run.
12. Click Next. The Define Transaction Mix Page displays.
13. Click Next. The Enter Script Name dialog displays and provides the following options:
2. Create a new Benchmark Factory Script file-Selecting this option allows you to
create and save a completely new script.
3. Add to existing Benchmark Factory Script file-Selecting this option allows you
to add the load scenario you are building to the current script file you are
working with.
14. Click Create a new Benchmark Factory Script file.
15. Click Next. The wizard completion dialog displays.
16. Click Finish. The Scripts creates and displays in the Script View.
Setting Script Timing
1. Click the Max TPS (1-5 by 1) node.
118
How Do I...
2. Click the Timing tab.

3. Enter the desired test length in Total Test Length, for example, 1 minute.
4. Move the slider bar to select the Pre-Sampling and Sampling times . ThePre-sampling time is the length of time
virtual users execute transactions A single unit of work in the testing process. For example, a Web page retrieval, a
SQL statement, or an email message. These are the components used to build a load scenario. in order to reach a
steady state before gathering statistics. Sampling time is the length of time used to collect statistics while virtual
users are executing transactions.
5. Click the Start all users as quickly as possible radio button.

Adding a Transaction to the Goal Test
119
How Do I...
2. Select Wizards | Transaction. The Transaction Wizard displays.
3. Click Next. The Define Transaction dialog displays.

4. Select Create SQL Statement Transaction.
5. Click Next. The SQL Statement dialog displays.
6. Enter the desired SQL statement. In this quick start, select * from a_updates where akey
= $BFRand(30) is entered. As you type a SQL statement, the $BFScript Wizard displays
and auto enters a BFScript.

8. Select Keying Time.
9. From the Keying Time drop-down select Normal.
120
How Do I...
10. In Mean enter 3000 milliseconds.

12. Click Finish. The transaction creates and displays in the Script View.
Creating Benchmark Objects
1. In order to create the a_updates table used in the SQL statement, Benchmark Objects must
be created. Click the Max TPS (1-5 by 1) job node.
2. Right-click. A drop-down displays.
3. Select Add Create/Delete Benchmark Objects. The Benchmark Objects Wizard displays.
121
How Do I...
4. Click Next.
5. From the drop-down, select the AS3AP standard benchmark.
6. Click Next. The Benchmark Scale displays.
8. Click Next. The Create/Delete benchmark objects dialog displays.
10. Click Finish.
11. You must create the benchmark objects before running the script. Right-click Create
Objects for AS3AP.
12. Select Move Up.
13. Move the Benchmark Object to the top node in the script view.

How Do I...
Submitting the job to run

1. Click the Max TPS(1-5 by 1) job node.
2. Select Wizards | Submit Job Wizard. The Submit Job dialog displays.
following options:
Section
Description
Profile
Displays the Profile Creation Wizard.
122
123
How Do I...
Section
Description
Displays the Profile setting dialog.
Actions
Allows you to:

Run Job: Runs the job.
No action: Closes the dialog after you select Submit.
Comments
Alerts
Allows you to send an email, pager, or net-send operator

alerts.
Job Counters
Displays the Job Counter dialog.
Stop job after first

error
Stops the job if an error is encountered.
3. Fill in the required parameters.

4. Click Submit. The job runs and displays in the Jobs Queue.
Viewing Realtime Results
1. Click the Message node. The Message node displays Job status, error messages, and
general information.
2. Expand the Run Status node. Run Status provides a real-time status on the test currently
running in the Job's Queue.
3. Click the Realtime Statistics node. The Real Time Statistics provides you with real-time
graphs and raw data. This data allows you to spot system-under test issues that may be
affecting server performance. The Real Time Statics node displays real-time counter data.
124
How Do I...
4. Click the Summary node. The Summary Graph displays. The Summary page displays user
selected statistics for the job timing period following completion of an iteration.
5. Click the Transaction Graph node. The Summary Graph displays. The transaction node
displays properties based on transaction type.
Viewing Run Reports Results
1. When the job completes, click the Run Reports node.
2. Select the desired run.

3. Double-click. The Run Reports opens to the Run ID node. This view contains an
summary of load scenarios statistics.
4. Expand the Summary Graphs node. Summary graphs provide a wide array of graphs that
allow you to view in detail your testing results.
5. Expand the Results node. Results provide information on individual user load and agent
load testing activity.
6. Close Run Reports when finished reviewing the load scenario.
Saving the Script
1. Click the Script tab.
2. Select File | Save. The Save As dialog displays.
3. Enter the desired script name in File Name.
4. Click Save.

Global Search and Replace allows you to search for and replace text from the selected item and
all of its children.
To use Global search and replace
1. In the Script View, right-click the User Scenario, Load Scenario, or Transaction node
you wish to search and replace.
2. Select Global/Search and Replace. The Global Search and Replace dialog displays.
125
How Do I...
3. Enter the desired parameter in Find What.

4. Enter the desired parameter in Replace With.
5. Check Match Case or Match Whole word if desired.
6. Click Replace.
Creating an ODBC Trace File

To create an ODBC trace file
1. Click the Start button from the Windows taskbar.
2. Select Settings | Control Panel. The Control Panel window displays.
3. Double-click on Administrative Tools. The Administrative Tools window displays.
4. Launch Data Source (ODBC). The ODBC Data Source Administrator window displays.
5. Click the Tracing tab.
6. Click the Start Tracing Now button. The Start Tracing Now button changes to Stop
Tracing Now button.
7. Run the ODBC transactions to be traced.
8. When the transactions complete, click on the Stop Tracing Now button.
9. Click OK.

Overview
An Oracle Real Application Cluster (RAC) combines several independent servers/nodes into one
database system. RAC is a clustered database solution that provides two major functions:
scalability and availability to a business continuum. Scalability is a relative term, based on a
simple rule that as the number of users accessing the system increases, the RAC configuration
should be able to handle increased activity. Availability is the ability of the system to provide
continuous service when one or more of the components in the cluster fail.
Two important criteria used to achieve these goals focus on the cluster being able to load
balance across all nodes, and when one or more nodes in the cluster fails, the users failover.
126
How Do I...
When a node in an Oracle RAC cluster fails, an Oracle database and the remaining nodes
continue to run, providing uninterrupted service to users. Oracle RAC distributes database
demands across several nodes, performing the function of load balancing. When more capacity is
required, additional nodes can be added with minimal effort. Organizations spending time and
money setting up a RAC cluster must ensure their systems have the stability to withstand user
environment demands when adding nodes. An Oracle Real Application Cluster (RAC) combines
several independent servers/nodes into one database system. RAC is a clustered database solution
that provides two major functions: scalability and availability to a business continuum.
Scalability is a relative term, based on a simple rule that as the number of users accessing the
system increases, the RAC configuration should be able to handle increased activity. Availability
is the ability of the system to provide continuous service when one or more of the components in
the cluster fail.
environment demands when adding nodes.

This Quick Start presents a load testing methodology and steps required to implement, manage,
and perform full diagnostic capabilities of the Oracle RAC cluster, in order to obtain optimum
user service times regardless of the demands placed on the RAC environment.

l
Setting up the tnsnames.ora file
Creating the Benchmark Factory cluster profile
Running the RAC Load Test
Importing an Oracle Trace File
Viewing RAC Testing Results

Navigate to the tnsnames.ora file located in Oracle-Home\network\admin. Transparent Network
Substrate (TNS) is Oracle's networking architecture. TNS provides a standard application
interface that enables network applications to transparently access underlying network protocols.
Open the file in a text editor. Edit the tnsnames.ora file to set up clustering connections. For the
first entry, the Net Service Name, in this example MyTNS, is used to define the root Net Service
Name. The root Net Service Name is not used when running a test. The remaining Net Service
Names must be named with the root and a number. In this example, MyTNS is the root and
MyTNS1, MyTNS2, and MyTNS3 will be used when the test is run. The "MyTNS" and
127
How Do I...
"MyTNS1" connections are duplicates and are required for Benchmark Factory to perform load
testing. MyTNS2 and MyTNS3 are for the other two machines in the cluster. Figure 1
provides an excerpt from a tnsnames.ora file set up for a three node cluster.
Figure 1-tnsnames.ora file set up for a three node cluster.

Creating the Benchmark Factory Clustering Profile
The profile contains information required to connect to the system-under-test, such as server
name, IP address, and user name.
128
How Do I...
To create the Benchmark Factory Clustering profile

1. Start Benchmark Factory if required .
b. Select Programs Quest Software Benchmark Factory for Database Benchmark Factory for
Databases 5.5. Benchmark Factory launches. The Run Quick Start Dialog displays. Launch the Quick
Start if desired.
3. Click Wizards | Load Scenario.
4. Click Next. The Load Scenario dialog displays.
5. Click Validate Oracle RAC Cluster Scalability.
6. Click Next. The Profile Selection dialog displays.
7. Click New. The Profile Creation Wizard displays.
8. Click Next. The driver selection dialog displays.
9. From the driver drop-down, select Oracle.
10. Click Next. The Connection Information dialog displays.
12. Click Next. The Profile Name dialog displays.

15. Click Finish. The Profile Selection dialog displays.
16. Click Next. The Edit Scale dialog displays.
17. Click the TPC-C Benchmark Edit button. The Benchmark 'TPC-C' Properties page displays.
18. Select the desired Benchmark Scale. The larger the scale selection, the larger the database that will when created.
It is suggested that a scale of 1 be selected, unless you desire a larger database to test with.
Note: Clicking the Advanced tab allows you to map benchmark objects. For these
procedures, please see Advanced Creation Objects (Object Mapping).
129
How Do I...
19. Click OK.

20. Click the TPC-H Benchmark Edit button.
21. Select the desired benchmark scale. The TPC-H benchmark creates large testing databases. A scale of 1
is suggested.
22. Click OK.
23. Click Next. The Node Selection dialog displays.
24. Enter the desired number of nodes in Nodes. In this example, 10 nodes are selected.
25. Under Method, click the Benchmark Factory radio button.
26. Click Next. The Measurement Intervals dialog displays.
27. Enter the desired User Load.
l
Create a new Benchmark Factory Script file-Selecting this option allows you to
Add to existing Benchmark Factory Script file-Selecting this option allows you to
add the load scenario you are building to the current script file you are working with.
29. Select Create a new Benchmark Factory script.

32. Click Finish. The script creates and displays in the Script View.

Benchmark Factory allows you to replay production workload and SQL scripts in different
database environments, and places enormous stress on a database. By replaying production
activity in a test database, potential performance problems can be identified before occurring in
production. A system typically breaks under extreme load. By identifying system capacity and
performance bottlenecks before they occur, Benchmark Factory facilitates proactive testing,
130
How Do I...
which in turn reduces downtime, development costs, and potential loss of revenue.
Benchmark Factory allows you to import a variety of user loads that include:
l
Industry standard benchmark to measure the performance of a system-under- test.

Benchmark Factory provides the AS3AP, Scalable Hardware, TPC-B, TPC-C, TPC-D, and
TPC-H standard benchmarks.
Oracle trace files with bind variables.
Imports from various tools such as Quest Performance Analysis SQL from the repository
for replay.

After creating the RAC script, you can enhance your script by importing an Oracle trace file
workload by selecting the trace file, applying filtering options, and then replaying the file. When
Benchmark Factory imports the Oracle trace activity it maintains concurrency between
transactions and inter-transaction timings that simulate a production environment. Activity as it
happened in production can be simulated in a test database using an unlimited number of
concurrent users.
Note: To create an Oracle trace file which includes bind parameters, a level number of 4 must be
used. For example, to start an Oracle trace for a user session, use the following command:
ALTER SESSION SET EVENTS '10046 trace name context forever, level 4'
Up to this point, you have created the clustering profile and a script that contains the TPC-C and
TPC-H benchmarks. Next you import trace files required to run the job.
1. In the script created, select the Script node. This allows the Oracle trace file import load
scenario to be placed in its own job.
2. Select Wizards|Load Scenario.

3. Click Next.
131
How Do I...
4. Click Replay load from an Oracle Trace file.

5. Click Next. The Oracle 8i/9i/10g/11g Trace Input dialog displays.
6. Click on the Add Trace button. The Select File Location window displays.
7. Navigate to and select the appropriate trace file(s).
8. Click Open. The trace file displays under the Trace File Name column.
9. Repeat steps 6 8 until all trace files are added.
10. Click Next. The Oracle Trace Activity window displays.
11. Click the appropriate Date Range radio button.
12. If appropriate, select the "From" and "To" dates and times.
13. Uncheck the Include system activity (SYS USER) box to include system activity.
Note: It is recommended to not check the Include system activity (SYS USER) check box.
This imports Oracle SQL statements routinely run by Oracle for maintenance reasons and
is not required for the load testing process.
create a and save a completely new script.
16. Add to existing Benchmark Factory Script file-Selecting this option allows you to
15. Check the Add to existing Benchmark Factory Script file radio button.
17. Click Finish. The trace files create and display in the Script View.
132
How Do I...
Running the Job

Since the Benchmark Factory node balancing method was selected in the RAC wizard, each job
has been assigned a profile specifically designed to test specific nodes by adding them to the
testing process. Benchmark Factory sets up a load test with a single instance and individual jobs
for groups of nodes as they are added to the testing process. This allows you to first run the
single instance node and gather results. Then you can tune that nodes and make the same
changes to your other nodes. Next you can add your nodes to the testing process in small groups,
allowing you to easily see if issues occur due to the additions of a group of nodes.
1. Select the Single Instance node.

3. Select Submit Job. The Submit Job Dialog displays.
133
How Do I...
following options:
Section
Description
Profile
Actions
Allows you to:

Comments
134
How Do I...
Section
Description
Alerts

operator alerts.
Job Counters

error

6. To run the jobs designed to test the remaining nodes created for this test, repeat steps 1-6.
Benchmark Factory provides a wide array of graphs to interpret your RAC testing results.
Individual nodes testing results can be viewed for comparison.
1. Select Tools Run Reports. Run Reports displays.
135
How Do I...
2. You can select a single test or multiple test to view testing results. In this example,
multiple tests are selected to view the RAC testing results.
3. Click Show Test Results. Run Reports Displays.
4. To view RAC testing results for the nodes selected, click Cluster Performance Graph.
5. To view other testing results, click the desired result from the User Load drop-down.
136
How Do I...

Benchmark Factory allows you to save jobs in the job queue back to the Script View from the
Jobs Queue. This provides an easy way to make changes to existing load tests.
To save the selected job to the Script View
1. Select the desired job in the Jobs Queue.
3. Select Save to Script.
4. The job is placed in the Script View.
Schedule A Job
Benchmark Factory allows you to run a job immediately after creation or schedule a job to run at
a future time. Scheduling allows you to set predetermined dates, recurrences, daily frequencies,
and durations of jobs.
Note: When using the recurring option, use Ready Job in place of Run Job.
Click here for definitions on scheduling parameters.
To schedule a job
1. Click the Job node.
2. Click the Schedule tab.
3. Check the Enabling check box.
4. Enter the desired date you want to start the job in On date.
5. Enter the desired time in At time.
6. If you wish only to run the job once, submit the job. If you wish to run the job on a
recurring basis, proceed to step 7.
137
How Do I...
7. Check the Recurring check box.

8. Click the desired radio button in Occurs.
9. Set the daily frequency in Every.
10. Enter the desired daily occurrence.
11. Click the desired Daily Frequency radio button.
12. Enter the desired parameters.
13. Click the desired duration radio button.
14. Enter the desired parameters.
15. Submit the Job if it is in the Script View or Ready the job if it is in the Jobs Queue by
clicking the Ready Job button on the Jobs Toolbar.
About Latency
Latency is the speed in which transactions are sent to a server or the period between receiving
results of one transaction and sending the next transaction to a serve. Benchmark Factory allows
you to select the following latency models and distribution models:
Latency Types
Distribution Models
No Delay (page 84)
Interarrival Time (page
138
How Do I...
Latency Types
Distribution Models
85)
Keying Time (page 85)
Negative Exponential Distribution (page

86)
Think Time (page 85)

Using the maximum number of agents on an Agent provides better utilization of machine
configuration. For example, you can run one machine with ten users and another with 100 users.
This allows light weight machines to be used with more powerful machines.
You can set the maximum number of users on an Agent using one of the following procedures:
l
From Agent Settings
From the Agent View in Benchmark Factory
To set the number of users from Agent Settings

1. On the Agent menu select Options|Settings. The Agent Settings dialog displays.
2. Enter the desired number of users in Max. Virtual Users.
139
How Do I...
3. Click OK.
To set the number of users from the Agent View in the Benchmark Factory Console
1. Right-click on the Agent in the Agent view of Benchmark Factory. A drop-down
displays.
2. Select Settings. The Agent Settings dialog displays.
3. Enter the desired number of users in Max. Virtual Users.

4. Click OK.
140
How Do I...

When running a load test, you have the option of staggering user startup. This allows you to
adjust startup loads if, for example, you are having issues with overloading of servers, when all
of the users attempt to start at the same time.
To stagger user startup
1. Click the Load Scenario node.
3. In the User Startup parameters select the desired parameters.
Start all users as quickly as possible: Starts all users immediately after a test begins.
Start all users at even intervals for: Sets the length of time to start the users in. The
value divided by the number of new users of new users to start in the iteration,
determines how often a user is started.
Start a new user every: Starts a new user, then waits the entered number of seconds
before starting the next user.
Running Benchmark Factory with Multiple Agents

To configure Benchmark Factory for Databases with multiple Agents you must:
l
Load the proper client libraries (i.e. Oracle, SQL server, etc) on the Benchmark Factory
Console and Agent Machine.
Install Benchmark Factory for Databases on the console and agent machines.
Benchmark Factory Console

Benchmark Factory allows you to run multiple Agents for load testing. There is no required
setup for the Agents on the Benchmark Factory Console. You view the Agents results as they
run from the Benchmark Factory Console.
141
How Do I...
To view the Agents from the Benchmark Factory Console:

1. Click Start | All Programs | Benchmark Factory for Databases | Benchmark Factory
for Databases.
2. Select View | Agent. The Agent view displays. Statistics from All Agents connected to
the Benchmark Factory Console will display here.
Benchmark Factory Agent

Each Benchmark Factory Agent must be configured with the address of the Benchmark Factory
Console. Each Agent sends load testing results back to the Benchmark Factory Console.
To configure Benchmark Factory Agents
1. Click Start | All Programs | Benchmark Factory for Databases | Agent.
2. Select Options | Settings.
3. In Machine Name/IP address, enter the address of the Benchmark Factory Console
Machine. For example, com123456.
4. Click OK.
5. Repeat this procedure for each Agent machine
Running a Job With Multiple Agents

To run a job with multiple agents
1. Create the desired job.
2. Start the agents and connect them to the console.
3. Run the job.
4. As the job runs, all connected Agents will display in the Agent View of the Benchmark
Factory Console.
5
Overview
Note: To create load scenarios, you must be in the Script View.
The Load Scenario Wizard is a starting point for creating Benchmark Factory scripts and walks
you through the steps to build a complete workload testing script.
Note: Only one Load Scenario can be run at a time.
From this wizard you can test:
l
Industry standard benchmark (TPC-C, TPC-E, TPC-H...) (See "Industry Standard

Benchmark (TPC-C, TPC-H...)" (page 192) for more information.)
Capture andReplay Oracle Workload See "Capture and Replay Oracle Workload" (page
Replay load from an Oracle trace file (See "Replay Load from an Oracle Trace File" (page
197) for more information.)
Validate Oracle RAC Cluster Scalability (See "Validate Oracle RAC Cluster Scalability"
Test Database Scalability (See "Test Database Scalability" (page 195) for more
information.)
Test SQL for Scalability (See "Test Database Scalability" (page 195) for more
information.)
Test Stored Procedures for Scalability (See "Test Stored Procedures for Scalability" (page
Capture and Replay SQL Server Workload (See Capture and Replay SQL Server
Workload for more information.
Replay load from SQL server trace table (See "Replay Load from SQL Server Trace Table"
Import from Foglight Performance Analysis (See "Import from Foglight Performance
Analysis Repository" (page 202) for more information.
Import from a Delimited Text file (See "Import from a Delimited Text File" (page 203) for
more information.)
Replay load from an ODBC trace file (See "Replay Load From an ODBC Trace File"
143
Import from Oracle Dynamic Performance View (See "Import from Oracle Dynamic
Performance View" (page 206) for more information.)
Import from SQL files (See "Import from Quest Tools Export Files" (page 207) for more
information.)
Create a custom load scenario (Scalability, Goal...) (See "Create a Custom Load Scenario
(Scalability, Goal...)" (page 208) for more information.)
6
Settings
About Settings
The Settings window displays default values used when creating new scripts. Changes to these
settings affect only new scripts, not existing scripts.
To Display Benchmark Factory Settings
l
Select Edit | Settings. The Settings dialog displays.
The Settings window provides the following dialogs:

l
General
Benchmarks
Timing
User Load
Latency
Repository
Agent
General Settings
The General settings dialog specifies workplace settings and error handling options.
145
Settings
Section
Field
Description
Workplace
Settings
Reload last script at

startup
Loads the last saved script file when

the program is launched.
Error Handling
Stop after first error
Default setting for a new jobs.

Terminates the test when a server error
occurs.
Directories
Script Files
The default location used when saving

scripts. These files usually reside on a
local drive, but may reside on a
network file server.
Error Logs
Location of error log files.
146
Settings
Section
Retrieval
Options
Field
Description
Data Files
Location of data files.
Allows you to limit the number of rows retrieved during SQL

transaction execution.
Benchmarks Settings
The Benchmarks dialog specifies benchmark properties and options.
Section
Benchmarks
Options/Icons
Description
Show Properties button.
Displays benchmark properties.
Benchmark Implementation
Show Benchmark DLL Properties icon.

Displays DLL information.
147
Settings
Timing Settings
The Timing dialog specifies timing phases associated with an item.
Note: Setting the timing phase sets the default value for a new load scenario.
Section
Field
Property Definition
Time Phase
Total Iteration Length
The composite of Pre-Sampling and

Sampling time.
Pre-Sampling
The length of time users execute

transactions in order to reach a steady state
148
Settings
Section
Field
Property Definition
before statistics are collected.
User Startup
Sampling
The length of time to collect statistics

while the users are executing transactions.
Start all users as quickly as

possible
Begins all users immediately when the test

begins.
Start all users at even

intervals for [time]
Starts the users at the set interval

determined they the set time divided by
the number of users to start.
Start a new user every [n]

seconds
Starts a new user at the set interval.
User Load (Measurement Intervals)

The User Load dialog specifies the number of virtual users for each iteration of a test.
149
Settings
Note: Setting the user load sets the default value for a new load scenario.
Section
Field/Icon
Description
Determines user load to add to the
selected list.
User Load(s)
Add Single User

Load
Adds the specified user load to add to

Selected User Load(s).
Add a Range of
User Loads
Adds a range of user loads to selected

user load(s).
150
Settings
Section
Field/Icon
Description
Start
Starting user load.
End
Ending user load.
Interval
Interval of the user load.
Adds a user load to the Selected Users

Load(s)
Selected User Load

(s)
Lists the selected user loads.
Removes the selected user load.
Removes all user loads.
LatencySettings
The Latency settings dialog specifies the speed in which transactions are sent to the server to model
real world user interactions.

Settings
Note:Setting the latency sets the default value for a new load scenario.
Latency Type:
Distribution Models
No Delay
Absolute
Interarrival Time
Uniform
Keying Time
Negative Exponential
Think Time
Normal
Poisson
151
152
Settings
Repository Settings
Note: If you create a new Benchmark Factory 5.5 repository, earlier versions of Benchmark
The repository is a database where all of the test results are stored. Benchmark Factory inserts test
results into the repository and provides an easy way to access all test results data. By default, the
Databases. The Repository can reside on another database server if required, to change the
database select the Data Source Name of the ODBC connection for the new database.
Repository settings allows you to edit DSN, perform ODBC administration, and test the
connection. Benchmark Factory also provides a Repository Manager and Data Repository
Migration wizard to assist you with repository management.
Note: If the database structure does not exist on the selected database, a prompt to create the
structure will appear when OK is clicked.
The supported database types are:
l
MySQL
SQL Server
Sybase Adaptive Server Anywhere (iAnywhere)
Oracle
153
Settings
Section
Connection
Parameters
Field/Icon
Description
Data Source
Name
Data Source name of the ODBC

connection used to connect to the
repository database.
User Name
The User Name used to log into the

selected database.
Password
The Password associated with the user

name used to log into the database.
Displays the ODBC connection

information dialog for the selected data
source.
154
Settings
Section
Field/Icon
Description
Displays the ODBC Data Source

Administrator dialog, used to add and
edit ODBC connections.
Tests the connection of the currently

selected ODBC Data Source.
Agent Settings
The Agent Settings tab allows you to manage Agent startup, shutdown, and maximum virtual
user per agent settings.
Section
Field
Description
Agent
Management
Start agents
automatically
Automatically starts Agent when

Benchmark Factory begins to run a
test.
Shutdown agents
automatically
Automatically shuts down Agent after

Benchmark Factory finishes running a
test.
155
Settings
Section
Field
Description
Maximum virtual
users per agent:
Allows you to set the maximum

number of virtual users per agent.
For example, if you are running a load
scenario of 1200, and the maximum
virtual user per agent is 400, three
agents will automatically start.
7
Profiles
Overview
Profiles are sets of information that Benchmark Factory uses to communicate with the systemunder-test. This information may include, server name, IP address, and user name. You create
profiles using the Profile Creation Wizard.
Edit Profiles
Benchmark Factory Profiles can be created, deleted, and changed to meet script requirements.
Click here for procedures on editing profiles.

Profiles
Buttons
Description
Creates a new profile using the Profile Creation Wizard.
Deletes the highlighted profile
Displays the profile properties.
157
8
Drivers
Driver Selection
Drivers connect Benchmark Factory to the respective database. Benchmark Factory provides the
following database drivers:
l
IBM DB2CLI
Microsoft SQL Server (Native)
Microsoft SQL Server (ODBC)
MySQL (ODBC)
ODBC
Oracle
Sybase
Adding BFScript to the Profile Page

You can add BFScripts to the Benchmark Factory Data Source Names, allowing you to have
individual agents test against a different server. BFScripts have been enabled on the User name
and Password fields.
To add BFScripts to the Data Source Name
1. Click inside Data Source Name.
2. Type the desired script name.
IBM DB2CLI
The IBM DB2CLI driver requires the following connection parameters:
Note: BFScripts have been enabled on the

User name and Password fields.
Database Alias:The name of the

database.
User Name:The name of the user

associated with the server.
Password:The password associated

with the user.
159
160
Microsoft SQL Server (Native)

Note: When using SQL Server 2005, the Microsoft SQL Server (ODBC) driver is the preferred
driver. Microsoft is no longer installing DB-Library for database client connectivity.
The Microsoft SQL Server driver requires the following connection parameters:

User Name and Password fields.
Server Name:The name or the IP

address of the Microsoft SQL server.
Database:The name of the database

to be tested.


with the user.
Note: The Microsoft SQL Server driver

features a trusted security option. Checking
the Use trusted security check box allows
you to create a profile without entering a
User Name or Password.
Microsoft SQL Server (ODBC)

Note: When using SQL Server 2005, the Microsoft SQL Server (ODBC) driver is the preferred
driver. Microsoft is no longer installing DB-Library for database client connectivity.
The Microsoft SQL Server (ODBC) driver requires the following connection parameters:
Note: BFScripts have been added to the

Data Source name field on the ODBC
Profile page. BFScripts have been enabled
on the User Name and Password fields.
l
Data Source Name: The name of the

MS SQL Server ODBC data source.
User Name: The name of the user

Password: The password associated

with the user.
MySQL (ODBC)
The MySQL ODBC Server driver requires the following connection parameters:

l
Data Source Name:The name of the

MySQL ODBC data source.


with the user.
MySQL ODBC
The MySQL ODBC Server driver requires the following connection parameters:

l
Data Source Name:The name of the

ODBC data source.
User Name:The user name


with the user.
Oracle
The Oracle driver requires the following connection parameters:
161

User name and Password fields.
User/Schema: System under test you

are connection to.
Password: Password for system you

are connecting to.
TNS Tab: Allows you to connect to

a database using the TNS names.
Direct Tab: Allows you to connect

to a database using Host, Port, Server
name or SID.
Connect as: Type of connection you

connect to the database with:
Normal, SYSDBA, or SYSOPER.
Connect Using: Provides a dropdown to select the data base type (for
example, Oracle XE or 11g) to
connect to.
Make this the BMF default home:

Sets this profile connection as the
default.
Verify connection before

proceeding: Verifies the profile is
properly connected to the database
before clicking Next.
Sybase
The Sybase driver requires the following connection parameters:
User Name and Password fields.
Server Name:The name or the IP

address of the Sybase server.
Database:The name of the Sybase

server database.


with the user.
162
163
Profile Properties
Overview
Profile properties provides the following tabs:
l
General
Options
Real-Time Counters
Statistics Tab
Clustering (Oracle only)
General Tab
The General Tab allows you to enter database parameters, installed client connection parameters,
and test profiles for connection.
164
Section
Field/Icon
Description
Parameters
User/Schema
Name of the user and schema for the

database you are logging into.
Password
Password for the databse.
TNS
Transparent Network Substrate name

allowing the database to
communicate across a network.
Direct
Installed
Clients
Test Profile
Connect as:
Alllows you to connect as the

system database administor,
Connect Using:
Allows you to select the type of

client you are connecting to.
Make this the BMF default

home
Clicking this checkbox defaults

Benchmark Factory to this client.
Test Profile
Confirms a good connection

between Benchmark Factory and the
database client.
165
Options Tab
The Profile Options tab allows you to adjust timeout and reconnect.
Section
Field/Icon
Description
Timeout
Time
The maximum amount of time

Benchmark Factory will try to log
on to the system-under-test. If this
amount of time is reached,
Benchmark Factory will return an
error.
Infinite Timeout
Prevents the logon to the systemunder-test from timing out and

returning an error.
Enable Reconnect
Enables Benchmark Factory to

attempt to reconnect to the systemunder-test if the connection is lost.
Reconnect
Section
166
Field/Icon
Description
Number of reconnect
attempts
The number of times to attempt to

reconnect before aborting.
Time to wait between

reconnect attempts (seconds)
How long to wait before attempting

to reconnect.
Profile Properties Real-Time Counters Tab

Real-time counters keep track of statistics during the running of a job. By default the following
real-time counters are recorded when running a job even though they are not displayed:
l
AVG_TIME
BPS
DEADLOCKS
TOTAL_ERRORS
MAX_TIME
RPS
TOTAL_BYTES
TOTAL_ROWS
TPS
USERLOAD
The Real-Time Counters tab displays on the profile and job property pages. The combination of these
settings are used to create a complete list of real-time counters that are recorded during test
execution. From this tab you can:
167
1. Add real-time counters to the profile

2. Delete real-time counters from the profile
To Add Real-Time Counters
1. Click the Add New Counter button
dialog displays.
. The Browse Performance Counters
2. Click the Select counters from list radio button.

3. Select the desired counter(s).
4. Click Add.
5. Click Close.
To delete Real-Time counters
1. From the Real-Time Counters tab Select the counter you wish to delete.
168
2. Click the Delete the highlighted counter button
169
Clustering Tab
Clustering is the process of using two or more computer systems that work together. Multiple
servers are linked to handle variable workloads or to provide continued operation in the event
one fails. Computers may be multiprocessor systems. A cluster of four computers with four CPUs
each, provides a total of sixteen CPUs processing at one time.
The Clustering tab in Benchmark Factory allows you to perform Oracle Real Application
(RAC)Testing.
Click here for procedures on how to conduct Oracle RAC testing.
170
9
Wizards
Overview
Benchmark Factory wizards streamline the load testing processing and provide the flexibility to
customize testing workloads.
The following wizards are provided:
l
Benchmark Objects
Execute File
Profile Creation
BFScript
Submit Job
Transaction
User Scenario

The Benchmark Objects Wizard creates or clears the database objects (tables/indexes, etc.)
needed to run industry standard benchmarks. The Benchmark Objects Wizard is able to run if
you have a root node job, load scenario, Create/Delete Benchmark object, or Execute File node
selected. The Create/Delete Benchmark object node will be added just below the item selected or
as a child of the job.
The Benchmark Objects Wizard:
l
Selects the benchmark to create objects for
Determines the scale of a benchmark
Creates an instruction to create the benchmark objects
172
Wizards
To create a Benchmark Object

Note: To create, delete, and modify database tables/indexes you must have the user login
rights required.
2. Select Wizards | Benchmark Objects.
3. Click Next.
4. From the drop-down, select the desired benchmark.
5. Click Next.
6. Select the desired benchmark scale in Benchmark Scale. When setting up a load test,
Benchmark Factory allows you to change the scaling factor of the standard benchmarks
provided. Scales factors increase the size of a database during the testing process allowing
realistic testing. By the scale factors, the number of rows added to the tables increases,
allowing for larger userloads to be used, placing a greater stress on the system under test.
During load testing, you can run various test with numerous scaling factors to ensure your
database performs at various performance levels.
7. The Advanced Creation Option allows you to set advanced properties for tablespaces and
datafiles, used for the creation of standard benchmark tables and indexes. To add mapping
options click Advanced Creation Option or proceed to step 18.

13. Click OK.
16. Click Save.
17. Click OK.
173
Wizards
18. Click Next. The Create/Delete benchmark objects dialog displays and provides the
following options:
l
Create Benchmark Objects: Creates and initializes benchmark objects (tables, indexes,
etc.) required to run the standard benchmark transactions.
Always delete objects before create: Deletes existing benchmark objects (tables,
indexes, etc.) before creating new ones.
Note: Checking the "Always delete objects before create" check box will delete all
benchmark objects in the selected database and then re-create the objects in the
databases. When the checkbox is unchecked, Benchmark Factory checks for the
benchmark tables. If they exist, then the creation will be skipped. The scale factor is
not checked.
Delete Benchmark Objects: Deletes existing benchmark objects (tables, indexes, etc.).

20. Click Next. The Benchmark Objects Wizard completion dialog displays.
21. Click Finish. The Benchmark Object node creates and displays in the Script View.

Overview
The Data Repository Migration Wizard allows you to migrate the Benchmark Factory testing
repository from one database type to another. For example, from a MySQL database to a
Microsoft SQL Server database.
To use the Data Repository Migration Wizard
1. Click the Start button from the Windows task bar.
2. Select All Programs | Quest Software | Benchmark Factory for Databases | Repository
| Repository Data Migration. (Go to step 3).
OR
3. Select All Programs | Quest Software|Benchmark Factory for Database | Repository |
Repository Manager. The Repository Manager displays.
4. Click Data Migration. The Data Migration Wizard displays.
174
Wizards
5. Click Next. The Choose a Data Source dialog displays.

6. From the Data Source Name drop-down, select the datasource type to migrate from.
7. Enter the user name and password.
8. Click Next. The Choose a Destination dialog displays.
9. From the Data Source Name drop-down, select the datasource type to migrate to.
10. Click Next. The Data Migration Wizard completion dialog displays.
11. Click Finish.
Execute File Wizard

Execute File Wizard
During the running of a workload, you may want to add an executable file (.bat or .exe) as part
of a job. This gives you the flexibility of setting up your database before running a workload.
For example, you could run a .bat file to set up a prepare SQL statement before running a
workload.
175
Wizards
To execute a file:
2. Select Wizards | Execute File.
3. Click Next. The Execute File dialog displays.
4. Enter the path or navigate to the desired file.
5. Click the Enforce Timeout checkbox if desired. Enforce Timeout allows you to enter the
desired number of seconds Benchmark Factory waits for the execution to complete before
continuing the job.
6. Enter the desired number of seconds.
8. Click Finish. The Execute File is created and displays in the Script View.

Overview
Note: To create load scenarios, you must be in the Script View.
The Load Scenario Wizard is a starting point for creating Benchmark Factory scripts and walks
you through the steps to build a complete workload testing script.
Note: Only one Load Scenario can be run at a time.
From this wizard you can test:
l
Industry standard benchmark (TPC-C, TPC-E, TPC-H...) (See "Industry Standard

Benchmark (TPC-C, TPC-H...)" (page 192) for more information.)
Capture andReplay Oracle Workload See "Capture and Replay Oracle Workload" (page
Replay load from an Oracle trace file (See "Replay Load from an Oracle Trace File" (page
Validate Oracle RAC Cluster Scalability (See "Validate Oracle RAC Cluster Scalability"
Test Database Scalability (See "Test Database Scalability" (page 195) for more
information.)
Test SQL for Scalability (See "Test Database Scalability" (page 195) for more
information.)
176
Wizards
Test Stored Procedures for Scalability (See "Test Stored Procedures for Scalability" (page
Capture and Replay SQL Server Workload (See Capture and Replay SQL Server
Workload for more information.
Replay load from SQL server trace table (See "Replay Load from SQL Server Trace Table"
Import from Foglight Performance Analysis (See "Import from Foglight Performance
Analysis Repository" (page 202) for more information.
Import from a Delimited Text file (See "Import from a Delimited Text File" (page 203) for
more information.)
Replay load from an ODBC trace file (See "Replay Load From an ODBC Trace File"
Import from Oracle Dynamic Performance View (See "Import from Oracle Dynamic
Performance View" (page 206) for more information.)
Import from SQL files (See "Import from Quest Tools Export Files" (page 207) for more
information.)
Create a custom load scenario (Scalability, Goal...) (See "Create a Custom Load Scenario
(Scalability, Goal...)" (page 208) for more information.)
Capture and Replay Oracle Workload

The Capture and Replay Oracle Workload wizard allows you to capture Oracle workloads, then
replay those workloads to view transactions and response times.
To capture andreplay an Oracle workload:
1. Select Wizards | Load Scenario.
or
Click New in the toolbar.
The Load ScenarioWizard displays.
2. Click Next.
3. Select Capture and Replay Oracle Workload.
4. Click Next. The Select or Capture Workload dialog allows you to previously captured
trace, delete a capture, or create the capture workload and execute it.

Wizards
5. Click the Capture button
The Choose Database dialog displays.

6. In this dialog you provide database connection details.
177

Wizards
The Choose Database dialog provides the following fields and buttons:
l
Home-Sets the desired database home, for example, 10g. This field is
not required.
Connect As-Allows you to connect as Normal, SYSDBA, or SYSOP.
User-User name of the database being connected to.
Password-Password of the database being connected to.
TNS-Allow you to connect to a database using an Oracle TNS connection.
Direct-Allows you to connect to a database using the host, port, and

service name or SID.
Test Connection-Click to test the database connection.
7. Click Next.
178
179
Wizards
TheSelect the Capture Method dialog displays. For details on which method to use, see
Pros and Cons of Using Trace or Fine Grain Analysis Files When Testing
withBenchmarkFactory.
Capture using Oracle Trace files-Captures workloads using trace files

written to a specified server directory. These files are captured at the
session level.
Capture using Fine Grained Auditing-Captures workloads using the

Oracle Fine Grained Auditing objects that capture activities at the database
table and view levels.
Tablespace for capture table-Allows you select the required tablespace for
Fine Grained Auditing captures.
Drop allBenchmark Factory objects checkbox-Clicking this checkbox

drops all captured database objects, leaving no footprint during the test.
8. Select the required collection method and click Next.

The Select BMF Capture and Replay dialog displays.
9. This dialog allows you to set the shared directories between the client and server.

Wizards
BMF Capture Directory (Accessed from Server)

l
The directory that the Oracle database server uses for its input and output
during workload captures and replays. This directory can be in one of
three locations:
o
On the database server as a local directory, but require that directory

to be reachable by the Benchmark Factory client PC (Windows
network share, Samba, or NFS mount).
On the Benchmark Factory client PC as a local directory, but

requires that directory to be reachable by the database server
(Windows network share, Samba, or NFS mount).
On the network file server as a local directory, but that requires

that directory to be reachable by both the database server and
the Benchmark Factory client PC (Windows network share,
Samba, or NFS mount). This is the recommended scenario for
the following reasons:
n
This minimizes total network I/O required for capture

and replay.
The shared directory can be placed on an I/O subsystem with

sufficient I/O bandwidth to handle both the concurrent I/O
writing and the cumulative size of the trace and export files.
180
181
Wizards
BMF Replay directory (Accessed from Client)-The directory that the

Oracle database server uses for its input and output during workload
captures and replays. This directory can be in one of three locations:
o
On the database server as a local directory, but requires that

directory to be reachable by the Benchmark Factory client PC
On the Benchmark Factory client PC as a local directory, but

requires that directory to be reachable by the database server
On the network file server as a local directory, but that requires that
directory to be reachable by both the database server and the
BenchmarkFactory client PC (Windows network share, Samba, or
NFS mount). This is the recommended scenario for two reasons:
n
This minimizes total network I/O required for capture

and replay.
The shared directory can be placed on an I/O subsystem with

sufficient I/O bandwidth to handle both the concurrent I/O
writing and cumulative size of the trace and export files.
Select the required settings.

10. Click Next.
The Capture Settings dialog displays.
11. This dialog allows you to set the capture name and select the performance collection and
reporting method.

Wizards
Capture/Project Name-The name assigned by Benchmark Factory to

templates and files used for export. You can customize this name to match
the test being run.
Performance Collection andReporting Methods-Provides the following

collection method types:
l
Statspack-Benchmark Factory calls the Oracles Stats Pack utility to

create a performance data repository, collect before and after workload
capture, replay Stats Pack snapshots of database wide performance statistics,
and then generates a different report between the capture and replay results.
Unchecking Statspack grays out the Password for user "PERFSTATso this
capture method cannot be used.
Note: "perfstat" is the default password.
AWR Report-Benchmark Factory uses the optional Oracle Enterprise

Manager (OEM) Diagnostic Pack Automatic Workload Repository
(AWR) to collect database wide performance statistics, collect before and
after workload capture and replay AWR snapshots of database wide
performance statistics, and generate a differences report between the capture
and replay results.
Note: A valid licenses is required to use the optional OEM
182

Wizards
Diagnostics Pack.
l
Drop all Benchmark Factory capture created database objects (one

trigger, one PL/SQL package and some tables)upon capture completion
(successful or not) thus leaving no footprint.-Checking this box drops all
objects created during the test.
Select the required capture method and click Next.

The Export Scope (Objects to Watch) dialog displays.
12. This dialog allows you to select schemas for export.
None/Skip-Use this option if:

o
You already have a backup plan.
Want to manually backup the database before capture.
The workload is a read-only and no backup of data is needed.
Entire database-Exports the entire database selected.
Selected schemas-Allows you select individual schemas for capture.

a. Click on the required schema under All Schemas.
b. Click
to move the required schema to Selected Schema.
183
184
Wizards
Include schemas that are related to the selected for export-Select this checkbox to
ensure any related schemas are included in the export.
13. Select the required options and click Next.

The Capture Scope (Sessions to Watch) dialog displays.
14. The Capture Scope dialog allows the capture of individual schemas or the entire database.
None/Skip-Does export any data from the selected schema.
Entire database-Captures all schemas for the entire database.
Selected schemas-Captures only the selected schemas you want to

capture data on.
a. Click on the required schema under All Schemas.
b. Click
to move the required schema to Selected Schema.
Include any schemas owning objects relate to those selected for exportChecking this box includes objects owned by the schema during the
export process.
15. Make the required selection and click Next.

The Capture Scope dialog displays.
16. This dialog allows you to set the capture scope for the selected schema.
185
Wizards
Entire database-Captures the entire database.
Selected Schemas-Allow you to select individual schemas for the

capture scenario.
Include activities from related schemas-Check this box if you want to

capture data from activities in related schemas.
17. Make the required selection and click Next.

The Filtered Settings dialog displays.
18. The Filtered Settings dialog allows you to add filters during the capture process. This step
is optional.
To add a filter:
a. Click Add.
b. Click on the Filter, Condition, or Value column to select the
required field.

Wizards
To remove a filter:
a. Click on the required filter.
b. Click Remove.
19. Make the required selections and click Next.
The Capture Control dialog displays.
186
187
Wizards
20. The Capture Control dialog provides the following options:

l
Immediately-Starts the capture immediately.
Scheduled (via database schedule)-Allows you to set a date and time to begin the
job and enter a description.
Capture Automatically Stop After-Allows you to stop the data capture in hours
and minutes.
21. Click Next.

The Submit Page dialog displays.
188
Wizards
The Submit Page provides a summary of captured/exported data. If a capture is already in

process or a capture was interrupted, the Manage Activity dialog displays. Close this
dialog after reviewing.
22. Click Submit.
The Login dialog displays. The testing workload begins to run and Capture Scenario
Status dialog displays.
23. This dialog provides details on the capture status.
189
Wizards
This dialog provides the following tabs:

l
Capture Details-Provides details of the scenario created and being run.
Active Traces-Provides trace details.
Session History-Provides a graph of the sessions times to complete.
24. After the capture completes, click Close.

The captured workload is selected and displays by default in the Load
Scenario Wizard.
25. The captured workload is selected and displays by default in the Load Scenario Wizard.
190
Wizards
26. Select the required Capture name and click Next.

The Select Target Server dialog displays.
27. The Select Target Server dialog provides the following options:
l
Test against source server (selected database name)-Replays workload against

the same server used to during capture.
Select a profile for the target server-Allows you to select a server to replay the
workload against.
28. Select the required option and click Next.

29. The Load Scenario Wizard completion dialog displays. To submit the job automatically
to the Benchmark Factory jobs queue, click Do you want the job to be automatically
submitted to the jobs Queue?

Wizards
30. Click Finish.

The Submit Job dialog displays
191
192
Wizards
Select the required options.

31. Click Submit. The test is submitted to the Jobs Queue and runs. After job completion, test
results are displayed.
32. To view full test results, click For full test results, click here.
Industry Standard Benchmark (TPC-C, TPC-H...)

This option creates load scenarios from industry standard benchmarks (AS3AP, Scalable
Hardware, TPC-C, TPC-B, TPC-D, and TPC-H). This includes the steps to create/load all the
required benchmark objects to execute the standard benchmark workloads. These synthetic
workloads can be used when no real-world data is available to be imported for database
load testing.
To create an Industry Standard Benchmark script
or
Click New. The Load Scenario Wizard welcome dialog displays.
2. Click Next. The Define load scenario dialog display.
3. Click Industry standard benchmark (TPC-C, TPC-H...).
4. Click Next. The Select Benchmark dialog displays.
193
Wizards
5. Select the desired benchmark: AS3AP, Scalable Hardware, TPC-B, TPC-C, TPC-D,
and TPC-H.
7. From the Profile drop-down, select the desired profile or click New display the Profile
Creation Wizard and create a profile.
8. Click Next. The Benchmark Scale dialog displays.
9. Select the desired Benchmark Scale.
10. Click the Advanced Creation Option button to map individual benchmark objects.
11. Click OK on the Advanced Creation Option page.
13. Select the desired user load.
17. If the Do you want the job to be automatically submitted to the jobs queue? checkbox
displays, then checking this will cause the Submit Job dialog to display when this
wizard completes.
18. Click Finish. The workload creates and displays in the Script View.
Validate Oracle RAC Cluster Scalability

This options creates load scenarios to validate an Oracle RAC cluster by creating a single
instance through multiple instance cluster load tests using OLTP, database warehousing, and a
mixed workload. This utilizes benchmark objects and transactions from the TPC-C and the TPCH benchmarks.
To validate Oracle RAC cluster scalability
or
Click New. The Load Scenario Wizard displays.
2. Click Next. The Define load scenario dialog display.
194
Wizards
5. From the Profile drop-down, select the desired profile or click New to display the Profile
Creation Wizard and create a new profile.
6. Click Next. The Edit Scale dialog displays. The Edit Scale dialog allow you to set the
scale and map the objects for the TPC-C and TPC-H benchmarks.
7. If you do not desired to edit either benchmark, click Next and proceed to step 23.
8. To edit the properties of a benchmark, click the Edit button for benchmark (TPC-C or
TPC-H) you wish to edit. The benchmark properties dialog displays.
9. Select the desired scale in Benchmark Scale.
10. To edit the advanced properties of the benchmark, select the Advanced tab. Here you
can map properties for tablespaces and datafiles, used for the creation of standard
benchmark tables and indexes.
11. Check the Use advanced options for creation of benchmarks objects check box.
16. Select the desired profile from Profile drop-down.
17. Select the desired mapping options.
18. Click OK.
21. Click Save.
22. Click OK. The Edit Scale dialog displays.
24. Enter the desired number of nodes in Enter maximum number of nodes (2-100).
25. Select the desired load balancing method.
27. Select the desired load user loads.
l

195
Wizards
wizard completes.
Test Database Scalability

Benchmark Factory allows you to test database scalability to ensure capacity and user demands
can be met. The Load Scenario Wizard walks you through this process.
To test database scalability
1. Select Wizards |Load Scenario.
or
2. Click Next. The Define load scenario dialog displays.
3. Click Test Database Scalability.
5. From the Profile drop-down, select the desired profile or click New to display the Profile
Creation Wizard and create a new profile.
6. Click Next. The Edit Scale dialog displays. The Edit Scale dialog allows you to choose
scales and advanced creation options for the TPC-C and TPC-H benchmarks.
7. If you do not desired to edit either benchmark, click Next and proceed to step 23.
8. To edit the properties of a benchmark, click the Edit button for benchmark you wish to
edit. The benchmark properties dialog displays.
9. Select the desired scale in Benchmark Scale.
10. To edit the advanced properties of the benchmark, select the Advanced tab. The advanced
creation option allows you to set advanced properties for tablespaces and datafiles, used
for the creation of standard benchmark tables and indexes.
11. Check the Use advanced options for creation of benchmarks objects check box.
16. Select the desired profile from the profile drop-down.
17. Select the desired mapping options.
18. Click OK.
196
Wizards

21. Click Save.
22. Click OK. The Edit Scale dialog displays.
wizard completes.
Test SQL For Scalability

The SQL Scalability test allows you to execute SQL statements, allowing users to spot potential
issues not seen with a single execution. Users can run variations of a SQL statement in order to
find the SQL that will perform the best under a load test.
To create a scalability test
or
2. Click Next. The Define Load Scenario dialog displays.
3. Select Test for SQL Scalability.
4. Click Next. The Define Iteration Length dialog displays.
5. When running a scalability test, you have two options for determining iteration length:
l
Number of executions per iteration-Each transaction is executed by each user for a

specified number of times (recommended).
Execution time per iteration-Executes each transaction for the specified length
of time.

8. You have three testing options at this point:
197
Wizards
1. You can click Next which takes you to step 12.

2. Add a User Scenario using the User Scenario Wizard by clicking the Add User Scenario.
3. Add transaction(s) using the Transaction Wizard by clicking Add a SQL Statement.
9. Select the desired option and follow the wizard steps.
10. After completing the chosen wizard, you are taken back to the Define Iteration
Length dialog.
11. Add another transaction or user scenario, or Click Next. The Measurements Interval
dialog displays.
12. Select the desired virtual user loads.
l

wizard completes.
Replay Load from an Oracle Trace File

The SQL contained within an Oracle Trace file can be imported into Benchmark Factory. When
this wizard completes, a User Scenario(s), with the timed event sequence of captured SQL
Transactions, will be placed into a Replay Load Scenario (workload). Importing an Oracle Trace
file allows you to replay your database activity with the same timing as was originally captured
on an existing in-house Oracle database application.
To create a import from an Oracle Trace File
or

Wizards
Click New.
The Load Scenario Wizard displays.
2. Click Next.
The Select Load Scenario dialog displays.

6. Navigate to and select the appropriate file(s).
198
199
Wizards

13. Click Next.
14. One of the following actions takes place:
1. If the trace import file exceeds the Benchmark Factory limit for displaying individual
transaction, the following dialog displays:
200
Wizards
If you click Yes, the trace import file import continues. Individual SQL will be
converted to XML files. You can then edited the XML files in the Benchmark Factory
Session Editor. Click the desired option.
b. If the trace import does not exceed the import limit for displaying individual transactions,
this dialog will not display.
15. The Enter Script Name dialog displays and provides the following options:
wizard completes.

Stored procedures enables developers who write stored procedures in Oracle and SQL Server to
test performance and scalability of the logic under varying user load levels. Benchmark Factory
allows you to import SQL from database stored procedures into a Mix Load Scenario (workload).
This gives you the ability to import SQL transactions, from stored procedures, for further testing
under load.
To test stored procedure for scalability
or
Click New. The Define load scenario dialog displays.
2. Click Next.
3. Click Test Stored Procedure for Scalability.
5. From the Profile drop-down, select the desired profile.
201
Wizards
6. Click Next. The Select Stored Procedures dialog displays. (Click here for stored
procedure examples.)
7. Check the desired procedures.
8. Click Next. If the selected procedures contain bind parameters, the Bind Parameters
dialog displays.
9. From SQL Name, select the desired SQL statement.
10. To add a parameter, right-click on the parameter. A drop-down displays.
11. Click Add.
12. Edit the bind parameter as needed.
14. Select the desired user loads.
wizard completes.
Replay Load from SQL Server Trace Table

Note: Refer to:
l
Creating a SQL 2000 Trace Table Using the SQL Server Profiler
Creating a SQL 2005 Trace Table Using the SQL Server Profiler ,or
Creating a SQL 2008/2008 R2 Trace Table Using the SQL Server Profiler
for procedures on creating trace files.

The SQL contained within a SQL Server trace table, generated by SQL Profiler 2000, 2005 or
2008/2008 R2, can be imported into Benchmark Factory. When the Load Scenario Wizard
completes a User Scenario, with the timed event sequence of captured SQL Transactions, it is
placed into a Replay Load Scenario (workload). Importing a SQL Server 2000/2005 trace table
allows you to replay your database activity with the same timing as was originally captured on
an existing in-house SQL Server database application.
202
Wizards
Note: To import a trace file, you will need to use SQL profiler to save the file as a trace table
and then import the table.
To replay a load from SQL Server:
or
3. Click Replay load from SQL Server trace table.
4. Click Next. The SQL Server Authentication window displays.
5. Select a host from the Select a host drop-down list or type in the host.
6. Select the desired connection method and fill in the required connection parameters.
7. Click Next. The SQL Server Trace Table dialog displays.
8. Select a database from the Select a Database drop-down list.
9. Select a table from the Select a Table drop-down list.
10. Click Next. The SQL Server Trace Processing window displays.
11. When the processing is complete, click Next. The Enter Script Name dialog displays and
provides the following options:
l

wizard completes.
Import from Foglight Performance Analysis Repository

Benchmark Factory allows you to import SQL from the Foglight Performance Analysis
repository. When the Load Scenario Wizard completes, a Mixed Workload containing the SQL
Transaction selected from the Performance Analysis Repository filters will be created. This gives
you the ability to import a real-world workload captured from an existing in-house database
application.
203
Wizards
To import from Foglight Performance Analysis Repository

or
3. Click Import from Foglight Performance Analysis Repository.
4. Click Next. The Performance Analysis Service Selection dialog display.
5. Select the desired service type from the drop-down list.
8. Click Next. The Foglight Performance Analysis dialog displays.
9. Click on Time Range link and edit the Time Range and select the desired timing filters.
10. Click OK.
11. Select the desired instance view and view filters.
13. Select the desired user load(s).
14. Click Next. The Enter Script dialog displays and provides the following options:
wizard completes.

The SQL contained within a delimited text file can be imported into Benchmark Factory. When
this wizard completes, a User Scenario, with the ordered sequence of captured SQL Transactions,
will be placed into a Replay Load Scenario (workload). Importing a delimited text files allow
204
Wizards
you to load test your database using workloads from an existing in-house generic database
applications.
To Import Delimited Text File
or
3. Click Import from a Delimited Text File.
4. Click Next. The Text File Input dialog displays.
5. Click Browse.
6. Navigate to and select the desired file.
7. Click Open.
8. Click Next. The Select file format dialog displays.
9. Click the desired format (Delimited or Fixed field).
1. If Fixed Field is selected, select the desired Row delimiter from the drop-down.
2. Select the desired rows to skip in Skip row.
or
1. If Delimited is selected, select the desired row delimiter, text qualifier, and rows to skip.
2. Click Next. The Specify Column Delimiter dialog displays.
3. Select the desired properties.
4. Click Next. The Fixed Field Column Positions dialog displays.
l

wizard completes.
205
Wizards
Replay Load From an ODBC Trace File

The SQL contained within an ODBC trace file, generated by the ODBC Data Source
Administrator, can be imported into Benchmark Factory. When the Load Scenario Wizard
completes a User Scenario(s), with the ordered sequence of captured SQL Transactions, it will be
placed into a Replay Load Scenario (workload). Importing an ODBC trace file allows you to
load test your database using workloads from an existing in-house ODBC database application.
Please click here for procedures on creating an ODBC trace file.
To Import from an ODBC trace file
or
2. Click Next.
3. Click Replay load from an ODBC trace file.
4. Click Next. The ODBC Trace Input dialog displays.
5. Type in or browse to the ODBC trace file to import.
6. Click Next. The ODBC Trace Processing window displays. Wait for the processing to
complete.
wizard completes.
206
Wizards
Import from Oracle Dynamic Performance View

Benchmark Factory allows you to import SQL from Oracle Dynamic Performance Views. When
the Load Scenario Wizard completes, a Mixed Workload containing the most often executed,
time consuming, or recently executed SQL Transaction will be created. This gives you the
ability to import SQL transaction identified by the Oracle Dynamic Performance Views for
further testing under a load.
To import an Oracle Dynamic Performance View
or
Click New.
3. Click Import from Oracle Dynamic Performance View.
4. Click Next. The Oracle DPV Connection dialog displays.
5. Enter the appropriate connection information.
6. Click Next. The Oracle DPV Import dialog displays.
7. Click the desire radio button.
8. Enter the number of desired SQL Statements in Maximum number of SQL statements
returned (100 max).
Note: It is recommended to not check the Include system activity (SYS USER) check box. This
imports Oracle SQL statements routinely run by Oracle for maintenance reasons, and is not
required for the load testing process.
10. Click Next. The Measurement Interval dialog displays.
l

wizard completes.
207
Wizards
Import from Quest Tools Export Files

Benchmark Factory allows you to import SQL files exported from other Quest Software products.
When the Load Scenario Wizard completes, transactions containing the exported SQL will be
created. This gives you the ability to import SQL transactions, identified by Quest Software
products, for further testing a under a load.
To import from SQL files
or
3. Click Import from SQL files.
4. Click Next. The SQL Tuning File Input dialog displays.
5. Click Browse.
6. Navigate to the desired tuning file.
7. Click Open.
l

wizard completes.
208
Wizards
Create a Custom Load Scenario (Scalability, Goal...)

Custom workloads allow you to create workloads from user provided SQL. These tests provide
flexibility for your load testing requirements. The following customs load scenarios are provided:
l
Mix Test-Runs a transaction mix based upon weights for a specified time at each
predetermined user load level.
Replay Test-Runs multiple transactions with each one running independently on a

specified number of users.
Goal Test-Used to find maximum throughput or response time values. A transaction

mix is executed at user load levels, determined by setting a beginning, ending, and
interval value.
Scalability Test-Compares the performance of SQL Statement variations under a

workload. Each transaction will execute individually for each user load and
timing period.
Mix Test
A Mixed Workload test runs for a specified time at each predetermined user load level. Each
user will run a transaction mix based upon the weights defined on the transactions tab. For
example, if a test has two transactions, A and B, with A having a weight of one and B having a
weight of four, on average B will run four times for every time A is run once. The run order will
be randomly generated for each user so they are not all running the same transaction
simultaneously. That run order is used for that user each time the test is performed to ensure
reproducible results.
To create a mix test
Or
3. Click Create a custom load scenario (Scalability, Goal, ...).
5. Click the Mix radio button.
6. Click Next. The Define Transaction Mix dialog displays. You have three testing options
at this point:
209
Wizards
1. You can click Next and create a blank mix test, and proceed to step 10.
2. Add a User Scenario using the User Scenario Wizard by clicking the Add User
Scenario button.
8. After completing the chosen wizard, you are taken back to the Define
Transaction Mix dialog.
9. Add another workload, or Click Next. The Measurements Interval dialog displays.
10. Select the desired virtual user load.
l

wizard completes.
Replay Test
A Replay Test runs multiple transactions with each one running independently on a specified
number of users. The test will run until the defined number of executions for each transaction or
a specified time limit is reached.
To create a replay test
1. Select Wizard | Load Scenario.
Or
5. Click the Replay radio button.
210
Wizards
6. Click Next. The Define Replay Items dialog displays.

7. Check the Execute by time check box if you want the test to run a specified time.
1. You can click Next and create a blank replay test, and proceed to step 11.
Scenario button.
10. After completing the chosen wizard, you are taken back to the Define Replay
Items dialog.
11. Add another workload, or Click Next. The Enter Script dialog displays and provides the
following options:
l

wizard completes.
Goal Test
Goal Test. When run, the specified goal criterion is evaluated at the end of each iteration and the
test ends once the goal or maxim user load has been reached.
To create a goal test
Or
211
Wizards

6. Click Next. The Goal Page dialog displays. The Goal tab provides the following options:
l
Find maximum userload where response time is less than [n] ms: Selecting this

9. Click Next. The Define Transaction Mix dialog displays.
1. You can click Next and create a blank test and proceed to step 13.
Scenario button.
12. Add another workload, or Click Next.
13. The Enter Script dialog displays and provides the following options:
wizard completes.
212
Wizards
Scalability Test
The SQL Scalability test allows you to execute SQL statements, allowing users to spot potential
issues not seen with a single execution. Users can run variations of a SQL statement in order to
find the SQL that will perform the best under a load test.
To create a scalability test
or
4. Click Next.
5. Click the Scalability radio button.
6. Click Next. The Define Iteration Length dialog displays.
7. When running a scalability test, you have two options during the running of the
workload:
l
Number of executions per iteration-Each transaction is executed by each user for a

specified number of times (recommended).
Execution time per iteration-Executes each transaction for the specified length
of time.

1. You can click Next and create a blank scalability test and proceed to Step 12.
Scenario button.
12. Add another workload, or Click Next. The Measurements Interval dialog displays.
13. Select the desired virtual user load(s).
213
Wizards

wizard completes.

Overview
Profiles are used by Benchmark Factory to connect to the system-under-test. The profile contains
information required to connect to the system-under-test, such as server name, IP address, and
user name. The Profile Creation Wizard walks you through steps required to create a Benchmark
Factory profile.
To create a profile
1. Select Wizards | Profile Creation.
2. Click Next.
3. From the Driver drop-down list, select the required driver type.
4. Click Next.
5. Enter the User/Schema.
6. Enter the Password.
7. From the TNSDriver drop-down, select the required database.
or
Click the Direct tab and enter the required fields.
8. From the Connect as drop, select the required connection, Normal, SYSDBA, or
SYSOPER.
9. Select the installed client if applicable.
214
Wizards
10. Click the Verify connection before proceeding check box to verify the connection to the
specified database.
11. Click Next.

12. Enter a profile name.
13. Click Finish.
BFScript
BFScript Wizard
Benchmark Factory scripting allows you to randomize the load testing process. Scripting can be
utilized by selecting the SQL Statement node and inserting the script in the properties view. A
field has scripting capabilities if its right-click menu contains the Script Wizard option.
The Script Wizard is a quick and easy way to use Benchmark Factory scripts. The Script Wizard
consists of a script category and a script function. Each script function has a short description
included and if applicable, its parameters.

Wizards
There are two features within Benchmark Factory that assist you when using scripting
capabilities:
l
Script Wizard
Script Assist
Click here for an overview of $BFScripts.

Script Wizard
1. Right-click the edit control. A drop-down display.
2. Select BFScript Wizard from the drop-down. The Script Wizard displays.
215
216
Wizards

4. Some scripts require further parameters settings. For a complete overview of individual
script settings, please refer to Scripts Overview.
If scripts require parameters:
1. Click Next.
5. Click Finish. The script and parameters display in the Script Wizard.
Script Assist
1. Type in the desired SQL statement. When $BF is typed the Script Assist is activated
allowing you to insert the desired script.
217
Wizards

3. Press Enter. The script displays.
4. Type in parameters as required.

Note: To run the SQL Scalability Wizard from Toad you must have Benchmark Factory loaded.
The Benchmark Factory console does not have to be open to run the SQL Scalability Wizard.
Overview
The SQL Scalability test allows you to execute SQL statements, letting users spot potential
issues not seen with a single execution. Users can run variations of a SQL statement generated
by SQL Tuning in order to find the SQL that will perform the best under a load test.
Creating a SQL Scalability load scenario requires the following steps.
1. Creating the SQL tuning connection
2. Entering the desired SQL statement
3. Running the statement using the in Benchmark Factory SQL Scalability Wizard
218
Wizards
Creating the SQL tuning connection

To create the SQL Turning connection
1. In Quest Central or Toad, connect to an Oracle or SQL Server database.
2. From the Quest Central main toolbar select Tools|SQL Tuning to launch.
Entering the desired SQL statement
To enter the desired SQL statement
1. Enter the desired SQL statement in SQL Text.
2. Click the Optimize Statement icon. Clicking this icon executes multiple syntax
transformation rules to produce a list of semantically equivalent SQL statements.
Running the SQL statement using the in Benchmark Factory SQL Scalability Wizard
1. Click the Benchmark Factory drop-down icon and select the desired option. Three options
are provided:
2. Test for Scalability-Tests the currently displayed SQL.
3. Test All for Scalability-Tests all SQL statements.
4. Test Selected for Scalability-Tests the selected SQL statements.
2. The Benchmark Factory SQL Scalability Wizard displays. Click Next. The Measurement
Interval dialog displays.
3. Enter the desired user load.
4. Click Next. The Iteration Length dialog displays.
5. Enter the desired number of executions per iterations or executions per iteration.
7. Select the desired latency.
8. Click Next. The connection information dialog displays.
9. Enter the connection parameters.
10. Click Next. The Connection Agent dialog displays. Benchmark Factory Agent(s) reside
on each physical client machine and spawn multiple virtual-user sessions. Each Agent
generates an information screen with tabs that contain a variety of Agent information.
Benchmark Factory can control hundreds of Agent machines.
11. If there are more than 20 users in your test, you will need to run an agent. Click Start
Local Agent.
12. Click Next. The Benchmark Factory SQL Wizard completion dialog displays.
13. Click Finish. The SQL Scalability dialog displays. From this dialog, you can control,
monitoring, and view SQL testing results. This dialog provides the following tabs
and buttons:
219
Wizards
Overview Tab
The Overview tabs provides transactions per second testing results for individual user loads and
iterations.
Transactions
The Transactions view provides the following statistics on the SQL statements being run:
l
Executions
Rows
Bytes
Errors
Average Transaction Time
Minimum Transaction Time
Maximum Transaction Time
220
Wizards
Real-Time
Real-Time Statistics provides real-time graphs and raw data. This data allows you to spot systemunder test issues that may be affecting server performance. Right-clicking inside the graph
displays a drop-down that allows you to change graph settings and view.
Summary
The Summary graph provides real-time and raw data summary graphs. The Summary graph
displays real-time counter data. Right-clicking inside the graph displays a drop-down that allows
you to change graph settings and view.

Wizards
Messages
The Output window displays messages about the job being run.
221
222
Wizards
Submit Job Dialog

Submit Job Dialog
A job contains load scenario, execute file, creation/deletion of benchmark objects, or a
combination of each. The Submit Job Wizard walks you through the process of adding a job to
the jobs queue. See "Jobs Queue View" (page 45) for more information.
To submit a job to the Jobs Queue
1. Select the Job node.
2. Right-click.
3. Select Submit Job. The Submit Job dialog displays.

Wizards
following options:
Section
Description
Profile
Displays the Profile Creation Wizard. See

"Profile Creation Wizard" (page 213) for more
information.
Displays the Profile setting dialog. See

"General Tab" (page 163) for more information.
Actions
Allows you to:

Run Job: Submits the job to the queue and runs the
job.
Run Job when queue becomes idle: Submits the job
to the jobs queue and puts it in a ready state so that
it will run when the currently ready or running job
completes.
223
224
Wizards
Section
Description
Run Job per schedule: Submits the next job

scheduled.
Start Agent and Run Job: Starts the agent and runs
the job.
Start Agent and Run Job when queue becomes
idle: Starts the agent and runs the job when all
previous jobs have finished.
Start Agent and Run Job per schedule: Starts the
agent and runs the next job scheduled.
No action: Closes the dialog after you select
Submit.
Comments
Alerts

operator alerts when the job completes.
Job Counters
Allows you to add Job Counters to the job.
Stop job after

first error
When checked, the job will stop if an error is

encountered.
4. After filling in the desired parameters, click Submit.
Transaction Wizard
Transaction Wizard
A transaction is a single unit of work in the testing process, used in load scenarios and user
scenarios. For example, a SQL statement. The Transaction Wizard walks you through the steps
required to create a transaction. From the Transaction Wizard you can:
225
Wizards
Create SQL Statement Transactions
Create Prepare SQL Statement Transaction
Create Execute prepared SQL Statement Transaction
Import Transactions

Stored procedures enables developers who write stored procedures in Oracle and SQL Server to
test performance and scalability of the logic under varying user load levels. Benchmark Factory
allows you to import SQL from database stored procedures into a Mix Load Scenario (workload).
This gives you the ability to import SQL transactions, from stored procedures, for further testing
under load.
To test stored procedure for scalability
or
Click New. The Define load scenario dialog displays.
2. Click Next.
3. Click Test Stored Procedure for Scalability.
5. From the Profile drop-down, select the desired profile.
6. Click Next. The Select Stored Procedures dialog displays. (Click here for stored
procedure examples.)
7. Check the desired procedures.
8. Click Next. If the selected procedures contain bind parameters, the Bind Parameters
dialog displays.
9. From SQL Name, select the desired SQL statement.
10. To add a parameter, right-click on the parameter. A drop-down displays.
11. Click Add.
12. Edit the bind parameter as needed.
226
Wizards
wizard completes.
Create SQL Statement Transaction

Create SQL statement transaction allows you to add a SQL statement to your workload.
To import a SQL statement transaction
1. Select a load scenario, user scenario, or transaction.
2. Click Create SQL Statement Transaction.
3. Click Next. the SQL Statement dialog displays.
4. Enter the desired SQL Statement.
or
Click the Script Wizard icon to Launch the BFScript Wizard.
5. After enter the desired SQL statement or completing the BFScript Wizard, Click Next.
The Real World Latencies dialog displays.
6. Select the desired latency type.
7. Enter the desired mean and value time.
9. Click Finish. The SQL statement creates and displays in the Script View.
227
Wizards
Create Prepared SQL Statement Transaction(s)

Prepared SQL statements can be executed during a load test by using an execute prepared SQL
Statement transaction. Bind variables can be added to prepared SQL statement transactions.
Execution of Prepared SQL Statements
In order to accommodate different testing needs, Prepared SQL Statements Transactions have the
ability to Prepare and/or execute the SQL. The execution of the transaction will differ depending
on the implementation method. The following provides an explanation of implementation
methods when running Prepared SQL Statements in Benchmark Factory.
l
When a Prepare SQL Statement transaction is contained within a User Scenario, it will
always prepare the SQL and, if Execute after prepare is checked (Figure 1) , it will
execute the SQL immediately after the preparation.
If the transaction is not within a User Scenario and Execute after prepare is not
checked, it will always prepare the SQL without executing.
If the transaction is not within a User Scenario and Execute after prepare is checked, the
statement will be prepared once for each user and immediately executed, and then
executed without a prepare each time after that point. This enables a Load Scenario to be
optimized for better real world simulation.
Figure 1-Execute after prepare checkbox
228
Wizards
The following procedure provides examples of:

l
Creating a prepared SQL statement transaction
Creating a prepared SQL statement transaction with a bind variable
To create a prepared SQL statement transaction

1. Click the Load Scenario or User Scenario node.
2. Select Wizards|Transaction. The Transaction Wizard displays.
4. Click Create Prepare SQL Statement Transaction.
6. Enter the desired SQL Statement.
or
7. After enter the desired SQL statement or completing the BFScript Wizard, Click Next.
The Real World Latencies dialog displays.
11. Click Next. The Transaction Wizard completion dialog displays.
To Create a prepared SQL statement transaction with a bind variable
1. Click the Load Scenario, User Scenario, or Transaction node.
2. Select Wizards|Transaction. The Transaction Wizard displays.
4. Click Create Prepare SQL Statement Transaction.
6. Enter the desired SQL Statement
or

Wizards
7. Right-click inside the parameter list control. A drop-down displays.
8. Select Add.
9. Right-click inside the type list control.
229

Wizards
10. Select the desired type from the list control.

11. Click inside Parameter.
12. Enter the desired Parameter.
13. Click inside of Value.
14. Enter the desired value.
15. Enter other bind variables as required.
230
231
Wizards

Create Execute Prepared SQL Statement Transaction

These transactions will run the SQL statement contained with the reference prepared SQL when
called. If the statement has not been prepared, an error will generate.
232
Wizards
To create SQL statement transaction

1. Click the Load Scenario or User Scenario node.
2. Select Wizards |Transaction Wizard. The Transaction Wizard displays.
3. Click Next. The Define transaction dialog displays.
4. Click Create Execute prepare SQL Statement Transaction.
6. Enter the name of the desired Prepare SQL Statement transaction.
8. Select the desired latency types(s) and enter the desired times.
Import Transactions
Import transactions allows you to import a transaction(s) from the AS3AP, Scalable Hardware,
TPC-B, TPC-C, TPC-D or TPC-H benchmark standard benchmarks to build workloads.
To import a transaction
1. Select Wizards | Transaction Wizard. The Transaction Wizard displays.
2. Click Next. The Define Load Transaction dialog displays.
3. Click Import Transaction.
4. Click Next. The Import Transactions dialog displays.
5. From the Benchmark drop-down list select the desired benchmark.
6. Select the desired transactions by clicking on the gray square in the first column.
9. Enter the desired mean or value time.
233
Wizards

The User Scenario Wizard allows you to build user scenarios. A user scenario is a series of one
or more transactions to be executed in order, normally associated with user behavior simulated
against the system-under-test. From the User Scenario Wizard you can:
l
Replay load from SQL Server trace table
Import from a delimited text file
Replay from an ODBC trace File
Create a custom user scenario

The SQL contained within an Oracle Trace file can be imported into Benchmark Factory. When
the Load Scenario Wizard completes, a User Scenario, with the timed event sequence of captured
SQL Transactions, will be placed into a Replay Load Scenario (workload). Importing an Oracle
Trace file allows you to replay your database activity with the same timing as was originally
captured on an existing in-house Oracle database application.
To import and replay an Oracle Trace File
1. Select a user scenario node and right-click.
2. Select New User Scenario. The User Scenario Wizard displays.
3. Click Next.
4. Select Replay load from an Oracle Trace file.
5. Click Next.
6. Click Add Trace.
7. Navigate to the trace file you want to import.
8. Select the file.
9. Click Open.
10. Click Next. The file processes.
234
Wizards
11. Click Next.

12. Click Finish. The user scenario(s) are created and displayed in the Script View.
Replay Load from a SQL Server Trace Table

The SQL contained within a SQL Server trace table, generated by SQL Profiler 2000 or 2005,
can be imported into Benchmark Factory. When the User Scenario Wizard completes, a User
Scenario, with the timed event sequence of captured SQL Transactions, it is placed into a Load
Scenario (workload) in your current script. Importing a SQL Server 2000/2005 trace table allows
you to replay your database activity with the same timing as was originally captured on an
existing in-house SQL Server 2000/2005 database application.
Note: For instructions on trace file/table creation, please see:
or
To Import from a SQL Server Trace Table
1. Select a load scenario, user scenario, or transaction node.
2. Click Next.
3. Select Wizards |User Scenario. The User Scenario welcome dialog displays.
4. Click Replay Load From SQL Server 2000/2005 trace table.
5. Click Next. The SQL Server 2000 Authentication window displays.
6. Select a host from the Select a host drop-down list.
7. Select the desired connection method and fill in the required connection parameters.
8. Click Next. The SQL Server 2000/2005 Trace Table dialog displays.
9. Select a database from the Select a Database drop-down list.
10. Select a table from the Select a Table drop-down list.
11. Click Next. The SQL Server 2000/2005 Trace Processing window displays.
12. Wait for the processing to complete.
14. Click Finish. The user scenario creates and displays in the Script View.
235
Wizards

The SQL contained within a Delimited Text file can be imported into Benchmark Factory.
When this wizard completes, a User Scenario, with the ordered sequence of captured SQL
Transactions, will be placed into a load scenario (workload) in your script. Importing a delimited
text files allow you to load test your database using workloads from an existing in-house generic
database applications.
To Import Delimited Text File
2. Select Wizards |User Scenario. The User Scenario welcome dialog displays.
3. Click Import from a Delimited Text File.
4. Click Next. The Text File Input dialog displays.
5. Click Browse.
6. Navigate to and select the desired file.
7. Click Open.
8. Click Next. The Select file format dialog displays.
9. Click the delimited or fixed field radio button.
10. Click Next. The Specify Column Delimiter or Fixed Field Column positions dialog
displays, depending on the selection on the previous page.
12. Select the desired SQL Column.
Replay Load from an ODBC Trace File

The SQL contained within an ODBC trace file, generated by the ODBC Data Source
Administrator, can be imported into Benchmark Factory. When this wizard completes a User
Scenario, with the ordered sequence of captured SQL Transactions, it will be placed into a load
scenario (workload) in your script. Importing an ODBC trace file allows you to load test your
database using workloads from an existing in-house ODBC database application.
To Import from an ODBC trace file
1. Select a Load Scenario, User Scenario, or Transaction.
2. Select Wizards|User Scenario. The User Scenario welcome dialog displays.
3. Click Next.
4. Click Replay Load from an ODBC trace file.
236
Wizards
5. Click Next. The ODBC Trace Input dialog displays.

6. Type in or browse to the ODBC trace file to import.
7. Click Open.
8. Click Next. The ODBC Trace Processing window displays. Wait for the processing
to complete.
Create a Custom User Scenario

Custom user scenarios allow you to create workloads from user provided SQL. These tests
provide flexibility for your load testing requirements.
To create a custom user scenario
2. Select Wizards |User Scenario. The User Scenario Wizard welcome dialog displays.
3. Click Next. The Define user scenario dialog displays.
4. Click Create a custom user scenario.
5. Click Next. The Define User Scenario dialog displays.
6. To add a SQL transaction to your user scenario, click Add SQL Statement. The
Transaction Wizard displays.
8. Select the desired transaction type.
9. Follow the wizard procedures.
10. When the wizard completes, the transaction displays in the Define User Scenario dialog.
11. Click the Add SQL Statement button to add another transaction.
or
Click Next. The Enter User Scenario Name dialog displays.
12. Enter the desire name.
10
Benchmark Factory TestingResults
Analysis is the ultimate goal of your load test. When you run a test, you get results both in realtime and in a saved format. Benchmark Factory provides you with testing results that are easy to
interpret and allows you to attribute individual results to individual tasks and users.
Whether you are viewing a real-time or historical graph, Benchmark Factory uses the same
graphing tool. In most cases you will find that the graphs are presented in the most meaningful
form. The Benchmark Factory graphing tool allows you to customize graphs to tailor your load
testing viewing requirements.
Benchmark Factory provides the following testing results types:
l
During the running of a workload (Real-time results)
After the workload completes (Run Reports)
Exported report information to Excel using the command line
Benchmark Factory Testing Results Terminology

The following provides a list of commonly used Benchmark Factory testing terms.
The following terms can be expressed as:
Average The average of all recorded values for the statistic over the sampling period.
Minimum The minimum value the statistic obtained over the sampling period.
Maximum The maximum value the statistic obtained over the sampling period.
90th Percentile This is usual associated with a timing statistic. This is the time value where
90 percent of all values recorded for a statistic fell below.
Term
Definition
Bytes
The number of bytes from data received from a SQL statement.
Bytes Per Second
The number of bytes processed per second over the sampling

period. This is just the Total Bytes divided by the Sampling
238
Term
Definition
period in seconds.
Response Time
The time it takes from when the SQL is sent to the server
responds.
Retrieval Time
The time it takes from the server responds to a SQL statement

till the last byte of data/results is obtained.
Rows
The number of rows received from a SQL statement.
Rows Per Second (RPS)
The number of rows received per second over the sampling

period. Similar to above.
Transactions Per Second

(TPS )
The transactions, or SQL statements, processed by the server

per second. A transaction in Benchmark Factory can be more
than a single SQL statement, such as the TPC-C transaction
New Order, this transaction inserts a new order by inserting
one record into the new order table and 5 7 items in the
orderline table.
Transaction Time
(sometimes listed as just
Time)
The sum of the Response and Retrieval time.
Exporting Test Results to Excel Using the

Command Line
Benchmark Factory allows you to export testing results to Excel using the Command line.
To export testing results from Benchmark Factory to Excel using the command line:
1. Select Start | All Programs | Accessories | Command Prompt.
2. Navigate to the Benchmark Factory for Databases bin directory.
Example: CD C:\Program Files\Quest Software\Benchmark Factory for
Databases\bin
239
3. Enter BMFRunHistory.exe x 34 88 108. Benchmark Factory testing results with Run

ids of 34, 88, and 108 export to Excel.
Real-Time Load Testing Results

Overview-Viewing Real-time Statistics Workloads
During the running of a database load test, real-time statistics provide insight into the
performance of the database being tested. Numerous datapoints allow you to examine exactly
how your database is performing. Benchmark Factory provides the following nodes to view realtime load testing statistics:
l
Run Status Node
Realtime Statistic
Summary
Transaction Graph
Change Graph Views

Right-clicking on a Benchmark Factory graph displays a drop-down that allows you to
customize graph settings. Whether you are viewing a real-time graph or one in Benchmark
Factory Run Reports, Benchmark Factory uses the same graphing tool. In most cases you will
find that the graphs are presented in the most meaningful form (Line Graphs or Bar Graphs based
on the data being presented.)
Displays Benchmark Factory Run Reports
Shows in a data form or graph form
Displays the Graph Legend
Displays the graph toolbar
Prints the graph
Copies Data to Clipboard
Copies Graph to Clipboard
Loads a saved graph configuration
Saves a graph configuration
Sets the current configuration as default
Clears the current chart configuration


Toggling to Graph displays the data in graph form.
Graph Legend
Toggling to Graph Legend displays a legend on the side of the graph.
240

Toolbar
Toggling to Toolbar displays the graph toolbar.
Print
Choosing Print displays the Print Dialog.
Copies the data to the clipboard.
Copies the graph to the clipboard.
Load Configuration
Benchmark factories graphs allows you to save graph configurations.
Save Configuration
Saves a graph configuration.
Set as Default
Sets a configured graph as default.
241
242
Clear Configuration
Clears a graph configuration.
Summary Tab
The Summary Tab properties displays information on the job when it is running. You can
monitor the status of the test, which userload its running, transactions per second (TPS), and a
variety of other datapoints that give you insight into the actual performance your database
provides under the userload being tested.
Iteration Overruns
Iteration overruns occur at the end of an iteration to allow time for all transactions submitted
within the test iteration cycle to complete, so that all transaction statistics can be collected. For
example, an agent may execute a transaction during the last five seconds of test iteration, if this
transaction takes 15 seconds to complete, an iteration overrun of 10 seconds will occur.

The Real Time Statistics node provides you with real-time graphs and raw data. This data allows
you to spot system-under-test issues that may be affecting server performance. The Real Time
243
Statistics node displays real-time counter data. Right-clicking inside the graph displays a dropdown that allows you to change graph settings and view.
To view Real-Time Statistic
1. In the Jobs Queue, click the job running.
2. Select Click here to Add Datapoints. The Add Datapoints dialog displays.
244
3. Click OK. The graph displays with the selected datapoints. (To change graph views,
right-click. See "Change Graph Views" (page 364) for more information.)
Run vs.Userload Tab

The Run vs. Load tab compares the user load transaction against the selected datapoint allowing
you to view how the various datapoints are being affected by the job running.
To view Run Vs.Userload tab testing results:

2. Click the datapoints Vs. Userload drop-down and select the desired datapoint.
3. To change graph views, right-click. See Change Graph Views for more information.
245

Transaction Node
The Transaction node displays user selected transactions statistics for the job timing period
following completion of an iteration.
To view the Real-time statistics node
1. Click the Summary node in the Jobs Queue. The Summary graph displays in the
Properties view.
246

2. Click the datapoints Vs. Userload drop-down.

4. To change graph views, right-click. A drop-down displays.
5. Select the desired option. Click here for procedures on changing graph views.
247
248
Benchmark Factory Run Reports

Run Reports

l
Transaction Time
Bytes/Second (BPS)
249
Rows/Second (RPS)
Total Bytes
Total Errors
Total Rows
Response Time

250

individual nodes.
251
Change Graph Views

Prints the graph



Graph Legend
252

Toolbar
Print
253
254

Load Configuration
Save Configuration
Set as Default
Clear Configuration
Run Reports Viewer

You can view a list of all Benchmark Factory Run Reports outside or inside of
Benchmark Factory
Accessing a List of all Benchmark Factory Run Reports from Outside of
Benchmark Factory
To access a list of all of Benchmark Factory Run Reports from outside of the Benchmark
Factory console
1. Click the Start button from the Windows task bar.
2. Select Programs |Quest Software | Benchmark Factory for Database | Run Reports.
Run Reports displays.
255
See "Using Benchmark Factory Run Reports" (page 257) for more information.
Accessing a List of All Benchmark Factory Jobs from Inside of
Benchmark Factory
To access all Benchmark Factory Jobs from inside of Benchmark Factory
1. From the Benchmark Factory console click Tools | Run Reports. Run Reports displays.
Note: For procedures on viewing individual reports, please see Viewing Load Testing Results.
Importing and Exporting Benchmark Factory Test Results from Run
Reports
From Run Reports, you can import or export run results.
To import or export testing results:
1. Open Run Reports.
2. Click on the required Import or Export button.

Navigate to the required folder to import or export the test results.

3. Click Open or Save.
View Load Testing Results

Benchmark Factory Run Reports allows you to:
l
View a single run report
View multiple run reports
To view a single run report

1. From the Jobs or Scripts view select Run Reports for the desired script.
256
257
3. Click the Show Test Results icon

. Run Reports displays in the upper right-hand
corner of Run Reports. Run reports displays.
4. Click here for procedures on using Benchmark Factory Run Reports.
Using Benchmark Factory Run Reports

Benchmark Factory Run Reports provides a wide array of graphs, charts, and data that allow you
to a analyze load testing scripts.
258
Navigating in Benchmark Factory RunReports

To open a test, double-click on the test name, or select the test name and click the Show Test
Results icon in the upper right corner of the dialog.

When Benchmark Factory Run Reports opens, the Results Summary (page 260) displays.
Breadcrumbs at the top of view allow you to navigate to different views.
To navigate Benchmark Factory RunResults:

1. Select the required view, for example, Testbed.
259
260
2. Click Details.
The Testbed view displays.
3. To navigate back to Results Summary, click the Results Summary breadcrumb in the
upper left corner.
Run Reports provides the following views:

l
Results Summary (page 260)
Workload (page 268)
Testbed (page 269)
Database Under Test (page 269)
Results Summary
Results Summary view provides graphs and user load statistics for the selected test. The
following graphs and tables are provided:
l
ResultsSummaryGraph (page 261)
Other Results SummaryGraphs (page 263)
(page 263)

ResultsSummaryGraph
Note: Not all test provide the Results Summary graph.
This is a customizable graph that allows you to view selected datapoints.
To add or remove datapoints to the Results Summary graph:

1. In the lower section of the graph, click Customize.
261

2. In the upper left corner, click Click here to Add Datapoints.
The Add Datapoints dialog displays.

2. Select the required datapoints.
262

3. Click Ok.
The selected datapoints display.
Other Results SummaryGraphs

The Results Summaryview allows you to view other graphs that provide test details.
263
264
To view other Results SummaryGraphs:

1. In the Results Summary view, click More Graphs.
The following graphs display:

l
Userload (page 264)
Realtime Summary (page 265)
Realtime Detail (page 266)
Userload
Understanding how user loads affect the performance of a database is essential to end user
satisfaction. The Userload graph plots user load against:
l
Response Time
TotalBytes
Total Errors
Total Rows
Response Time
Reviewing these datapoints allows you to fully understand how "real-world" userloads affect
database performance.
265
Realtime Summary
The Realtime Statistic graph allow you to view what "actually" happened during a load test.
You can plot userload against:
l
Transaction/Second
Total Rows
Errors
Total Bytes
Rows/Second
Minimum Time
Maximum Response Time
Deadlocks
Bytes/Second
Average Time
Bytes/Second
266
Realtime Detail
The Realtime Detail graph allows you to view what actually happened during the running of a
load test. This allows you to view the actual timing events. From the Realtime Detail graph
you can view:
l
Average Response Time
Average Time
Bytes/ Second
Deadlocks
Maximum Response Time
Maximum Time
Minimum Response Time
Minimum Time
Rows/Second
TotalBytes
Errors
TotalRows
Transactions/Second
User Load
267
Userload Statistics Table

This table is a snapshot of the workload test. You to quickly review a complete overview of
database that includes, userload, transactions per second (TPS), average response time (sec).
From this table you can drill down to view userload stastics.
To drill down from the UserloadStatistics table:
1. Click on the required Userload.

The Userload Results view displays.
2. From this view, you drill down further by clicking on required User Scenario name.
This displays detailed test results broken down by row name.
Workload
The Workload view provides testing details.
268
269
Clicking Details displays the transactions mix, timing, userload, and testing options used
for the test.
Testbed
The Testbed view shows data on agent configuration and processes during the running of a job.
Clicking Details displays a table with machine and operating system details.
Database Under Test

This view displays connection details.

Clicking Details displays connection information details.
View Multiple Run Results

Benchmark Factory allows you to view and compare multiple results using Run Reports.
To view multiple Run Reports results
1. Select the desired runs.
270
271
2. Click the Show Test Results icon

displays
in the upper right corner of the dialog. Run Reports
3. Click on an individual test run to view those results.
The results for that test display.

4. Click on Details for Workload, Testbed, or Database Under Test to drill down on
testing results.
The following graphic shows Testbed drill down results.
5. Click More Graphs in Results Summary.
272

The Results Summary graphs display.
Note: To change Run Reports graph views, refer to Changing Graph Views.
View other graphs as required.
273

6. Close Run Reports when finished viewing run results.
Export BenchmarkFactory Test Results to Excel

Benchmark Factory allows you to export test results to Microsoft Excel from:
Note: Microsoft Excel must be installed to perform the following procedure.
To export load test results from Run Reports
1. In Run Reports, click the Excel icon
in the upper left-hand corner.
274

2. Accept the default settings.

3. Click OK. Excel opens.
4. Select the desired tab and review the information.
5. Print the reports if desired.
6. Save the reports if desired
7. Select File | Exit.
275
11
Benchmarks
Overview
A benchmark is a performance test of hardware or software. A benchmark simulates real-world
application work loads. These models of real applications can then be run against the system
being evaluated. Models work better than the actual application and offers reproducible results.
The real application, on the other hand, has too many variables that can change the results over
several test sessions.
Realistic Expectations When Using Benchmarks

Industry standard benchmarks represent real application workloads, their run results depend on
workload definition, (user load, latency, etc.) as well as server configuration and tuning
parameters. During the running of a benchmark, deadlock errors can occur. This is not an issue
with Benchmark Factory, but more of an issue with the tuning of the workload as well as the
system-under-test. For example, it is would be unrealistic to expect no errors from a workload
that runs 1000 users updating a single row on a small table. To further trouble shoot any tuning
issues, we recommend Quest Software Spotlight products.
What Benchmarks Measure

Benchmarks measure performance that analyze:
l
Raw performance of a complete system
Raw performance of a specific subsystem (disk, video, CPU, memory, etc.)
Performance of a computer running a particular application
Performance of applications running on a network
Capacity of a system. This is often refereed to as capacity planning.
Provided Benchmarks
Benchmark Factory provides the following standard benchmarks:
l
AS3AP
Scalable Hardware
TPC-B
TPC-C
TPC-D
277
Benchmarks
TPC-E
TPC-H
AS3AP Benchmark
The AS3AP benchmark is an American National Standards Institute (ANSI) Structured Query
Language (SQL) relational database benchmark. The AS3AP benchmark provides the
following features:
l
Tests database processing power
Built-in scalability and portability that tests a broad range of database systems
Minimizes effort in implementing and running benchmark tests
Provides a uniform metric and straight-forward interpretation of benchmark results
Systems tested with the AS3AP benchmark must support common data types and provide a
complete relational interface with basic integrity, consistency, and recovery mechanisms. The
AS3AP tests systems ranging from a single-user microcomputer Database Management System
(DBMS) to a high-performance parallel or distributed database.
Best Practices
Do not load-test against a production server if possible. Load-testing and benchmarking on a
production server significantly degrades performance. In some cases, load-testing can cause a
server to fail. However, if testing against a production server, take the following precautions:
l
Perform the testing when no other users are on the system and no automated processes are
running. Users and automated processes can adversely affect testing results
Have a recovery plan and backup all data prior to testing
Determine how long it will take to restore a production server if it went down during
load-testing
Perform manual testing. Manual testing ensures that no unexpected outside activity takes
place during the testing process
Reinitialize the Database

To reinitialize a testing database, run a job containing a Benchmark Object node.
There are two ways to create a Benchmark Object node:
l
Run the Benchmark Object Wizard to add the Delete Benchmark objects for AS3AP node
to a job as needed. See "Benchmark Objects Wizard" (page 171) for more information.
278
Benchmarks
Run the Load Scenario Wizard to create a new script containing the Create Objects for
the Benchmark, Associated Load Scenarios for the Benchmark, and Delete Objects for the
Benchmark. Running the delete objects for 'AS3AP' job will clean the environment.
Scaling Factor
The AS3AP benchmark scales by factor of 10.
Scalable Hardware Benchmark

The Scalable Hardware benchmark measures relational database systems. This benchmark is a
subset of the AS3AP benchmark and tests the following:
l
CPU
Disk
Network
Any combination of the above three entities
How the Scalable Hardware Benchmark Works

The scale factor determines the amount of information initially loaded into the benchmark tables.
For the Scalable Hardware benchmark, each scale factor represents one user accessing the system.
Two tables are created in the database, and they are loaded with a varying number of rows.
For each virtual user, a separate set of data must be created. Therefore the scale factor used when
loading the database should be the size of the maximum user load. For example, with user loads
of 1, 5, and 10, a scale factor of 10 should be used.
Scaling Factor
The Scalable Hardware benchmark has a scaling factor of one.
Best Practices
Load-testing against production servers
l
279
Benchmarks
load-testing

l
Run the Benchmark Object Wizard to add the Delete Benchmark objects for the Scalable
Hardware node to a job as needed.
Benchmark. Running the delete objects for 'Scalable Hardware" job will clean the
environment.
TPC-B Benchmark
l
Overview
Best Practices
Scaling Factor
Overview
The Transaction Processing Council is an organization that establishes transaction processing and
database benchmark standards. For a complete overview and detailed explanation of the TPC-B
Benchmark, please refer to:
TPC-B Benchmark
Certification of Transaction Processing Council (TPC)

Testing Results
Transaction Processing Council testing results cannot be published as certified unless the testing
procedure is audited and approved by the TPC organization. If not certified, the testing results
can be published as a "TPC-B like" test.
Best Practices
The following provides best practices for the TPC-B Benchmark.
280
Benchmarks

l
load-testing

l
Run the Benchmark Object Wizard to add the Delete Benchmark objects for TPC-B node
to a job as needed.
Benchmark. Running the delete objects for 'TPC-B' job will clean the environment.
History Tables
The TPC-B benchmark is made up of only one transaction that updates three tables and inserts a
record into a history table. Inserting one record into one history table limits testing performance.
The Benchmark Factory properties page allows the user to set the number of history tables to
create during a test. The best ratio of history tables to virtual users is based on database
configuration and hardware. The number of history tables to use is determined by the tester.
Scaling Factor
The TPC-B benchmark scales by a factor of one.
TPC-C Benchmark
l
Overview
TPC-C Tables
281
Benchmarks
Best Practices
Overview
Note: For a detailed overview of the TPC-C Benchmark, please refer to the Transaction
Processing Council - TPC-C Benchmark.
The TPC-C benchmark is an online transaction processing benchmark that simulates
environments that have a number of terminal operators that send transactions to a database. This
benchmark is focused on the concept of an order-entry type environment with transaction that
include orders, payment recording, order status, and stock level monitoring. This benchmark
portrays the activities of a wholesale supplier. However, the TPC-C is not limited to one
particular business segment. It can represent numerous categories of a business that sell or
distribute products and services.
The TPC-C benchmark simulates a wholesale parts dealer operating out of warehouses. This
Benchmark scales as a company, in theory, expands their business or number of facilities. As the
TPC-C benchmark scales, so do the number components for the benchmark, for example, the sales
districts and customers.
TPC-C Tables
The scale factor determines the amount of information initially loaded into the benchmark tables.
For the TPC-C benchmark, each scale factor represents one warehouse as per TPC-C specification.
The TPC-C benchmark involves a mix of five concurrent transactions of different types and
complexity. The database is comprised of nine tables with a wide range of records.
A maximum of 10 users should be run against each warehouse. For example, user loads of 1, 5,
and 10, set the scale to 1. If using other user load values, change the scale factor accordingly.
The TPC-C database consists of the following tables:
Warehouse
District
Customer
History
New_Order
Order
282
Benchmarks
Order_Line
Item
Stock

Testing Results
can be published as a "TPC-C like" test.
Best Practices
The following provides best practices for the TPC-C Benchmark.
l
load-testing

l
Run the Benchmark Object Wizard to add the Delete Benchmark objects for TPC-C node
to a job as needed.
Benchmark. Running the delete objects for 'TPC-C' job will clean the environment.
283
Benchmarks
TPC-D Benchmark
l
Overview
TPC-D Transaction Details
Best Practices
Scaling Factor
Overview
database benchmark standards. For a complete overview and detailed explanation of the TPC-D
TPC-D Benchmark

Testing Results
can be published as a "TPC-D like" test.
Best Practices
The following provides best practices for the TPC-D Benchmark.
l
load-testing
284
Benchmarks

l
Run the Benchmark Object Wizard to add the Delete Benchmark objects for TPC-D node
to a job as needed.
Benchmark. Running the delete objects for 'TPC-D' job will clean the environment.
Scaling Factor
The TPC-D benchmark scales by the following factors:
l
0.10
1.00
10.00
30.00
100.00
300.00
TPC-E Benchmark
l
Overview
Best Practices
Scaling Factor
Overview
database benchmark standards. For a complete overview and detailed explanation of the TPC-E
Benchmark, refer to:
TPC-E Benchmark
285
Benchmarks

Testing Results
can be published as a "TPC-E like" test.
Best Practices
The following provides best practices for the TPC-E Benchmark.
l
load-testing
Scaling Factor
The TPC-E benchmark scales by factor of 500.
TPC-H Benchmark
l
Overview
Best Practices
Scaling Factor
Overview
database benchmark standards. For a complete overview and detailed explanation of the TPC-H
286
Benchmarks

TPC-H Benchmark
Certification of Transaction Processing Council Testing

Results
can be published as a "TPC-H like" test.
Best Practices
The following provides best practices for the TPC-H Benchmark.
l
load-testing

l
Run the Benchmark Object Wizard to add the Delete Benchmark objects for TPC-H node
to a job as needed.
Benchmark. Running the delete objects for 'TPC-H' job will clean the environment.
Scaling Factor
The TPC-H benchmark scales by the following factors:
l
1.00
10.00

Benchmarks
30.00
100.00
300.00
1000.00
3000.00
10000.00
30000.00
100000.00
287
12
BFScripts
About Scripts
Note: All edit field with a yellow background are scriptable.
Benchmark Factory provides a scripting capability as a means of inserting built-in functions for
use in scripts. A function is a formula that takes one or more values (arguments), performs an
operation, and returns a value that simulates real-world user activity. Functions are used alone or
as building blocks in creating complex user activity. Randomized data is important when
attempting to simulate real-world user activity because data that is random, prevents a server
from using data stored in its cache. Click here for procedures on how to use scripts.
The following provides a list of available scripts:
Category
Script Function
Dates and Numbers
$BFCreditCardExp
$BFCurrentDate
$BFCurrentDateTime
$BFDate
File Access
$BFFileArray
Global Variables
$BFGetGlobalVar (page 294)

$BFSetGlobalVar (page 294)
$BFSetGlobalVarRtn (page 295)
Name and Address
$BFAddress
$BFAddress2
$BFCity
$BFCompany
$BFCountry
$BFEMail
$BFFirstName
$BFFullName
$BFLastName
$BFMiddleInitial
$BFPhone
$BFState
$BFZipCode
$BFFormat

BFScripts
$BFProd
$BFSum
Random Numbers
$BFRand
$BFRandRange
$BFURand (page 303)
$BFURandRange
Random Strings
$BFCreditCard
$BFRandList
$BFRandMultiList
$BFRandStr
$BFURandList (page 307)
$BFList
String Manipulation
$BFAsc
$BFChr
$BFConcat
$BFLeft
$BFLen
$BFLower
$BFMid (page 311)
$BFRight
$BFTrim
$BFTrimLeft
$BFTrimRight
$BFUpper
Test Info
$BFGetVar
$BFMaxNode
$BFNode
$BFProfile
$BFRunID
$BFSetVar
$BFSetVarRtn (page 316)
$BFUserCounter
$BFUserID
$BFUserLoad
$BFNextUserload
$BFPrevUserload
289
290
BFScripts
Using Scripts in Username and Password Fields

Note: All edit field with a yellow background are scriptable.
Scripts can be used in profiles to login with a variety of users and passwords. If scripts are
to be used in a password field, the password needs to be contained entirely in a script such
as $BFConcat.
Example: $BFConcat("USER",$BFUserID(),"X") Will generate: "USER1x" for Virtual
User 1; "USER2x" for Virtual User 2; ...
This example can be useful to log in each virtual user with a unique username and password
designated for this use. In this case users should be created in the test system with a username
and password containing the virtual user number that will be used.
Dates and Numbers

$BFCreditCardExp
Description:
Gets the global variable.
Parameters:
N/A
Syntax:
$BFGetGlobalVar('myvar')
$BFCurrentDate
Description:
Allows you to change the date format used to populate date fields.
Parameters:
Format string-The mix of static text and variables as needed.
Variables
%a
%A
Definition
Abbreviated weekday name
%b
Full weekday name

Abbreviated month name
%B
Full month name
%c
Date and time representing appropriate locale
291
BFScripts
%d
%H
%I
%j
%m
%M
%p
%S
%U
%w
%W
%x
%X
Day of month as decimal number (01-31)

Hour in 24-hour format (00-23)
Hour in 12 hour format (01-12)
Day of year as decimal number (001-366)
Month as decimal number (01-12)
Minute as decimal number (00-59)
Current locale's A.M./P.M indicator for 12 hour clock
Second as decimal number (00-59)
Week of year as decimal number, with Sunday as first day of the week (0053)
Weekday as decimal number (0-6; Sunday is 0)
Week of year as decimal number, with Monday as first day of the week (0053)
Date representation for current locale
Time representation for current locale
%y
%Y
%z, %Z
Year without century, as decimal (00-99)

Year with century, as decimal number
%%
Percent sign
Either the time-zone name or the time zone abbreviation,

depending on registry settings; no characters if time zone is
unknown
Syntax:
$BFCurrentDate (Format String)
Example:
$BFCurrentDate ("%m/%d/%Y) ; returns 03/28/2005
BFCurrentDateTime
Description:
Allows you to change the date/time format used to populate date fields.
Parameters:
Format string-The mix of static text and variables as needed.
Variables
%a
%A
Definition
Abbreviated weekday name
%b
Full weekday name

Abbreviated month name
%B
Full month name
%c
%d
Date and time representing appropriate locale

Day of month as decimal number (01-31)
Hour in 24-hour format (00-23)
%H
292
BFScripts
%I
%j
%m
%M
%p
%S
%U
%w
%W
%x
%X
Hour in 12 hour format (01-12)

Day of year as decimal number (001-366)
Month as decimal number (01-12)
Minute as decimal number (00-59)
Current locale's A.M./P.M indicator for 12 hour clock
Second as decimal number (00-59)
Week of year as decimal number, with Sunday as first day of the week (0053)
Weekday as decimal number (0-6; Sunday is 0)
Week of year as decimal number, with Monday as first day of the week (0053)
Date representation for current locale
Time representation for current locale
%y
%Y
%z, %Z
Year without century, as decimal (00-99)

Year with century, as decimal number
%%
Percent sign
Either the time-zone name or the time zone abbreviation,

depending on registry settings; no characters if time zone is
unknown
Syntax:
$BFCurrentDateTime (Format String)
Example:
$BFCurrentDateTime ("%m/%d/%Y %H:%M:%S") ; returns 03/28/2005 14:18:52
$BFDate
Description:
Randomly generates a date between a start date and a end date.
Parameters:
nStart-The start date. mm/dd/yyyy

nEnd-The end date.
mm/dd/yyyy
Syntax:
$BFDate(nStart,nEnd)
Example:
$BFDate("09/14/2004","09/14/2005") ; returns "11/04/2004"
293
BFScripts
File Access
$BFFileArray
Description:
Selects an item from a list. Returns a single item from a comma-delimited file.
The item returned depends on the mode selected. The syntax of the statement is
also slightly different for each mode. Each virtual user gets a different seed
value to generate unique sequences. Each agent machine must have a file with
the name and path that is specified in the script function. If $BFFileArray is to
return strings, the items in the file must be in double-quotes
Parameters:
File name-The name and path of the source file.

Retrieval Mode:
CURRENT:Select the item in the specified column of the current row.
RANDOM:Select a random item from a given column.
SELECTION:Select an item based on column and row.
SEQUENTIAL:Select each item sequentially.
UNIQUE:Select a non-repeating item from a given column.
nColumn-Which column of values to select. If none is given, a value of 1 is
assumed.
nRow-Which row to select for the item.
Note: This value is only needed for the SELECTION mode.
Syntax:
$BFFileArray(FileName, Retrieval Mode[,nColumn][, nRow])
Example:
If myfile.dat consists of:

"George Washington", "Martha"
"Franklin Roosevelt","Eleanor"
"Gerald Ford","Betty"
$BFFileArray( "c:\path\myfile.dat", SELECTION, 2, 3)
Returns Betty.
$BFFileArray( "c:\path\myfile.dat", RANDOM,2)
Returns either Martha, Eleanor, or Betty. Will return Martha, Eleanor, or
Betty the next time it is run.
$BFFileArray( "c:\path\myfile.dat", UNIQUE,2)
Returns either Martha, Eleanor, or Betty. Will return Martha or Betty the
next time it is run if it returned Eleanor the first time.
$BFFileArray( "c:\path\myfile.dat", SEQUENTIAL, 1)
Returns George Washington then Franklin Roosevelt then Gerald Ford.
$BFFileArray( "c:\path\myfile.dat", CURRENT, 2)
Returns Betty if the previous script function returned Gerald Ford.

BFScripts
Global Variable
$BFGetGlobalVar
Description:
Generates the GetGlobal Variable.
Parameters:
N/A
Syntax:
$BFGetGlobalVar('myvar')
$BFSetGlobalVar
Description:
Generates the Set Global Variable
Parameters:
N/A
Syntax:
$BFSetGlobalVar('myvar', 'my value')
294

BFScripts
$BFSetGlobalVarRtn
Description:
Generates the Set Global Variable Return.
Parameters:
N/A
Syntax:
$BFSetGlobalVarRtn('myvar', 'my value')
Name and Address

$BFAddress
Description:
Returns a randomly generated street address string.
Parameters:
N/A
Syntax:
BFAddress()
Example:
$BFAddress() ; returns "904 Oak Lane"
$BFAddress2
Description:
Returns a second randomly generated street address string

containing an apartment number, suite number, or villa.
Parameters:
N/A
295
296
BFScripts
Syntax:
$BFAddress2()
Example:
$BFAddress2() ; returns
"Apt 5442"
$BFCity
Description:
Generates a random city name.
Parameters:
N/A
Syntax:
$BFCity()
Example:
$BFCity() ; returns "Trend Blue of Asia"
$BFCompany
Description:
Returns a random company name
Parameters:
N/A
Syntax:
$BFCompany()
Example:
$BFCompany(); returns "Quest Software, Inc."

BFScripts
$BFCountry
Description:
Returns a randomly generated country string.
Parameters:
N/A
Syntax:
$BFCountry()
Example:
$BFCounty() ; returns "Canada"
$BFEmail
Description:
Returns a random email address string.
Parameters:
N/A
Syntax:
$BFEmail()
Example:
$BFEmail() ; returns "Rita_Smyth@TrendBlue.net"
$BFFirstName
Description:
Returns a random first name string.
Parameters:
N/A
Syntax:
$BFFirstName()
Example:
$BFFirstName() ; returns: "Hary D. Jones"
297

BFScripts
$BFFullName
Description:
Returns a random full name string.
Parameters:
N/A
Syntax:
$BFFullName()
Example:
$BFFullName() ; returns "Harry Jones"
$BFLastName
Description:
Returns a random last name string.
Parameters:
N/A
Syntax:
$BFLastName()
Example:
$BFLastName() ; returns "Jones"
$BFMiddleInitial
Description:
Returns a random middle initial character.
Parameters:
N/A
Syntax:
$BFMiddleInitial()
298
299
BFScripts
Example:
$BFMiddleInitial() ; returns "H"
$BFPhone
Description:
Returns a randomly generated telephone string.
Parameters:
N/A
Syntax:
$BFPhone()
Example:
$BFPhone() ; returns
"(348)889-6599"
$BFState
Description:
Returns a randomly generated state string.
Parameters:
N/A
Syntax:
$BFState()
Example:
$BFState() ; returns "CA"
300
BFScripts
$BFZipCode
Description:
Returns a randomly generated zip code string.
Parameters:
N/A
Syntax:
$BFZipCode()
Example:
$BFZipCode() ; returns "52076"
$BFFormat
Description:
Formats a series of up to 16 numbers. If the amount of numbers

in the series is greater than 16, the Maximum Parameters
Exceeded message displays.
Parameters:
Format string -A %d for each number in the series.

n1-The first number in the series.
n2-The second number in the series.
n16-The last number in the series.
Syntax:
$BFFormat(Format string, n1, n2, , n16)
Example:
$BFFormat("%d is a crazy number", $BFRandRange(10,200)) ;

returns "34 is a crazy number"

BFScripts
$BFProd
Description:
Returns the product of a series of floating-point numbers.
Parameters:
f1-A floating-point number.

f2- A second floating-point number.
fn-The last floating-point number.
Syntax:
$BFProd(f1, f2, fn)
Example:
$BFProd(5, 2.5, .01) returns ; "0.125"
$BFSum
Description:
Returns the summation of a series of numbers.
Parameters:
f1-The first number to be summed.

f2-The second number to be summed.
fN-The last number to be summed.
Syntax:
$BFSum(f1, f2, , fN)
Example:
$BFSum(10, 20, 30) ; returns "60"
301
302
BFScripts
Random Numbers
$BFRand
Description:
Returns a random
integer between 0
and nMax. Each
virtual user gets a
different seed
value to generate
the same unique
sequences for each
run.
Parameters:
nMax-The
maximum integer
to be returned by
the function.
Syntax:
$BFRand(nMax)
Example:
$BFRand(100) ;
returns "45"
$BFRandRange
Description:
Returns a random integer ranging between nMin and nMax

inclusive. Each virtual user gets a different seed value to
generate sequences.
Parameters:
nMin-The minimum integer in a range.

nMax-The maximum integer in a range.
Syntax:
$BFRandRange(nMin,nMax)
Example:
$BFRandRange(200, 210) ; returns "202"
303
BFScripts
$BFURandRange
Description:
Returns unique integers ranging between the value of

nMin and the value of nMax inclusive. Each virtual user
gets a different seed value to generate unique sequences
for each run.
Parameters:
nMin-The minimum range integer to return.

nMax-The maximum range integer to return.
Syntax:
$BFURandRange(nMin,nMax)
Example:
$BFRandRange(1,100) ;
returns 100 the first time this function executes and 95,
and 85 for subsequent executions of this function.
$BFURand
Description:
Returns a unique (non-repeating) random integer ranging between 1 and nMax.

Each virtual user gets a different seed value to generate unique sequences for
each run.
Parameters:
nMax-The maximum integer to be returned.
Syntax:
$BFURand(nMax)
Example:
$BFURand(100) ;
returns 78 the first time this function executes, and 50, and 19 for
subsequent executions of this function.
304
BFScripts
Random String
$BFCreditCard
Description:
Returns a random credit card number string.
Parameters:
N/A
Syntax:
$BFCreditCard()
Example:
$BFCreditCard() ; returns "7970 2384 8889 6599"
$BFRandList
Description:
Returns a string randomly selected from the list of items. If no

weight is specified, a weight of 1 is assumed. Each virtual user
gets a different seed value to generate unique sequences.
Parameters:
string1-The first string in a list to return.

nWeight1-Positive integer indicating the relative weight of the
first string.
string2-The second string in a list to return.
second string.
305
BFScripts
stringN-The last string in a list to return.

nWeightN-Positive integer indicating the relative weight of
the last string.
Syntax:
$BFRandList(string1[:nWeight1], string2[:nWeight2], ,
stringN[:nWeightN])
Example:
$BFRandList("Red", "White":2, "Blue":5)

The value White will be returned 2 times more often than the
value Red. The value Blue will be returned 5 times more often
than the value Red.
$BFRandMultiList
Description:
Randomly selects multiple strings based on probabilities from

a list. If Weight is omitted, a value of 100 is assumed. The
probability that any string is include in the returned value is
determined by the value of nWeight. Each string included in
the return value is separated by a comma.
Parameters:
string1-The first string in a list to return.

first string. Probability should be expressed as a value between
0 and 100.
string2-The second string in a list to return.
second string. Probability should be expressed as a value
between 0 and 100.
stringN-The last string in a list to return.
nWeightN-Positive integer indicating the relative weight of
the last string. Probability should be expressed as a value
between 0 and 100.
306
BFScripts
Syntax:
$BFRandMultiList(string1[:nWeight1], string2[:nWeight 2],

, stringN"[: nWeightN])
Example:
$BFRandMultiList("Red", "White":50, "Blue":25) ;

returns a value that always contains Red, contained White
50% of the transactions, and Blue 25% of the transactions.
$BFRandStr
Description:
Returns a random string determined by a mode and having a

length n.
Parameters:
Length n-The length of a string.

Mode constant-One of the following:
ALPHA-Returns only random letters.
ALPHA_LOWER-Returns random lowercase letters.
ALPHA_NUM-Returns random letters and numbers.
ALPHA_UPPER-Returns random uppercase letters.
ANY-Returns any random ASCII characters.
NUM-Returns only numbers.
RANGE-Returns ASCII characters in the range specified.
Syntax:
$BFRandStr(n,[constant],[1..255],[1..255])
Example:
$BFRandStr(10) ; Returns 10 random characters.

$BFRandStr(1,RANGE,65,67) ; Returns either A, B, or C.
307
BFScripts
$BFURandList
Description:
Randomly select unique items from a list.
Parameters:
string1-The first string to return from a list.

string2-The second string to return from a list.
stringN-The last string to return from a list.
Syntax:
$BFURandList(string1,string2, stringN)
Example:
$BFURandList("Red", "White", "Blue", "Yellow") ; returns

Blue the first time this function executes then Yellow, Red,
White, and Blue for subsequent executions of this function.
Notice that Blue did not repeat until all other colors returned.
$BFList
Description:
Returns an item from a list. The item returned depends on the

mode selected.
Parameters:
Retrieval Mode:
RANDOM: Select a random item from the list.
SEQUENTIAL: Select each item sequentially.
UNIQUE: Select a non-repeating item from the list.
string1-The first string to return from a list.
string2-The second string to return from a list.
stringN-The last string to return from a list.
Syntax:
$BFList(Retrieval Mode, string1,string2, stringN)
Example:
$BFList (Sequential "1", "2", "3", "4")

Returns 1
2
etc.

BFScripts
$BFList (Random 1, 2, 3, 4)
Returns 2
3
3
1
2
4
etc.
$BFList (Unique, "1", "2", "3","4")
Returns 2
4
3
1
2
etc.
String Manipulation
$BFAsc
Description:
Returns the ANSI value of the first character of a string.
Parameters:
String-Characters enclosed in quotation marks.
Syntax:
$BFAsc(string)
Example:
$BFAsc("George") ; Returns "71"
$BFChr
Description:
Returns the character associated with the specified

ANSI code.
308

BFScripts
Parameters:
n-An integer representing an ANSI code.
Syntax:
$BFChr(n)
Example:
$BFChr(68) ; returns "D"
$BFConcat
Description:
Returns a string containing two or more strings.
Parameters:
string1-The first string to return.

string2-The second string to return.
stringN-The last string to return.
Syntax:
$BFConcat("string1", "string2", , stringN)
Example:
$BFConcat ("The connection interval ", "determines how

often an ", "agent attempts to connect to the Benchmark
Factory Console.") ; returns "The connection interval
determines how often an agent attempts to connect to the
Benchmark Factory Console"
$BFLeft
Description:
Returns the first n character of a string.
Parameters:

nLength-The length of the string to return.
309
310
BFScripts
Syntax:
$BFLeft(string,length)
Example:
$BFLeft("Benchmark Factory", 5) ; returns
"Bench"
$BFLen
Description:
Returns the number of characters in a string.
Parameters:
Syntax:
$BFLen(string)
Example:
$BFLen("Benchmark Factory") ; returns "17"
$BFLower
Description:
Returns a string after converting uppercase characters to

lowercase characters.
Parameters:
Syntax:
$BFLowerstring)
Example:
$BFLower("SAMPLING") ; returns "sampling"

BFScripts
$BFMid
Description:
Extracts a substring from a string.
Parameters:
String-Characters enclosed in a quotation marks.

nStart-The starting character position.
nLength-Number of characters to return.
Syntax:
$BFMid(string, nStart,nLength)
Example:
$BFMid("sunshine coast", 3, 5) ; returns "shine"
$BFRight
Description:
Returns the last n character of a string.
Parameters:

nLength-The number of characters to return.
Syntax:
$BFRight(string,nLength)
Example:
$BFRight("sunshine coast", 3) ; returns "ast"
$BFTrim
Description:
Returns a string void of leading and trailing spaces.
311
312
BFScripts
Parameters:
Syntax:
$BFTrim(string)
Example:
$BFTrim(" happy days are here to stay. ") ; returns "happy days
are here to stay"
$BFTrimLeft
Description:
Returns a string void of leading spaces.
Parameters:
Syntax:
$BFTrimLeft(string)
Example:
$BFTrimLeft(" hockey is great ") ; returns " hockey is great "
$BFTrimRight
Description:
Removes trailing spaces from a string.
Parameters:
nString-Characters enclosed in quotation marks.
Syntax:
$BFTrimRight(string)
Example:
$BFTrimRight(" removes trailing spaces from a string. ") ;

returns " removes trailing spaces from string. "
313
BFScripts
$BFUpper
Description:
Returns a string after converting lowercase characters to

uppercase characters.
Parameters:
nString value-Characters enclosed in quotation marks.
Syntax:
$BFUpper(string)
Example:
$BFUpper("sampling") ; returns "SAMPLING"
Test Info
$BFGetVar
Description:
Retrieves a previously stored value using

$BFSetVar. Allows a value to be passed from one
transaction to another in conjunction with
$BFSetVar, or when value is used multiple times
within a transaction. Each virtual user has its own
variable space, so values are not shared between
them.
Parameters:
VarName-An alphanumeric identifier of the value

stored.
Syntax:
$BFGetVar("VarName")
Example:
$BFSetVar("Totalrow", "2") $BFGetVar

("Totalrow") ; if $BFSetVar sets the variable
"Totalrow" to 2, $BFGetVar("Totalrow") returns
the number 2.
314
BFScripts
$BFMaxNode
Description:
Returns the total number of nodes for all users. This function
is intended only for Oracle clustering.
Parameters:
N/A
Syntax:
$BFMaxNode()
Example:
$BFMaxNode() ; returns: 2. This is the same value seen in the

Number of Nodes field in Clustering tab for an Oracle profile.
$BFNode
Description:
Returns the node number of the current user. This function is

intended only for Oracle clustering.
Parameters:
N/A
Syntax:
$BFNode()
Example:
$BFNode() ; returns "1"
Description:
Returns the current number of iterations of a test.
Parameters:
N/A
Syntax:
$BFNumberOfIterations()
Example:
$BFNumberOfIterations() ; returns "1"
315
BFScripts
$BFProfile
Description:
Returns driver specific information, such as database

name.
Parameters:
Profile (constant)-The following provides a list of

database type constants:
MSSQL (native), SYBASE:
l
SERVERNAME
DATABASE
USERNAME
ORACLE:
l
NETSERVICENAME
USERNAME
ODBC, MYSQL (ODBC), MSSQL (ODBC):

l
DATASOURCENAME
USERNAME
DB2:
l
DATABASEALIAS
USERNAME
Syntax:
$BFProfile(constant)
Example:
$BFProfile(USERNAME) ; returns 'sa'
$BFRunID
Description:
Returns the run ID of the current test.
316
BFScripts
Parameters:
N/A
Syntax:
$BFRunID()
Example:
$BFRunID() ; returns "186"
$BFSetVar
Description:
Stores a value for later use by $BFGetVar. Used to store a

value to be reused within its own transaction, or any
transaction in a given user scenario. Each virtual user gets its
own variable space, so values are not shared between them.
Typically, $BFSetVar is placed at the beginning of a dynamic
statement, as scripts are evaluated from left to right.
Parameters:
Variable Name (VarName)-An alphanumeric identifier of the

value stored.
Text to Store (Value)-A string. The value to be stored for
later retrieval.
Syntax:
Syntax:$BFSetVar("VarName", "Value")
Example:
$BFSetVar("Totalrow", "2") ; $BFSetVar sets the variable

"Totalrow" to 2
$BFSetVarRtn
Description:
Stores and returns a value to be reused within its own

transaction, or any transaction in a given user scenario. Each
virtual user gets its own variable space, so values are not
shared between them. Typically, $BFSetVarRtn is placed at
the beginning of a dynamic statement, as scripts are evaluated
from left to right.

BFScripts
Parameters:
Variable Name (VarKey)-A string to store the a value.

Text to Store (Value)-A string value to be stored for later
retrieval.
Syntax:
$BFSetVarRtn("Varkey","Value")
Example:
$BFSetVarRtn("Totalrows", "2") ; returns "2"
$BFUserCounter
Description:
Returns the
user
counter.
Parameters:
N/A
Syntax:
$BFUserCounter()
Example:
$BFUserCounter() ;
returns "1"
317

BFScripts
$BFUserID
Description:
Returns the current virtual user ID.
Parameters:
N/A
Syntax:
$BFUserID()
Example:
$BFUserID() ; returns "1"
$BFUserLoad
Description:
Returns the current user load for the test running.
Parameters:
N/A
Syntax:
$BFUserload()
Example:
$BFUserLoad() ; returns "4"
$BFNextUserload
Description:
Returns the next user load for the load scenario running.
Parameters:
N/A
Syntax:
$BFNextUserload()
318
319
BFScripts
Example:
$BFNextUserload()
When running with userloads 1, 4, 6, 10 this will return "6"
when running at userload 4.
$BFPrevUserload
Description:
Returns the previous user load for the load scenario

running.
Parameters:
N/A
Syntax:
$BFPrevUserload()
Example:
$BFPrevUserload()
When running with userloads 1, 4, 6, 10 this will
return "4" when running at userload 6.
13
Quick Starts
About the Quickstarts
These quick start procedures introduce you to the Benchmark Factory environment. The intent of
these procedures is not to show every aspect of the load testing process or every feature of
Benchmark Factory. They provide the necessary background to quickly explore and begin using
Benchmark Factory. The following quick starts are provided:
l
Real Application Cluster (RAC)
Upgrading Your Database Server Using Benchmark Factory
Oracle Trace Import-Database Replay
Standard Benchmark
Goal Testing

Overview
An Oracle Real Application Cluster (RAC) combines several independent servers/nodes into one
database system. RAC is a clustered database solution that provides two major functions:
scalability and availability to a business continuum. Scalability is a relative term, based on a
simple rule that as the number of users accessing the system increases, the RAC configuration
should be able to handle increased activity. Availability is the ability of the system to provide
continuous service when one or more of the components in the cluster fail.
environment demands when adding nodes. An Oracle Real Application Cluster (RAC) combines
several independent servers/nodes into one database system. RAC is a clustered database solution
that provides two major functions: scalability and availability to a business continuum.
Scalability is a relative term, based on a simple rule that as the number of users accessing the
system increases, the RAC configuration should be able to handle increased activity. Availability
is the ability of the system to provide continuous service when one or more of the components in
321
Quick Starts
the cluster fail.

environment demands when adding nodes.

This Quick Start presents a load testing methodology and steps required to implement, manage,
and perform full diagnostic capabilities of the Oracle RAC cluster, in order to obtain optimum
user service times regardless of the demands placed on the RAC environment.

l
Creating the Benchmark Factory cluster profile

Navigate to the tnsnames.ora file located in Oracle-Home\network\admin. Transparent Network
Substrate (TNS) is Oracle's networking architecture. TNS provides a standard application
interface that enables network applications to transparently access underlying network protocols.
Open the file in a text editor. Edit the tnsnames.ora file to set up clustering connections. For the
first entry, the Net Service Name, in this example MyTNS, is used to define the root Net Service
Name. The root Net Service Name is not used when running a test. The remaining Net Service
Names must be named with the root and a number. In this example, MyTNS is the root and
MyTNS1, MyTNS2, and MyTNS3 will be used when the test is run. The "MyTNS" and
"MyTNS1" connections are duplicates and are required for Benchmark Factory to perform load
testing. MyTNS2 and MyTNS3 are for the other two machines in the cluster. Figure 1
provides an excerpt from a tnsnames.ora file set up for a three node cluster.
322
Quick Starts
Figure 1-tnsnames.ora file set up for a three node cluster.

Creating the Benchmark Factory Clustering Profile
The profile contains information required to connect to the system-under-test, such as server
name, IP address, and user name.
To create the Benchmark Factory Clustering profile
Start if desired.
323
Quick Starts
3. Click Wizards | Load Scenario.

4. Click Next. The Load Scenario dialog displays.
7. Click New. The Profile Creation Wizard displays.
8. Click Next. The driver selection dialog displays.
9. From the driver drop-down, select Oracle.
12. Click Next. The Profile Name dialog displays.

15. Click Finish. The Profile Selection dialog displays.
16. Click Next. The Edit Scale dialog displays.
17. Click the TPC-C Benchmark Edit button. The Benchmark 'TPC-C' Properties page displays.
18. Select the desired Benchmark Scale. The larger the scale selection, the larger the database that will when created.
It is suggested that a scale of 1 be selected, unless you desire a larger database to test with.
Note: Clicking the Advanced tab allows you to map benchmark objects. For these
procedures, please see Advanced Creation Objects (Object Mapping).
19. Click OK.
20. Click the TPC-H Benchmark Edit button.
21. Select the desired benchmark scale. The TPC-H benchmark creates large testing databases. A scale of 1
is suggested.
22. Click OK.
24. Enter the desired number of nodes in Nodes. In this example, 10 nodes are selected.
25. Under Method, click the Benchmark Factory radio button.
324
Quick Starts
27. Enter the desired User Load.

l
29. Select Create a new Benchmark Factory script.

32. Click Finish. The script creates and displays in the Script View.

Benchmark Factory allows you to replay production workload and SQL scripts in different
database environments, and places enormous stress on a database. By replaying production
activity in a test database, potential performance problems can be identified before occurring in
production. A system typically breaks under extreme load. By identifying system capacity and
performance bottlenecks before they occur, Benchmark Factory facilitates proactive testing,
which in turn reduces downtime, development costs, and potential loss of revenue.
Benchmark Factory allows you to import a variety of user loads that include:
l
Industry standard benchmark to measure the performance of a system-under- test.

Benchmark Factory provides the AS3AP, Scalable Hardware, TPC-B, TPC-C, TPC-D, and
TPC-H standard benchmarks.
Oracle trace files with bind variables.
Imports from various tools such as Quest Performance Analysis SQL from the repository
for replay.
325
Quick Starts

After creating the RAC script, you can enhance your script by importing an Oracle trace file
workload by selecting the trace file, applying filtering options, and then replaying the file. When
Benchmark Factory imports the Oracle trace activity it maintains concurrency between
transactions and inter-transaction timings that simulate a production environment. Activity as it
happened in production can be simulated in a test database using an unlimited number of
concurrent users.
Up to this point, you have created the clustering profile and a script that contains the TPC-C and
TPC-H benchmarks. Next you import trace files required to run the job.
1. In the script created, select the Script node. This allows the Oracle trace file import load
scenario to be placed in its own job.
2. Select Wizards|Load Scenario.

3. Click Next.
7. Navigate to and select the appropriate trace file(s).
326
Quick Starts
15. Check the Add to existing Benchmark Factory Script file radio button.
17. Click Finish. The trace files create and display in the Script View.
Running the Job

Since the Benchmark Factory node balancing method was selected in the RAC wizard, each job
has been assigned a profile specifically designed to test specific nodes by adding them to the
testing process. Benchmark Factory sets up a load test with a single instance and individual jobs
for groups of nodes as they are added to the testing process. This allows you to first run the
single instance node and gather results. Then you can tune that nodes and make the same
changes to your other nodes. Next you can add your nodes to the testing process in small groups,
allowing you to easily see if issues occur due to the additions of a group of nodes.
1. Select the Single Instance node.

Quick Starts

3. Select Submit Job. The Submit Job Dialog displays.
following options:
327
328
Quick Starts
Section
Description
Profile
Actions
Allows you to:

Comments
Alerts

operator alerts.
Job Counters

error

6. To run the jobs designed to test the remaining nodes created for this test, repeat steps 1-6.
Benchmark Factory provides a wide array of graphs to interpret your RAC testing results.
Individual nodes testing results can be viewed for comparison.
1. Select Tools Run Reports. Run Reports displays.

Quick Starts
2. You can select a single test or multiple test to view testing results. In this example,
multiple tests are selected to view the RAC testing results.
329
330
Quick Starts
3. Click Show Test Results. Run Reports Displays.
4. To view RAC testing results for the nodes selected, click Cluster Performance Graph.
5. To view other testing results, click the desired result from the User Load drop-down.
331
Quick Starts
Upgrading Your Database Server Quick Start

Microsoft SQL Server 2005 entry into the market forces many IT professionals to look at the
upgrading of their current SQL Server 2000 Databases. IT organizations must perform testing
before considering any major upgrades. Determining the capacity of a Microsoft SQL Server
2000 platform and comparing that to the performance of SQL Server 2005 is essential. Such tests
help determine when an older version of your database will fail to scale, or become unstable,
enabling you to compare those shortcomings to the newer versions of the database software.
Before upgrading, three important questions must be asked:
l
Are you better able to meet important business requirements such as higher availability,
better scalability, or better performance?
Can you extract greater value out of the system by running it on the newest technology?
Are you able to maintain the old application more efficiently than before?
This quick start presents a load testing methodology to compare your current database servers
with newer versions to determine if upgrading helps you obtain better performance.

l
Creating a SQL Server 2000 Trace File using SQL Server 2000 SQL Profiler
Importing the Trace File Into Benchmark Factory for Databases
Setting User Load
Running the Load Test Against the SQL Server 2000
Running the Load Test Against the SQL Server 2005
Monitoring CPU Usage During the Test
Viewing Testing Results
Creating the SQL Server 2000 Trace Files Using SQL Profiler
Trace files obtain data in which events are logged during the running of a database application.
You can then use the trace table to play back the logged events. You may have a trace table
readily available, or can use Microsoft SQL Profiler to create a trace table.
For procedures on creating SQL Server 2000 trace files, please refer to:
l
Importing the Trace Table Into Benchmark Factory for Databases

Once you have created the trace table, the next step is to import that trace table into Benchmark
Factory for Database for the creation of a load scenario.
332
Quick Starts

To Benchmark Factory
Start if desired.

4. Click Next. The Select load scenario type dialog displays.
5. Select Replay load from SQL Server trace table.
6. Click Next. The SQL Server Authentication dialog displays.
7. Select the desired host from the Select a host drop-down.
8. Select the desired connection method.
9. Enter the user name and password if required.
10. Click Next. The SQL Server Trace Table dialog displays.
11. Select the desired the desired database from the List of SQL Server databases on the
selected host.
12. Select the desired trace table from List of SQL Server tables on the above database.
13. Click Next. The selected trace table processes.
14. When the processing completes, click Next. The Enter Script Name dialog displays and
provides the following options:
15. Select Create a new Benchmark Factory Script file.
16. Enter the desired name in Name or accept the default.
17. Click Next. The completion dialog display. Do not check the Do you want the job to be
automatically submitted to Jobs Queue.
18. Click Finish. The job creates and displays in the Script View.
Setting User Load
After importing the trace file, the next step is to set the timing and user load for the test.
1. In the Script view, select the load scenario replay that contains the imported trace table.
2. Right-click.
3. Select Copy to Mix Test. The Mix Test creates.
333
Quick Starts
4. Select the originally created replay.
5. Right-click.
6. Select Remove.
7. Click Yes when asked, "Are you sure you want to delete the selected item(s)?
8. Select the newly created Load Scenario.
9. In the properties view, select the Timing Tab. For this example, a 2:00 sampling length
is selected.
334
Quick Starts
10. In the properties view, select the User Load tab. For this example, user loads of 1 to 3000
are selected, with intervals of 250 Users.
Running the Load Test on the SQL Server2000

Next, you run the workload on the 2000 SQL Server.
1. In the Script View, select the job.
2. Select Wizards | Submit Job. The Submit Job dialog displays.
335
Quick Starts
following options:
Section
Description
Profile
Actions
Allows you to:

Comments
336
Quick Starts
Section
Description
Alerts

operator alerts.
Job Counters

error
Note: You must select the 2000 SQL Server profile.

Running the Load Test on the SQL Server 2005
Next, you run the trace file on the 2005 SQL Server.
1. In the Script View, select the same job you used to run against the SQL 2000 server.
(Imported trace file.)
2. Select Wizards Submit Job. The Submit Job dialog displays.
following options:
337
Quick Starts
Section
Description
Profile
Actions
Allows you to:

Comments
Alerts

operator alerts.
Job Counters

error
Note: You must select the 2005 SQL Server profile.

Monitoring Server Throughput During the Test
As you run the test on the SQL Server, you can monitor the database using Spotlight on SQL
Sever, to discover any issues that affect performance.
Viewing Testing Results
After running the test, you can use Benchmark Factory Run Reports to view testing results to see
how SQL Server 2000 compared to SQL Server 2005. The transaction per second comparison
between the two servers can be viewed. Figure 3 compares Transaction/Second vs. User Load
graph of the 2000/2005 test. Run 13 (blue) represents the SQL Server 2000 and Run 16 (green)
338
Quick Starts
represents the SQL Server 2005. Note that in this example, SQL Server 2005 dramatically
outperformed SQL Server 2000.
Figure 3-Comparing SQL Server 2000 to SQL Server 2005 Transaction/Second

The following graphic compares the Transaction Time between SQL Server 2000 and SQL
Server 2005. Note that in this example, the SQL Server 2005 dramatically outperformed SQL
Server 2000.
339
Quick Starts
Oracle Trace Import-Database Replay Quick Start

Overview
This quick start presents a load testing methodology to import and run an Oracle trace file
against the AS3AP benchmark, then view testing results. Using trace files in a database load test
allows you to run recorded production activity on a test database that includes concurrency
between sessions, session activity, and latencies that include bind variables.

l
Accessing the Benchmark Factory Script and Oracle trace files
Creating the AS3AP tables
Creating a profile
340
Quick Starts
Importing the Oracle trace files
Submitting the Job to the Jobs Queue
Viewing run reports
Saving the script
Accessing the Benchmark Factory script and Oracle Trace Files for this
Quick Start
Industry standard benchmarks have been developed to measure system performance. Each
benchmark has specific system configuration requirements that include tables, indexes, and data.
For this Quick Start, we will use the AS3AP Benchmark to create the environment to run the test
on. The Oracle Trace File Setup.bfp, OraDemo_2.trc, and OraDemo_2.trc files are provided to
create the AS3AP tables and run the load test. The following files are located in C:\Program
Files\Quest Software\Benchmark Factory for Databases\Quick Starts:
l
Oracle Trace File Setup.bfp
OraDemo_1.trc
OraDemo_2.trc
Creating the AS3AP Tables

The Oracle Trace File Setup.bfp has been provided to create the tables needed to run the job
created by the Oracle trace files.
Start if desired.

3. Click File | Open.
4. Navigate to C:\Program Files\Quest Software\Benchmark Factory for Databases\Quick Starts.
5. Select Oracle Trace File Setup.bfp.
6. Click Open. The Oracle Trace File Setup opens and displays in the script view.
7. Next, you must create the AS3AP tables to run the job. Select the AS3AP Tables job.
341
Quick Starts
8. Select the AS3AP Tables job.

9. Select Wizards | Submit Job. The Submit Job dialog displays.
The Submit Job dialog allows you to tailor your job needs and provides the following options:
Section
Description
Profile

Actions
Allows you to:
342
Quick Starts

Comments
Alerts

operator alerts.
Job Counters

error
10. Fill in the desired parameters.

11. Click Submit. The load test is submitted to the Jobs Queue and runs.
Importing the Oracle Trace Files

After creating the AS3AP tables, you create the Oracle trace file script.
1. Click the script tab located at the bottom of the Jobs Queue.
3. The Load Scenario Wizard displays. The Load Scenario Wizard is a starting point for
creating Benchmark Factory scripts and walks you through the steps to build a complete
workload testing script. Click Next.
5. Click Next. The Oracle 8i/9i/10g/11g trace file(s) to import window displays.
7. Navigate to the Benchmark Factory Quick Start directory. (C:\Program Files\Quest
Software\Benchmark Factory for Databases\Quick Starts.)
8. While holding down the Ctrl key, click on the OraDemo_1.trc and OraDemo_2.trc.
9. Click Open. The trace files display under the Trace File Name column.
10. Click the Next button. The Oracle Trace Activity window displays.
11. Click the Import entire trace file(s) radio button.
12. Click Next. Select the Create a New Benchmark Factory Script file radio button.
13. Enter the desired name, such as Oracle Trace File Import.
15. Uncheck the "Do you want the job to be automatically submitted to the jobs

Quick Starts
Queue?" check box.

16. Click Finish. The Submit Job Wizard displays.
Submitting the Job to the Jobs Queue

The job can now be submitted to the jobs queue.
1. Select the Jobs node.
2. Right-click and select Submit Job. The Submit Job dialog displays.
following options:
Section
Description
Profile
343
344
Quick Starts
Section
Description
Actions
Allows you to:

Comments
Alerts

operator alerts.
Job Counters

error

4. Click Submit. The load test is submitted to the Jobs Queue and runs.
Note: For user loads of 20 users or less, the Agent is not required.
345
Quick Starts

1. Click the Run Reports node for the Oracle Trace Import job.
3. Double-click. Run Reports opens to the Run ID node. This view contains a summary of
job statistics.
4. Expand Summary Graphs node. Summary graphs provide a wide array of graphs that
5. Expand the Results node. Results provides information on individual user load and agent
6. Close Run Reports when finished reviewing the job.
Saving the Script
1. Click the script tab in the Script View.
4. Click Save.
346
Quick Starts
Standard Benchmark Quick Start

Overview
This quick start presents a load testing methodology to run an AS3AP benchmark on a test
database, then view real-time and stored testing results to determine if your database lives up to
the required performance parameters.
A benchmark is a performance test of hardware or software. A benchmark simulates real-world
application workloads. These models of real applications can then be run on the system being
evaluated. Models works better than the actual application and offers reproducible results. The
real application, on the other hand, has too many variables that can change the results over
several test sessions.
Benchmarks measure performance that analyze:
l
Raw performance of a complete system
Create a profile
Raw performance of a specific subsystem (disk, video, CPU, memory, etc.)
Performance of a computer running a particular application
Performance of applications running on a network
Capacity of a system. This is often referred to as capacity planning.
Actions Performed Duirng this Quick Start

l
Creating a database load test script
Create a profile
Setting the benchmark scale and user load
Submit a Job to the Job Queue
View real-time results
View run reports
Save the Script
Creating a Database Load Test Script

Start if desired.
347
Quick Starts
3. Select Wizards|Load Scenario. The Load Scenario Wizard displays. The Load Scenario
Wizard is the starting point for all Benchmark Factory scripts. The New Script Wizard
5. Click Industry standard benchmarks (TPC-C, TPC-H...). Benchmark Factory
provides industry Standard Benchmarks that allows you to populate test databases
according to a selected industry standard benchmark and measures the performance of
a system-under-test.
Benchmark Factory provides the following standard benchmarks:
l
AS3AP
Scalable Hardware
TPC-B
TPC-C
TPC-D
TPC-H
6. Click Next. The Select Benchmark dialog displays. This dialog displays the standard
benchmarks that come with Benchmark Factory. For a complete overview of all
Benchmarks, please refer to Benchmark Overview.
7. Click the AS3AP benchmark.
348
Quick Starts
Creating a Profile
1. If no profile has been created, click New. The Profile Creation Wizard displays. If a
profile has been created, select the desired profile from the Profile drop-down.
2. Click Next. The Benchmark Scale dialog displays.
Setting the Benchmark Scale and User Load
1. Benchmark Factory allows you to set the desired benchmark scale to perform realistic
load tests. Select the desired benchmark scale in Benchmark Scale.
3. Add the desired User Load.
l
5. Select the Create a new Benchmark Factory script radio button.

6. Enter the desired name or accept the default.
8. Check the Do you want the job to be automatically submitted to the jobs Queue?
check box.
9. Click Finish. The Submit Job dialog displays.
349
Quick Starts
following options:
Section
Description
Profile

Actions
Allows you to:

Comments
Alerts

operator alerts.
350
Quick Starts
Job Counters

error

During the running of a database load test, real-time statistics provide insight into the
performance of the database being tested. Numerous datapoints allow you to examine exactly
how your database is performing.
Note: For procedures on changing Benchmark Factory graph views, please refer to Changing
Graph Views.
1. Click the Message node on the Jobs Queue tab. The Message node displays Job status,
error messages, and general information.
running in the Jobs Queue.
graphs and raw data. This data allows you to spot system-under-test issues that may be
Right-clicking inside the graph displays a drop-down that allows you to change graph
settings and view.
4. Click Click here to Add Datapoints.
5. Select the desired datapoints.
6. Click OK.
7. Right-click on the graph.
8. Select Graph Legend.
351
Quick Starts
10. From the Vs. Userload drop-down in the upper left-hand corner of the graph, select the
desired datapoint.
11. Click the Transaction Graph node. The transaction node displays properties based on
transaction type.
12. From the Vs. Userload drop-down in the upper left-hand corner of the graph, select the
desired datapoint.
352
Quick Starts

Factory and provides a comprehensive and detailed collection of load testing results. With
Benchmark Factory you can drill down into a database to view a wide array of information and
statistics that gives you accurate insight into database performance.
1. Click the Run Reports node.
353
Quick Starts
3. Double-click. Run Reports opens to the Run ID node. This view contains an summary of
load scenario statistics.
4. Expand Summary Graphs node. Summary graphs provide a wide array of graphs that
5. Expand the Results node. Results provides information on individual user load and agent
Saving the Script
After running a job, you can save a job to run again, or edit to run using different testing
parameters.
1. Click the Script tab in the Script View.
4. Click Save.

Goal tests are used to determine the maximum transaction per second (TPS) or userload of a
system. When performing a goal test, Benchmark Factory provides the following options:
Find maximum TPS: After each iteration completes, the TPS value is compared with the
previous two testing iterations. If the TPS value has dropped for the last two values, then the
maximum TPS has been discovered and the test stops. The maximum TPS is dictated by the
users system. Benchmark Factory submits the workload to the server and measures the TPS that
the server is experiencing for that workload. The users system dictates how fast the workload
will be process by the database server.
Find maximum userload where response time is less than [n] ms: This option allows you to
execute the transaction mix at each userload until the response time for an iteration is less than
the entered value.
Overview
This quick start presents a load testing methodology to run a goal test that finds the maximum
Transactions Per Second (TPS) of a database, by adding a SQL Statement Transactions to the
goal test, then viewing real-time and stored testing results to determine if your database lives up
to the required performance parameters.
354
Quick Starts
Goal Test. When run, the specified goal criterion is evaluated at the end of each iteration, and
the test ends once the goal or maxim user load has been reached.

l
Creating a blank goal testing script
Set script timing
Add a transaction to the goal test
Create Benchmark Objects
Submit the job to run
View Realtime Results
View Run Reports Results
Saving the Script
Creating a Blank Goal Testing Script

1. Start Benchmark Factory if required.
b. Select Programs Quest Software Benchmark Factory for Databases
Benchmark Factory for Databases. Benchmark Factory launches. The Run Quick
Start Dialog displays. Launch the Quick Start if desired.
3. Select Wizards | Load Scenario. The Load Scenario Wizard displays. The Load Scenario
Wizard is the starting point for all Benchmark Factory scripts. The Load Scenario Wizard
5. Click the Create a custom load scenario (Scalability, Goal...).
8. Click Next. The Goal Page displays. The Goal page allows you to:
l
Find maximum userload where response time is less than [n]ms: Selecting this
355
Quick Starts
10. Select the desired option. For this quick start, the Find maximum TPS is selected.
12. Click Next. The Define Transaction Mix Page displays.
3. Add to existing Benchmark Factory Script file-Selecting this option allows you
to add the load scenario you are building to the current script file you are
working with.
14. Click Create a new Benchmark Factory Script file.
16. Click Finish. The Scripts creates and displays in the Script View.
Setting Script Timing
356
Quick Starts

3. Enter the desired test length in Total Test Length, for example, 1 minute.
4. Move the slider bar to select the Pre-Sampling and Sampling times . ThePre-sampling time is the length of time
virtual users execute transactions A single unit of work in the testing process. For example, a Web page retrieval, a
SQL statement, or an email message. These are the components used to build a load scenario. in order to reach a
steady state before gathering statistics. Sampling time is the length of time used to collect statistics while virtual
users are executing transactions.
5. Click the Start all users as quickly as possible radio button.

Adding a Transaction to the Goal Test
357
Quick Starts
2. Select Wizards | Transaction. The Transaction Wizard displays.

4. Select Create SQL Statement Transaction.
6. Enter the desired SQL statement. In this quick start, select * from a_updates where akey
= $BFRand(30) is entered. As you type a SQL statement, the $BFScript Wizard displays
and auto enters a BFScript.

8. Select Keying Time.
9. From the Keying Time drop-down select Normal.
358
Quick Starts
10. In Mean enter 3000 milliseconds.

Creating Benchmark Objects
1. In order to create the a_updates table used in the SQL statement, Benchmark Objects must
be created. Click the Max TPS (1-5 by 1) job node.
3. Select Add Create/Delete Benchmark Objects. The Benchmark Objects Wizard displays.
359
Quick Starts
4. Click Next.
5. From the drop-down, select the AS3AP standard benchmark.
6. Click Next. The Benchmark Scale displays.
8. Click Next. The Create/Delete benchmark objects dialog displays.
10. Click Finish.
11. You must create the benchmark objects before running the script. Right-click Create
Objects for AS3AP.
12. Select Move Up.
13. Move the Benchmark Object to the top node in the script view.

Quick Starts
Submitting the job to run

1. Click the Max TPS(1-5 by 1) job node.
2. Select Wizards | Submit Job Wizard. The Submit Job dialog displays.
following options:
Section
Description
Profile
360
361
Quick Starts
Section
Description
Actions
Allows you to:

Comments
Alerts
Allows you to send an email, pager, or net-send operator

alerts.
Job Counters

error

1. Click the Message node. The Message node displays Job status, error messages, and
general information.
running in the Job's Queue.
graphs and raw data. This data allows you to spot system-under test issues that may be
362
Quick Starts
5. Click the Transaction Graph node. The Summary Graph displays. The transaction node
displays properties based on transaction type.
1. When the job completes, click the Run Reports node.

3. Double-click. The Run Reports opens to the Run ID node. This view contains an
summary of load scenarios statistics.
4. Expand the Summary Graphs node. Summary graphs provide a wide array of graphs that
5. Expand the Results node. Results provide information on individual user load and agent
Saving the Script
1. Click the Script tab.
4. Click Save.
14
Troubleshooting
Agent Connection
If the agent is having a problem connecting to the Benchmark Factory console, please check
the following
l
Verify that the console is running.
Verify that the agent is pointed to the console machine. Go to Options > Settings and
check the Machine Name/IP Setting.
Verify TCP connectivity between the Agent and Console. If there is a firewall in place,
allow for communication on Port #4568.
Use Benchmark Factory with SQL Server 2005

Client
Microsoft SQL Server 2005 does not include the DB-Library used by the Benchmark Factory
"Microsoft SQL Server (Native)" driver. Either the installed SQL Native ODBC driver or another
Microsoft SQL Server ODBC driver can be used with the Benchmark Factory "Microsoft SQL
Server (ODBC)" driver to connect to a "Microsoft SQL Server" database.
If you prefer to use the more optimized "Microsoft SQL Server (Native)" driver, a pre-SQL Server
2005 client can be installed to provide the DB-Library.
MySQL Initialization Settings

The MySQL repository database can take up a large amount of space on a system. The my.ini file
allows you to edit the initialization settings and adjust the size of the MySQL repository used by
Benchmark Factory. For additional information on setting MySQL database parameters, please
visit MySQL.com.
15
Appendix
Change Graph Views
Prints the graph


Appendix
Graph Legend
365

Appendix
Toolbar
Print


Load Configuration
Save Configuration
Set as Default
366
367
Appendix
Clear Configuration
Customize List Controls

Benchmark Factory allows you to customize list controls in columns to select the desired
viewing format.
To Customize Lists Controls
1. Right-click the desired column. A menu displays.
2. The following options are provided:

l
Sort Ascending: Sorts columns in ascending order.
Sort Descending: Sorts columns in descending order.
Customize Columns: Allows you to customize column headings.
Column Format: Allows you to customize column format.
Create a SQL 7 Trace Table Using the SQL Server

Profiler
To create a SQL 7 trace table
1. Initiate the SQL Profiler.
2. Click the New Trace icon. The Connect to SQL Server window displays.
3. Login to the SQL server. The Trace Properties window displays.
4. Type a name of a trace file into Trace Name.
5. Check the Capture To Table box. The Capture TO Table window displays.
6. Select a database name from the Database drop-down list.
368
Appendix
7. Select dbo from the Owner drop-down list.

8. Click OK.
9. Click on the Events tab.
10. Select TSQL from the Available events classes list.
11. Click the Add button.
12. Click the Data Columns tab.
13. Add the following columns (minimal):
14. EventClass
15. SQLUserName
16. ConnectionID
17. NTUserName
18. Duration
19. StartTime
20. Text
21. DatabaseID
14. Click OK.
15. Run SQL statements.
16. When finished running SQL statements, click the Stop this trace icon. The table creates.
Create a SQL 2000 Trace Table Using the SQL

Server Profiler
3. Select the appropriate SQL server from the drop-down list.
4. Check the Start SQL Server if it is stopped check box.
5. Select the appropriate Connect using radio button. Type the appropriate login name and
password if appropriate.
6. Click OK. The Trace Properties window displays.
7. Click the General tab.
8. Type a name of the trace into Trace name.
9. Check the Save to table check box. The Destination Table window displays.
369
Appendix
10. Select the appropriate database from the Database drop-down list.
11. Select the appropriate table from the Table drop-down list.
13. Click the Events tab.
14. Select TSQL from the Available events classes list.
15. Click Add.
16. Click the Data Columns tab.
17. Add the following columns (minimal):
15. EventClass
16. Login Name
17. ClientProcessID
18. NTUserName
19. Duration
20. StartTime
21. TextData
18. Click Run.
20. When finished running SQL statements, click the Stop selected trace icon. The
table creates.
Create a SQL 2005 Trace Table Using the SQL

Server Profiler
3. Select the appropriate server name from the drop-down list.
4. Select the appropriate authentication from the drop-down. Type the login name and
password if required.
5. Click Connect. The Trace Properties window displays.
6. Type a name of the trace into Trace name.
7. Check the Save to table check box. The Connect to Server dialog displays.
8. Select the appropriate server name from the drop-down list.
370
Appendix
9. Select the appropriate authentication from the drop-down. Type the login name and
password if required.
10. Click Connect. The Destination Table window displays.
11. Select the appropriate database from the Database drop-down list.
12. Select the appropriate owner from Own drop-down.
13. Select the appropriate table from the Table drop-down list.
15. Click the Events Selection tab.
16. Verify that all check boxes are checked in the TSQL-SQL:BatchStarting Events, and
TSQL-SQL:BatchComplete Events.
17. Click Run.
19. When finished running SQL statements, click the Stop selected trace icon. The table
creates.
Oracle Instant Client Installation

To run Benchmark Factory, you must have the client dll files installed.
To install the Oracle 10g Instant Client dlls
1. Place Oracle 10g instant Client on your system following the directions provided by
Oracle. For these instructions, please visit the Oracle 10g Web site.
2. Copy tnsnames.ora from a configured system to your system.
3. Edit your systems Environment Variables using one of the following methods:
4. Add to the path variable the directory the Instant Client files were copied to.
5. Add a new System variable Named TNS_Admin, and place the directory location of
the tnsnames.ora file in the value field.
Note: Environment Variables can be reached by right clicking on the My Computer Icon,
selecting properties, select Advanced tab, and select the Environmental Variables button.
371
Appendix
Migrating Repository Data Using the DOS

Command Line
The Repository is a database where all of the test results are stored. Benchmark Factory inserts
test results into the repository and provides an easy way to access the data. By default, the
Databases. The Repository can reside on another database server if required. To change the
database, select the Repository Manager or the repository tab in Settings.
Benchmark Factory allows you to migrate data from the DOS command line.
To migrate data using the DOS command line
1. Click Start | All Programs | Accessories | Command Prompt.
2. Navigate to the Benchmark Factory for Databases bin directory. Example: CD C:\Program
Files\Quest Software\Benchmark Factory for Databases\bin.
3. To display the syntax help, type:
bmfdatamigrationWizard.exe -?.
4. Press Enter. The usage parameters display.
5. To perform the migration use the following syntax:

BMFDataMigrationWizard -s [DSN,USER,PWD] -d [DSN,USER,PWD]
Example:
BMFDataMigrationWizard -s [MyDatabase,root,yourpassword] -d
[LocalServer,sa,sa]
372
Appendix
Store Procedure Examples

The following provides syntax examples of statements that call stored procedure. Click here for
procedures on creating a load scenario by importing stored procedures.
Database
Oracle
Examples
Example with parameters:
BEGIN
YOUR_PROC(:VAR1,:VAR2);
END;
Bind Parameter Settings
Parameter
:VAR1
:VAR2
Type
INTEGER
INTEGER
Value
5
8
Example without parameters:

BEGIN
YOUR_PROC2 ();
END;
Microsoft SQL Server
and Sybase

exec YOUR_PROC(@VAR1,@VAR2)
Parameter
@VAR1
@VAR2
Type
INTEGER
INTEGER
Value
55
77

exec YOUR_PROC2
IBM DB2, ODBC,
373
Appendix
and MySQL
call procedure-name (?,?,?,?);
Parameter
1
2
3
4
Type
STRING
INTEGER
INTEGER
INTEGER
Value
A
5
7
0

call procedure-name ();
Note: When using an ODBC connection, the syntax for the
statement will follow the syntax for the database, with the
exception of the parameters. The parameters within the statement
would be "?" and the bind parameters names should be 1,2,3,4...
Oracle Trace File Activation

Benchmark Factory allows you to import an Oracle trace files for use during database testing.
The following provides Oracle commands that can be used to enable Oracle trace file activity.
Note: Please consult Oracle documentation for current trace file commands.
To turn on and off the trace use the ALTER SESSION command:
1. Set the TIMED_STATISTICS and MAX_DUMP_ FILE_SIZE parameters used by
the sessions:
alter session
set timed_statistics=true
alter session
set max_dump_file_size=unlimited
2. Activate the trace:

alter session set events
'10046 trace name context forever, level 4'
3. Stop the trace:
374
Appendix
alter session set events

'10046 trace name context off'
To activate a trace you don't have read/write access to:

1. Acquire the V$SESSION.SID and V$SESSION.SERIAL# values of the session you
wish to trace.
2. Set the TIMED_STATISTICS and MAX_DUMP_FILE_SIZE parameters for the session
utilizing the following procedure calls:
dbms_system.set_bool_param_in_session(
sid => 55,
serial# => 1222,
parnam => 'timed_statistics',
bval => true)
dbms_system.set_int_param_in_session(
sid => 55,
serial# => 1222,
parnam => 'max_dump_file_size',
intval => 2147483647)
Note: If using a release before Oracle 8, release 8.1.6, these parameters can be changed with
ALTER SYSTEM commands.
3. Activate the trace.
dbms_support.start_trace_in_session(
sid => 55,
serial => 1222,
waits => true,
binds => true)
4. Stop the trace.

dbms_support.stop_trace_in_session(
sid => 55,
serial => 1222)
375
Appendix
Support Bundle
The Benchmark Factory Support Bundle allows you to zip up a .bfp file and send it to support
for review. The bundle contains the following information:
l
BMF version number
Settings
License information (just send the key files)
Error Logs and Result Logs from the associated directories
Files located in the data directory. This will be not only the xml files for imported users
scenarios, but also dump files.
Script files XML configuration files
A file that contains hardware information about the system running on it as well as the
output of the agents configurations from the repository
Information about versions of all loaded .dlls
To create a Benchmark Factory support bundle:

1. Select Help| Support Bundle.
2. Select the required modules to send.
3. Click Bundle.
The bundle is created and saved in:
376
Appendix
C:\Documents and Settings\"User Name"\My Documents\My Benchmark

Factory\BMFSupportBundle.zip.
System Requirements/Upgrade
Requirements/Supported Databases
System Requirements
Before installing Benchmark for Databases , ensure your system meets the following minimum
hardware and software requirements.
Repository
Benchmark Factory cannot run without a repository, and by default a MySQL repository is
installed during the install process. After installation, you can remove the MySQL service and
delete the MySQL repository, then select a repository of your choice. Benchmark Factory
supports the following databases:
l
Oracle
SQL Server
SQLite
MySQL
Sybase Adaptive Server Anywhere (iAnywhere)
Memory
512 MB of RAM recommended
Note: The memory required may vary based on the following:
l
Applications that are running on your system.
Size and complexity of the database.
Amount of database activity.
Number of concurrent users accessing the database.
Platform
1.0 GHz x86 or x64 compatible CPU
377
Appendix
Hard Disk Space

120 MB of disk space for installation Note: The disk space required may vary based on options
selected for installation.
Operating System
Microsoft
l
Windows 2000
Window 2003
Windows 2008
XP Professional
Vista
Windows 7
Database Server
Oracle: 8, 8i, 9i, 9.2, 10g, and 11g (Requires Oracle 8.1.7 client libraries or higher.)
Note: Oracle Capture/Replay of workloads must use Oracle 9i or higher.
SQL Server: 7, 2000, 2005, and 2008
Sybase: 12.5 and 15
DB2 LUW: 8.1.5 through 9
MySQL: 4.5 and 5.x
Note: All client libraries require the 32-bit versions.
Upgrade Requirements
l
Client libraries for database types used during the workload testing process must be
installed on all testing machines (Benchmark Factory and Agents).
The Benchmark Factory 3.3 and below files can be converted to Benchmark Factory for
Databases 4.x files. A script conversion program is available, and can be accessed by
logging on to Quest Software, then select Downloads and Upgrades | Benchmark
Factory for Databases| Special Patches| Benchmark Factory Project Converter.zip.
There is no upgrade path for the Benchmark Factory Repository version 3.3 or below.
If you create a new Benchmark Factory 5.5 repository, earlier versions of Benchmark

Appendix
Shortcut Keys
The following provides a list of shortcut keys used in Benchmark Factory.
Key
Action
ALT+1
Displays the Output View
ALT+2
Displays the Agent window
ALT+M
Creates an email message with the current script attached
ALT+R
Runs a job
ALT+S
Stops a job
CTRL+B
Displays the Benchmark Objects Wizard
CTRL+C
Copies the selected item in the Script View
CRTL+D
Saves a job back to the Script View
CTRL+E
Readies a job
CTRL+F
Displays the Execute File Wizard (Script view only)
CTRL+J
Displays the Submit to Job Wizard (Script view only)
CTRL+L
Displays the Load Scenario Wizard (Script view only)
CTRL+N
Displays the Load Scenario Wizard (Script view only)
CTRL+O
Displays the Select Script File to Open dialog (Script view only)
CTRL+P
Displays the Edit Profile dialog
CTRL+R
Displays the Enter a New Name dialog (Script view only)
CTRL+S
Saves the current script (Script view only)
CTRL+T
Displays the Transaction Wizard (Script view only)
CTRL+U
Displays the User Scenario Wizard (Script view only)
CTRL+V
Paste an item from the clipboard (Script view only)
CTRL+W
Displays the Profile Creation Wizard
CTRL+X
Cuts a selected item (Script view only)
378

Appendix
Key
Action
CTRL+ ALT+E
Readies all jobs
CTRL+H
Holds a job
CTRL+ALT+H
Holds all jobs
Delete
Deletes the selected item (Script view only)
END
Selects the end of the item with focus
HOME
Selects the beginning of the item with focus
LEFT ARROW
Collapses current selection if it is expanded, or select parent folder
RIGHT
ARROW
Expands current selection if it's collapsed, or select first subfolder
F1
Displays online help
F8
Displays the Setting dialog
F12
Displays the Save As dialog
PLUS SIGN on
numeric keypad
(+)
Expands the selected folder
LOCK+MINUS
SIGN on
numeric keypad
(-)
Collapses the selected folder
379
16
Copyright, Licensing, and Licensing
Agreement
Copyright
2012 Quest Software, Inc.
ALL RIGHTS RESERVED.
This guide contains proprietary information protected by copyright. The software described in
this guide is furnished under a software license or nondisclosure agreement. This software may be
used or copied only in accordance with the terms of the applicable agreement. No part of this
guide may be reproduced or transmitted in any form or by any means, electronic or mechanical,
including photocopying and recording for any purpose other than the purchasers personal use
without the written permission of Quest Software, Inc.
The information in this document is provided in connection with Quest products. No license,
express or implied, by estoppel or otherwise, to any intellectual property right is granted by this
document or in connection with the sale of Quest products. EXCEPT AS SET FORTH IN
QUEST'S TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FOR
THIS PRODUCT, QUEST ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS
ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTS
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT. IN NO EVENT SHALL QUEST BE LIABLE FOR ANY DIRECT,
INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES
(INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESS
INTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE OR
INABILITY TO USE THIS DOCUMENT, EVEN IF QUEST HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES. Quest makes no representations or warranties with respect
to the accuracy or completeness of the contents of this document and reserves the right to make
changes to specifications and product descriptions at any time without notice. Quest does not
make any commitment to update the information contained in this document.
If you have any questions regarding your potential use of this material, contact:
Quest Software World Headquarters
LEGAL Dept
5 Polaris Way
email: legal@quest.com
Refer to our Web site (www.quest.com) for regional and international office information.
381
Patents
Toad for Cloud Databases contains patent pending technology.
Trademarks
Quest, Quest Software, the Quest Software logo, Benchmark Factory, Spotlight, LiteSpeed,
Simplicity at Work, Toad, TOAD, T.O.A.D., and TOADWORLD and vToadare trademarks and
registered trademarks of Quest Software, Inc. For a complete list of Quest Softwares trademarks,
see http://www.quest.com/legal/trademark-information.aspx. Other trademarks and registered
trademarks used in this guide are property of their respective owners.

User Guide
September 2012
Licensing and Virtual User Packs

The Benchmark Factory for Databases Licensing dialog allows you to modify the license and
virtual user packs.
To access the licensing dialog:
1. Click Help | Licensing.

From this dialog you can:

l
Modify the Benchmark Factory License (page 382)
Modify the Benchmark Factory Virtual User Pack (page 383)
Modify the Benchmark Factory License

If required, you can modify your Benchmark Factory license key.
To to modify the license key:
1. Click the Benchmark Factory tab.
2. Click Modify.
3. From the License Type drop-down, select the required Quest product. The following
licenses can be updated:
l
Toad for Oracle
Quest Central for Oracle Load Generator key
Quest Central for SQL Server Load Generator key
Quest Central for DB2 Load Generator key
382
383
4. Click Edit Key.

5. Update the Authorization Key and Site Message.
6. Click OK.
7. Click Close.
Modify the Benchmark Factory Virtual User Pack

Benchmark Factory comes with 100 virtual users. Benchmark Factory Freeware comes with 20
virtual users. To add up to an unlimited number of virtual users, enter a virtual user key. To
enter one of the following key types, select the desired key from the drop-down and click
the Edit key.
Click here to learn more about adding virtual users.
To to modify the VirtualUser Pack:
1. Click the Virtual User Pack tab.
a. To send an email to Quest Support to extend a trial, click Extend Trial.
b. To visit the Quest web site and purchase a license, click Purchase License.
c. To modify the currant license, click Modify.
d. Click Virtual User Key.
e. Enter the authorization key and site message.
f. Click OK.
g. ClickOK.
h. Click Close.
Legal Notice
Privacy Policy for Optional User Registration
How to opt out
If you registered at the time of installation and you no longer wish to receive monthly
newsletters, you may opt out of receiving them by following the instructions included in each
newsletter or communication or by updating your preferences here: www.quest.com/unsubscribe.
What data is being collected?
384
When you register you must supply your email address and job role. You may also provide your
name and phone number but this is not required. All this information will be stored locally and
also be sent to Quest Software in Aliso Viejo, California. Whether or not you choose to register,
we will collect your license number for tracking purposes.
Where the data will be stored?
All the submitted data will be stored locally on the computer where you have installed the
software and in servers located at the Aliso Viejo, California facility in the United States.
How the data will be used?
The information that we collect will help Quest Software to deliver a more consistent and
personalized customer support and marketing experience. For example, we may use your
information to:
l
Communicate with you about products and services
Provide and improve service and support
Update you on new services and benefits
Provide personalized promotional offers
Select content to be communicated to you
Allow you to participate in contests and surveys
Contact you regarding Quest products or services that we think will be useful to you
How to operate the product without participating

The installation and operation of this product is not impacted by whether or not you register.
You may install any product and all the functionality provided by your license will be available.
How to operate the product without participating
The installation and operation of this product is not impacted by whether or not you register.
You may install any product and all the functionality provided by your license will be available.
Transaction Product Agreement
385
PLEASE READ THIS AGREEMENT CAREFULLY BEFORE USING THIS PRODUCT.

BY DOWNLOADING, INSTALLING OR USING THIS PRODUCT, YOU INDICATE
ACCEPTANCE OF AND AGREE TO THE TERMS AND CONDITIONS OF THIS
AGREEMENT. FOR ORDERS PLACED OUTSIDE OF THE UNITED STATES, PLEASE
SEE WWW.QUEST.COM/LICENSEAGREEMENTS FOR THE APPLICABLE
LOCALIZED VERSION OF YOUR AGREEMENT. IF YOU DO NOT AGREE TO THE
TERMS AND CONDITIONS OF THIS AGREEMENT OR THE LOCALIZED
AGREEMENT, DO NOT DOWNLOAD, INSTALL OR USE THIS PRODUCT. NEITHER
THIS AGREEMENT NOR THE LOCALIZED AGREEMENT SHALL SUPERSEDE ANY
OTHER SIGNED AGREEMENT BETWEEN YOU AND QUEST THAT EXPRESSLY
GOVERNS THE ORDER FOR THIS PRODUCT.
This Transaction Product Agreement (the Agreement) is made between Quest Software, Inc.,
with its principal place of business located at 5 Polaris Way, Aliso Viejo, California, 92656
(Quest) and you, the customer (Customer).
1. Definitions. Capitalized terms not defined in context shall have the meanings assigned
to them below:
(a) Affiliate means any legal entity controlling, controlled by, or under common control with a
party to this Agreement, for so long as such control relationship exists.
(b) Documentation means the user manuals and documentation that Quest delivers with the
Software, and all copies of the foregoing.
(c) Hardware means the hardware products purchased by Customer under this Agreement.
(d) License Type means the model by which the Software is licensed (e.g., by server, by
mailbox, by managed user) as may be indicated in the applicable Order.
(e) Partner means a reseller or distributor that is under contract with Quest or another Partner
and is authorized via the contract to resell the Products and/or Maintenance Services.
(f) Product Guide means the document located at http://www.quest.com/productguide that
contains the Product Terms.
(g) Product Terms means the usage rights and other terms associated with each License Type or
individual Product. The Product Terms for Software identified in an Order that is issued to Quest
shall be as stated in the Order, or, if no Product Terms are stated in the Order, then the Product
Terms for such Software shall be as stated in the Product Guide as of the date of the Order. The
Product Terms for Software ordered from a Partner shall be as stated in the Product Guide as of
the date of the Order.
(h) Products means the Software licenses and/or Hardware purchased by Customer under
this Agreement.
(i) Order means the document by which Customer orders the Product(s) or, if Customer orders
the Product(s) from Quests eStore (https://estore.quest.com), the process by which Customer
orders the Product(s). Orders executed by Customer and Quest shall be governed solely by the
386
terms of this Agreement and the applicable Order. Orders placed with Quest by Customer
purchase order only and all Orders placed through a Partner shall be governed solely by the
terms of this Agreement. Any conflicting or additional terms in or accompanying an Order will
not be binding on Quest unless Quest accepts such terms in writing. Each Order shall be the
Customers irrevocable commitment to purchase and pay for the Products and/or Maintenance
Services stated in the Order.
(j) Software means the object code version of the software that is delivered pursuant to an
Order as well as any corrections, enhancements, and upgrades to such software that Quest may
provide to Customer pursuant to this Agreement, and all copies of the foregoing.
2. Software License.
(a) Internal Use License. Subject to the terms of this Agreement, Quest grants to Customer, and
Customer accepts from Quest, a perpetual (unless otherwise set forth in an Order), non-exclusive,
non-transferable (except as otherwise set forth herein) and non-sublicensable license to (i) install,
execute, access, run, or otherwise use the quantities of each item of Software identified in the
applicable Order within the parameters of the Product Terms associated with the applicable
Product and License Type, (ii) make a reasonable number of additional copies of the Software to
be used solely for non-productive archival or passive disaster recovery purposes, so long as
neither the original and a copy nor two copies of the same Software are used at the same time,
and (iii) make and use copies of the Documentation as reasonably necessary to support
Customers authorized users in their use of the Software (collectively, License). Except for MSP
Licenses (as defined below), each License shall be used by Customer solely to manage its own
internal business operations as well as the business operations of its Affiliates.
(b) MSP License. If an MSP License is specifically identified in an Order, Customer shall be
granted a License to use the Software identified in the Order and the associated Documentation
as a managed service provider (MSP) to provide software and systems management services,
including, without limitation, application, operating system, and database implementation,
performance tuning, and maintenance services (Management Services), for the benefit of a
single named client (Client), pursuant to the terms of this Agreement and the MSP Use Terms
in the Product Guide.
(c) Evaluation License. If an Order indicates that Software is to be used for evaluation purposes
or if the Software is otherwise obtained from Quest for evaluation purposes, Customer shall be
granted a non-production License to use such Software and the associated Documentation solely
for Customers own internal evaluation purposes for an evaluation period of up to thirty (30)
days from the date of delivery of the Software, plus any extensions granted by Quest in writing
(the "Evaluation Period"). There is no fee for Customers use of the Software for non-production
evaluation purposes during the Evaluation Period, however, Customer is responsible for any
applicable shipping charges or taxes which may be incurred, and any fees which may be
associated with usage beyond the scope permitted herein. Customers opportunity for a free
evaluation of the Software is limited to one Evaluation Period per release of the Software.
Notwithstanding anything otherwise set forth in this Agreement, Customer understands and
agrees that evaluation Software is provided AS IS and that Quest does not provide a Warranty
or Maintenance Services for evaluation Licenses.
387
(d) Third Party Use. If Customer contracts with a third party who performs Software
implementation, configuration, consulting or outsourcing services (Service Provider), the
Service Provider may use the Software and Documentation Licensed by Customer hereunder
solely for purposes of providing such services to Customer, provided that (i) Customer ensures
that the Service Provider uses the Software and Documentation in accordance with the terms of
this Agreement, (ii) the use of the Software and Documentation by the Service Provider will not
violate the terms of the export restrictions set forth herein, and (iii) the Service Provider is not a
Quest competitor. Customer shall be jointly and severally liable to Quest for the acts and
omissions of its Service Providers in connection with their permitted use of the Software and
Documentation.
(e) Freeware. If a freeware version of Quest software (Freeware) is downloaded by Customer
from http:///a Quest website, the terms of such use shall be governed by the applicable Freeware
definition provided at: www.quest.com/productguide.
3. Restrictions. Except to the extent expressly permitted by applicable law, and to the extent
that Quest is not permitted by such applicable law to exclude or limit the following rights,
Customer may not reverse engineer, decompile, disassemble, or attempt to discover or modify in
any way the underlying source code of the Products, Documentation or any part thereof. In
addition, Customer may not (i) modify, translate, localize, adapt, rent, lease, loan, create or
prepare derivative works of, or create a patent based on the Products, Documentation or any part
thereof, or (ii) resell the Products or Documentation or use the Products or Documentation in any
commercial time share arrangement, in connection with the operation of any nuclear facilities, or
for purposes which are competitive to Quest. Each permitted copy of the Software and
Documentation made by Customer hereunder must contain all titles, trademarks, copyrights and
restricted rights notices as in the original. Customer understands and agrees that the Products may
work in conjunction with third party products and Customer agrees to be responsible for
ensuring that it is properly licensed to use such third party products. Notwithstanding anything
otherwise set forth in this Agreement, the terms and restrictions set forth herein shall not prevent
or restrict Customer from exercising additional or different rights to any open source software
that may be contained in or provided with the Products in accordance with the applicable open
source licenses.
4. Reservation of Rights and Ownership. Quest reserves any and all rights, implied or
otherwise, which are not expressly granted to Customer in this Agreement. Customer understands
and agrees that (i) the Products are protected by copyright and other intellectual property laws
and treaties, (ii) Quest and/or its suppliers own the title, copyright, and other intellectual
property rights in the Products, (iii) the Software is licensed, and not sold, and (iv) this
Agreement does not grant Customer any rights to Quests trademarks or service marks.
5. Hardware. In the event Customer acquires Hardware under this Agreement, title to such
Hardware shall pass to Customer upon shipment (unless such Hardware is rented, leased or
loaned to Customer).
6. Payment. Customer agrees to pay to Quest (or, if applicable, the Partner) the fees specified in
each Order, including any applicable shipping fees. Customer will be invoiced promptly
following delivery of the Products or prior to the commencement of any Renewal Maintenance
Period and Customer shall make all payments due to Quest in full within thirty (30) days from
the date of each invoice or such other period (if any) stated in an Order signed by Quest. Any
388
amounts payable to Quest by Customer that remain unpaid after the due date shall be subject to
a late charge of 1.5% of the invoice amount per month from the due date until such amount is
paid, or the maximum rate permitted by law if less.
7. Taxes. The fees stated in an Order may not include taxes. If Quest is required to pay sales,
use, property, value-added or other taxes based on the Products or Maintenance Services
provided under this Agreement or on Customers use of Products or Maintenance Services, then
such taxes shall be billed to and paid by Customer. This Section does not apply to taxes based
on Quests income.
8. Termination. This Agreement and/or the License(s) granted hereunder may be terminated (i)
by mutual agreement of Quest and Customer, (ii) by Quest, if Customer or a Service Provider
commits a material breach of this Agreement and fails to cure such breach to Quests reasonable
satisfaction within thirty (30) days following receipt of Quests notice thereof, or (iii) by
Customer for any reason upon thirty (30) days written notice to Quest. Upon termination of this
Agreement or expiration or termination of a License for any reason, all rights granted to
Customer for the applicable License(s) shall immediately cease and Customer shall immediately:
(i) cease using the applicable Software and Documentation, (ii) return the applicable Software to
Quest together with all Documentation and other materials associated with the Software and all
copies of any of the foregoing, or destroy such items, (iii) cease using the Maintenance Services
associated with the applicable License(s), (iv) pay Quest or the applicable Partner all amounts
due and payable up to the date of termination, and (v) give Quest a written certification that
Customer has complied with all of the foregoing obligations. Termination of this Agreement or a
License shall be without prejudice to any other remedies that the terminating party may have
under law, subject to the limitations and exclusions set forth in this Agreement. Any provision of
this Agreement that requires or contemplates execution after termination of this Agreement or
expiration of a License is enforceable against the other party and their respective successors and
assignees notwithstanding termination or expiration, including, without limitation, the
Payment, Taxes, Termination, Warranty Disclaimer, Infringement, "Limitation of
Liability," "Nondisclosure," Usage Verification, and General Sections of this Agreement.
9. Export. Customer acknowledges and agrees that the Products are subject to the export control
laws, rules, regulations, restrictions and national security controls of the United States and other
applicable foreign agencies (the "Export Controls"), and agrees not to export or re-export, or
allow the export or re-export of the Products or any copy, portion or direct product of the
foregoing in violation of the Export Controls. Customer hereby represents that (i) Customer is not
an entity or person to which shipment of Products is prohibited by the Export Controls; and (ii)
Customer will not export, re-export or otherwise transfer the Products to (a) any country subject
to a United States trade embargo, (b) a national or resident of any country subject to a United
States trade embargo, (c) any person or entity to which shipment of Products is prohibited by the
Export Controls, or (d) anyone who is engaged in activities related to the design, development,
production, or use of nuclear materials, nuclear facilities, nuclear weapons, missiles or chemical
or biological weapons.
10. Maintenance. During any Maintenance Period and for the applicable fees, Quest shall make
available to Customer the Maintenance Services for the Software as defined in this Section. The
first Maintenance Period begins on the date of delivery of the Software following an Order and
ends twelve (12) months thereafter unless otherwise set forth in the applicable Order (the Initial
Maintenance Period). Following the Initial Maintenance Period, Maintenance Services shall
389
automatically renew for additional terms of twelve (12) months (each, a Renewal Maintenance
Period) unless the renewal has been cancelled by either party giving written notice to the other
at least sixty (60) days prior to the first day of the applicable Renewal Maintenance Period.
Cancellation of Maintenance Services will not terminate Customers rights to continue to use the
Software. Maintenance fees shall be due in advance of a Renewal Maintenance Period and shall
be subject to the payment requirements set forth in this Agreement. The procedure for reinstating
Maintenance Services after it has lapsed is posted at http://support.quest.com/Maintenance_
Service.asp. Except as otherwise stated in the Product Guide, Maintenance Services shall be
available via the Internet, e-mail, or telephone and shall mean the following:
(a) Quest shall make available to Customer new versions and releases of the Software, including
Software corrections, enhancements and upgrades, if and when Quest makes them generally
available without charge as part of Maintenance Services.
(b) Quest shall respond to unlimited communications from Customer that report Software failures
not previously reported to Quest by Customer. Nothing in the foregoing shall operate to limit or
restrict follow up communication by Customer regarding Software failures.
(c) Quest shall respond to requests from Customers technical coordinators for assistance with the
operational/technical aspects of the Software; provided that Quest shall have the right to limit
such responses if Quest determines, in its sole reasonable discretion, that on-site consulting
services would be more appropriate to address the scope and nature of the requests. Any such
onsite consultation would be pursuant to a services agreement as agreed upon by the parties.
(d) Customer shall have access to Quests Support Web site at http://support.quest.com
(SupportLink).
(e) Maintenance Services are available during standard support hours (Business Hours) as
indicated on SupportLink. In addition, Customer may purchase Business Critical Support (i.e.
24x7 Severity Level 1 support) for certain Software. The list of Software for which Business
Critical Support is available and/or required is set forth on SupportLink.
(f) During Business Hours, Quest will respond within one (1) hour to a call from Customer which
reports a critical Software condition (a Severity Level 1 Problem). Customer must use
commercially reasonable efforts to provide Quest with the necessary remote access to facilitate
the identification and resolution of a Severity Level 1 Problem. Quests ability to identify and
resolve a Severity Level 1 Problem may be delayed without such remote access.
(g) The Maintenance Services for those Software products that Quest has obtained through an
acquisition or merger may, for a period of time following the effective date of the acquisition or
merger, be governed by terms other than those in this Section 10. The applicable different terms,
if any, shall be stated on SupportLink.
11. Warranties.
(a) Software Warranty. Quest warrants that, for a period of thirty (30) days following the initial
delivery of Software pursuant to an Order (the Warranty Period), (i) the media provided by
Quest, if any, on which the Software is recorded will be free from material defects in materials
and workmanship under normal use, (ii) the operation of the Software, as provided by Quest, will
substantially conform to the Documentation applicable to such Software, and (iii) the Software as
390
delivered by Quest does not contain any viruses, worms, Trojan Horses, or other malicious or
destructive code designed by Quest to allow unauthorized intrusion upon, disabling of, or
erasure of the Software (however, the Software may contain a key limiting use of the Software to
within the scope of License granted, and license keys issued by Quest for temporary use are timesensitive) (the Warranties). Customer must give written notice to Quest of any breach of the
Warranties no later than five days following the expiration of the Warranty Period.
Customers exclusive remedies, and Quests sole obligations, for any such breach of these
Warranties shall be as follows: (a) for the warranty in subsection (i), Quest shall, at its
expense, replace any defective media; (b) for the warranty in subsection (ii), Quest shall
correct or provide a workaround for reproducible errors in the Software that cause a breach of
the warranty within a reasonable time considering the severity of the error and its effect on
Customer, or, at Quests option, refund the license fees paid for the nonconforming Software
upon return of such Software to Quest and termination of the related License(s) hereunder; and
(c) for the warranty in subsection (iii), Quest shall provide a copy of the Software that is in
conformance with such warranty.
The foregoing Warranties shall not apply to any non-conformance (i) that Quest cannot
recreate after exercising commercially reasonable efforts to attempt to do so; (ii) caused by
misuse of the Software or by using the Software in a manner that is inconsistent with this
Agreement or the Documentation; or (iii) arising from the modification of the Software by
anyone other than Quest.
(b) Hardware Warranty. Hardware shall be warranted in accordance with the warranty
document delivered with the Hardware and/or included on the hardware manufacturers website.
In the event Customer acquires Hardware that is delivered with a third party warranty (Third
Party Warranty), Customer will rely solely on the applicable third party for all Third Party
Warranty obligations.
(c) Warranty Disclaimer. THE EXPRESS WARRANTIES AND REMEDIES SET FORTH IN
THIS SECTION ARE THE ONLY WARRANTIES AND REMEDIES PROVIDED BY QUEST
HEREUNDER. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, ALL
OTHER WARRANTIES OR REMEDIES ARE EXCLUDED, WHETHER EXPRESS OR
IMPLIED, ORAL OR WRITTEN, INCLUDING ANY IMPLIED WARRANTIES OF
MERCHANTABILITY, FITNESS FOR ANY PARTICULAR PURPOSE, NONINFRINGEMENT, SATISFACTORY QUALITY, AND ANY WARRANTIES ARISING FROM
USAGE OF TRADE OR COURSE OF DEALING OR PERFORMANCE. QUEST DOES NOT
WARRANT UNINTERRUPTED OR ERROR-FREE OPERATION OF THE PRODUCTS.
12. Infringement. Quest will at its own expense defend or settle any claim, suit, action, or
proceeding brought against Customer by a third party to the extent it is based on an allegation
that the Software directly infringes any patent, copyright, trademark, or other proprietary right
enforceable in the country in which the Software is delivered to Customer, or misappropriates a
trade secret in such country (a Claim). Additionally, Quest shall pay any judgments finally
awarded against Customer under a Claim or any amounts assessed against Customer in any
settlements of a Claim, and reasonable administrative costs or expenses, including without
limitation reasonable attorneys fees, necessarily incurred by Customer in responding to the
Claim. Quests obligations under this Section are conditioned upon Customer (i) giving prompt
written notice of the Claim to Quest; (ii) permitting Quest to retain sole control of the
391
investigation, defense or settlement of the Claim, and (iii) providing Quest with such cooperation
and assistance as Quest may reasonably request from time to time in connection with the
investigation, defense or settlement of the Claim. Quest shall have no obligation hereunder to
defend Customer against any Claim (a) resulting from use of the Software other than as
authorized in this Agreement, (b) resulting from a modification of the Software other than by
Quest, or (c) based on Customers use of the Software after Quest recommends discontinuation
because of possible or actual infringement, (d) based on Customers use of a superseded or altered
release of Software if the infringement would have been avoided by use of a current or unaltered
release of the Software made available to Customer, or (e) to the extent the Claim arises from or
is based on the use of the Software with other products, services, or data not supplied by Quest if
the infringement would not have occurred but for such use. If Customers use of the Software is
enjoined as a result of a Claim, Quest shall, at its expense and option either (i) obtain for
Customer the right to continue using the Software, (ii) replace the Software with a functionally
equivalent non-infringing product, (iii) modify the Software so that it is non-infringing, or (iv)
accept the return of the infringing Software and refund the license fee paid for the infringing
Software, pro-rated over a sixty (60) month period from the date of delivery of the Software
following an Order . This Section states the entire liability of Quest, and Customers sole and
exclusive remedy, with respect to a Claim.
13. Limitation of Liability. EXCEPT FOR (A) ANY BREACH OF THE "RESTRICTIONS" OR
"NONDISCLOSURE" SECTIONS OF THIS AGREEMENT, (B) AMOUNTS CONTAINED IN
JUDGMENTS OR SETTLEMENTS WHICH QUEST IS LIABLE TO PAY ON BEHALF OF
CUSTOMER UNDER THE INFRINGEMENT SECTION OF THIS AGREEMENT, OR (C)
ANY LIABILITY TO THE EXTENT LIABILITY MAY NOT BE EXCLUDED OR LIMITED
AS A MATTER OF LAW, IN NO EVENT SHALL QUEST, ITS AFFILIATES, OR
SUPPLIERS, OR CUSTOMER BE LIABLE FOR ANY LOSS OF REVENUE, LOSS OF
ACTUAL OR ANTICIPATED PROFITS, LOSS OF BUSINESS, LOSS OF CONTRACTS, LOSS
OF GOODWILL OR REPUTATION, LOSS OF ANTICIPATED SAVINGS, LOSS OF,
DAMAGE TO OR CORRUPTION OF DATA, OR FOR ANY INDIRECT, INCIDENTAL,
SPECIAL OR CONSEQUENTIAL LOSS OR DAMAGE OF ANY KIND, IN EACH CASE
HOWSOEVER ARISING, WHETHER SUCH LOSS OR DAMAGE WAS FORESEEABLE OR
IN THE CONTEMPLATION OF THE PARTIES AND WHETHER ARISING IN OR FOR
BREACH OF CONTRACT, TORT (INCLUDING NEGLIGENCE), BREACH OF
STATUTORY DUTY, OR OTHERWISE.
EXCEPT FOR (A) ANY BREACH OF CUSTOMERS PAYMENT OBLIGATIONS; (B) ANY
BREACH OF THE "SOFTWARE LICENSE," RESTRICTIONS," EXPORT OR
"NONDISCLOSURE" SECTIONS OF THIS AGREEMENT, OR ANY OTHER VIOLATION OF
THE OTHER PARTYS INTELLECTUAL PROPERTY RIGHTS; (C) QUESTS EXPRESS
OBLIGATIONS UNDER THE INFRINGEMENT SECTION OF THIS AGREEMENT; OR (D)
ANY LIABILITY TO THE EXTENT LIABILITY MAY NOT BE EXCLUDED OR LIMITED
AS A MATTER OF LAW, THE MAXIMUM AGGREGATE AND CUMULATIVE LIABILITY
OF QUEST, ITS AFFILIATES AND SUPPLIERS, AND CUSTOMER UNDER THIS
AGREEMENT, WHETHER ARISING IN OR FOR BREACH OF CONTRACT, TORT
(INCLUDING NEGLIGENCE), BREACH OF STATUTORY DUTY, OR OTHERWISE, SHALL
NOT EXCEED THE FEES PAID AND/OR OWED (AS APPLICABLE) BY CUSTOMER FOR
THE PRODUCTS OR MAINTENANCE SERVICES THAT ARE THE SUBJECT OF THE
CLAIM. FOR MAINTENANCE SERVICES OR A PRODUCT SUBJECT TO RECURRING
392
FEES, THE LIABILITY SHALL NOT EXCEED THE AMOUNT PAID AND/OR OWED (AS
APPLICABLE) FOR SUCH MAINTENANCE SERVICE OR PRODUCT DURING THE
TWELVE (12) MONTHS PRECEDING THE CLAIM.
Quests Affiliates and suppliers shall be beneficiaries of this Limitation of Liability" section and
Customers Service Providers are entitled to the rights granted under the "Third Party Use"
section of this Agreement; otherwise, no third party beneficiaries exist under this Agreement.
Quest expressly excludes any and all liability to Customers Service Providers, Clients and to
any other third party.
14. Nondisclosure. Confidential Information means information or materials disclosed by one
party (the Disclosing Party) to the other party (the Receiving Party) that are not generally
available to the public and which, due to their character and nature, a reasonable person under
like circumstances would treat as confidential, including, without limitation, the Disclosing
Partys personal data, financial information, marketing information, trade secrets, know-how,
proprietary tools, proprietary knowledge and proprietary methodologies. Confidential Information
of Quest further includes this Agreement, the Products (in source code and/or object code form),
the pricing and discounting offered by Quest to Customer hereunder, information regarding the
functionality and performance of the Products, benchmark test results regarding the Products, and
any Software license keys provided to Customer. Additionally, Confidential Information shall
include Individually Identifiable Health Information (as that term is defined in 45 CFR
164.501) or Nonpublic Personal Information (as that term is defined in Title V of the GrammLeach-Bliley Act of 1999) that Quest may come into contact with under this Agreement.
Confidential Information shall not include information or materials that (a) were, on the date of
disclosure, generally known to the public; (b) become generally known to the public other than
as a result of the act or omission of the Receiving Party; (c) were known to the Receiving Party
without an obligation of confidentiality prior to that party receiving the same from the
Disclosing Party; (d) the Receiving Party lawfully received from a third party without that third
partys breach of agreement or obligation of trust; or (e) are or were independently developed by
the Receiving Party without access to or use of the Disclosing Partys Confidential Information.
Additionally, it shall not be a breach of this Section for the Receiving Party to disclose the
Disclosing Partys Confidential Information as may be required by operation of law or legal
process, provided that the Receiving Party provides prior notice of such disclosure to the
Disclosing Party unless expressly prohibited from doing so by a court, arbitration panel or other
legal authority of competent jurisdiction. The Receiving Party shall not (1) make the Disclosing
Partys Confidential Information available to any Affiliates, directors, officers, employees,
consultants or representatives (collectively, the Representatives) who do not have a need to
know in order to carry out the purposes of this Agreement; (2) otherwise disclose the Disclosing
Partys Confidential Information to any third party without the written consent of the Disclosing
Party; or (3) use the Disclosing Partys Confidential Information for any purpose other than as
contemplated by this Agreement. The Receiving Party shall inform its Representatives of the
confidential nature of the Disclosing Partys Confidential Information and the requirements
regarding restrictions on disclosure and use as set forth in this Section and shall disclose the
Disclosing Partys Confidential Information only to its Representatives who are legally bound to
protect the Confidential Information under terms at least as restrictive as those provided herein.
The Receiving Party agrees to protect the Disclosing Partys Confidential Information from
unauthorized use or disclosure by exercising at least the same degree of care it uses to protect its
own similar information, but in no event less than a reasonable degree of care. The Receiving
393
Party shall be liable to the Disclosing Party for any disclosure or other breach in violation of this
Agreement by any of its Representatives. The Receiving Party shall promptly notify the
Disclosing Party of any known unauthorized use or disclosure of the Disclosing Partys
Confidential Information and will cooperate with the Disclosing Party in any litigation brought
by the Disclosing Party against third parties to protect its proprietary rights.
15. Usage Verification. At Quest's request, but not more frequently than once per year, Customer
shall furnish Quest with a document signed by an authorized representative verifying Customer's
installations and usage of the Products. Customer will permit Quest to review Customer's
deployment and use of the Products for compliance with the terms and conditions of this
Agreement. Any such reviews shall be scheduled at least fifteen (15) days in advance, shall be
conducted during normal business hours at Customer's facilities, and shall not unreasonably
interfere with Customer's business activities. If Customer's use of the Products is found to be
greater than contracted for Customer will be invoiced for the additional use and the unpaid fees
shall be payable in accordance with this Agreement. Additionally, if the unpaid fees exceed five
percent (5%) of the fees paid for the subject Products, then Customer shall also pay Quest's
reasonable costs of conducting the audit. This Section shall not limit or restrict any other rights
or remedies of Quest that are otherwise set forth in this Agreement or available at law.
16. General.
(a) Governing Law and Venue. This Agreement shall be governed by and construed in
accordance with the laws of the State of California, without giving effect to any conflict of laws
principles that would require the application of laws of a different state. The parties agree that
neither the United Nations Convention on Contracts for the International Sale of Goods, nor the
Uniform Computer Information Transaction Act (UCITA) shall apply to this Agreement,
regardless of the states in which the parties do business or are incorporated. Any action seeking
enforcement of this Agreement or any provision hereof shall be brought exclusively in the state
or federal courts located in the County of Orange, State of California, United States of America.
Each party hereby agrees to submit to the jurisdiction of such courts.
(b) Assignment. Except as otherwise set forth herein, Customer shall not, in whole or part, assign
or transfer any part of this Agreement or any rights hereunder without the prior written consent
of Quest. Any attempted transfer or assignment by Customer that is not permitted by this
Agreement shall be null and void.
(c) Severability. If any provision of this Agreement shall be held by a court of competent
jurisdiction to be contrary to law, such provision will be enforced to the maximum extent
permissible and the remaining provisions of this Agreement will remain in full force and effect.
Notwithstanding the foregoing, the terms of this Agreement that limit, disclaim, or exclude
warranties, remedies or damages are intended by the parties to be independent and remain in
effect despite the failure or unenforceability of an agreed remedy. The parties have relied on the
limitations and exclusions set forth in this Agreement in determining whether to enter into it.
(d) Use by U.S. Government. The Software is a commercial item under FAR 12.201.
Consistent with FAR section 12.212 and DFARS section 227.7202, any use, modification,
reproduction, release, performance, display, disclosure or distribution of the Software or
Documentation by the U.S. government shall be governed solely by the terms of this Agreement
and shall be prohibited except to the extent expressly permitted herein.
394
(e) Personal Data. Customer hereby acknowledges and agrees that Quests performance of this
Agreement may require Quest to process or store personal data of Customer, its employees and
Affiliates and to transmit such data internally within Quest or to Quest Affiliates. Such
processing, storage, and transmission shall only be to the extent necessary for, and for the sole
purpose of, enabling Quest to perform its obligations under this Agreement and may take place
in any of the countries in which Quest and its Affiliates conduct business, which may include
countries outside of the European Economic Area. Quest hereby affirms to Customer that Quest
Software, Inc. currently abides by the safe harbor framework as set forth by the U.S. Department
of Commerce regarding the collection, use and retention of data from the European Union.
(f) Notices. All notices provided hereunder shall be in writing, delivered personally, sent by
facsimile or e-mail, or mailed by first class mail, postage prepaid, addressed to the legal
department of the respective party or to such other address as may be specified in an Order or in
writing by either of the parties to the other in accordance with this Section. All notices, requests,
demands or communications shall be deemed effective upon personal delivery or four (4) days
following deposit in the mail in accordance with this paragraph.
(g) Disclosure of Customer Status. Quest may include Customer in its listing of customers and,
upon written consent by Customer, announce Customer's selection of Quest in its marketing
communications.
(h) Waiver. Performance of any obligation required by a party hereunder may be waived only by
a written waiver signed by an authorized representative of the other party, which waiver shall be
effective only with respect to the specific obligation described therein. Any waiver or failure to
enforce any provision of this Agreement on one occasion will not be deemed a waiver of any
other provision or of such provision on any other occasion.
(i) Injunctive Relief. Each party acknowledges and agrees that in the event of a material breach
of this Agreement, including but not limited to a breach of the Software License," Restrictions
or "Nondisclosure" Sections of this Agreement, the non-breaching party shall be entitled to seek
immediate injunctive relief, without limiting its other rights and remedies.
(j) Force Majeure. Each party will be excused from performance for any period during which,
and to the extent that, it is prevented from performing any obligation or service as a result of
causes beyond its reasonable control, and without its fault or negligence, including without
limitation, acts of God, strikes, lockouts, riots, acts of war, epidemics, communication line
failures, and power failures. Nothing in the foregoing shall be deemed to relieve Customer or its
Affiliates of its obligation to pay fees owed under this Agreement.
(k) Equal Opportunity. Quest Software Inc. is a federal contractor and Affirmative Action
employer (M/F/D/V) as required by the Equal Opportunity clause C.F.R. 60-741.5(a).
(l) Headings. Headings in this Agreement are for convenience only and do not affect the
meaning or interpretation of this Agreement. This Agreement will not be construed either in
favor of or against one party or the other, but rather in accordance with its fair meaning. When
the term including is used in this Agreement it will be construed in each case to mean
including, but not limited to.
(m) Entire Agreement. This Agreement is intended by the parties as a final expression of their
agreement with respect to the subject matter hereof and may not be contradicted by evidence of
395
any prior or contemporaneous agreement unless such agreement is signed by both parties. In the
absence of such an agreement, this Agreement shall constitute the complete and exclusive
statement of the terms and conditions and no extrinsic evidence whatsoever may be introduced
in any judicial proceeding that may involve the Agreement. In the event of a conflict between
the terms of this Agreement and the terms contained in an Order, the terms in the Order shall
only control if the Order is signed by both Quest and Customer; otherwise, the terms of this
Agreement shall control. Neither this Agreement, nor an Order, may be modified or amended
except by a writing executed by a duly authorized representative of each party. No other act,
document, usage or custom shall be deemed to amend or modify this Agreement or an Order.
Delivery of Products shall be FOB Shipping Point.
Appendix:Contact Quest
Contact Quest/About Quest Software
About Quest
Quest Software simplifies and reduces the cost of managing IT for more than 100,000 customers
worldwide. Our innovative solutions make solving the toughest IT management problems easier,
enabling customers to save time and money across physical, virtual and cloud environments. For
more information about Quest go to www.quest.com.
Established in 1987, Quest Software (Nasdaq: QSFT) provides simple and innovative IT
management solutions that enable more than 100,000 global customers to save time and money
across physical and virtual environments. Quest products solve complex IT challenges ranging
from database management, data protection, identity and access management, monitoring, user
workspace management to Windows management. For more information, visit www.quest.com.
Contacting Quest Software

Email
info@quest.com
Quest Software, Inc.
World Headquarters
Mail
5 Polaris Way
USA
Web site
www.quest.com
See our web site for regional and international office information.
ContactingQuest Support
Quest Support is available to customers who have a trial version of a Quest product or who have
purchased a Quest product and have a valid maintenance contract. Quest Support provides
unlimited 24x7 access to our Support Portal at www.quest.com/support.
From our Support Portal, you can do the following:
l
Retrieve thousands of solutions from our online Knowledge Base
Download the latest releases and service packs
Create, update and review Support cases
397
Contacting Quest
View the Global Support Guide for a detailed explanation of support programs, online services,
contact information, policies and procedures. The guide is available at: www.quest.com/support.
Create_Delete_Benchmark 62, 67
Index
Creation 100
A
Custom load scenario 208

D
Advanced 100
Advanced Creation Objects 100
Data 371
Agent 154
Database Quick Start 346

E
Agents Overview 36
AS3AP Benchmark 277
B
Environment 20
Environment#Understanding_
Benchmarks 21
Benchmark Factory Components 35-37,

41, 57, 74, 96, 105, 140, 214, 248
Excel 274
Benchmark Factory Views 44, 47
Excel-Command Line Exporting of Test

Results 238
Benchmark Object Wizard 101, 171

Execute File Wizard 105, 174
Benchmarks 17, 20, 101, 171, 276-280,
283, 285, 363
Exponential 86
BFCreditCardExp 290
Exporting 274
BFFileArray 293
Exporting Test Results to Excel 274

F
BFFormat 300
Factory 20-21, 363
BFMaxNode 314
BFNumberOfIterations 314
BFState 299
Getting Started 17
BFTrimRight 312
Global Search and Replace 113, 124
BFURandRange 303
Globally Search 113, 124
BFUserID 318
Goal Test 210

C
Goal Testing Quick Start 115, 353

H
Clustering 169
Command Line Wizard 371
Create 227
Create Execute prepared SQL Statement
Transaction 231
How Do I...
Create a Load Scenario 142, 175
I
Import Stored Procedure 200, 225
399
Index
Iteration Overruns 77, 242

J
Profile Creation Wizard 92, 213

Profiles 92, 94, 156, 213
Job 45, 64, 136
Project Node 48
Job Wizard 94, 222
Project View 44
L
Latency 86
Properties Overview 47
Properties View 47
Latency_Keying 85
Latency_Uniform 86
Queue#Jobs_Queue_Toolbar 45
Load Scenario Wizard 176, 192-193, 195196, 200-202, 204-210, 212, 225
Quick 107, 217
Local Area Connections 363
Quick Starts 115, 125, 320, 331, 339, 346,

353
Migrating 371
Replay Test 209
Mix 208
Reports 37, 57, 74, 96, 248
MySQL Initialization Settings 363
Repository Manager 41
Results 237, 274
Negative 86
rpose of this Quic 125, 320
New 17
Run 37, 57, 74, 96, 113, 248
Node 48, 62, 67
Run Reports 37, 57, 74, 96, 248

O
Objects 100
Oracle's Instant Client 370
Run Reports Viewer 254

Running Benchmark Factory with Multiple
Agents 140
S
Oracle Trace Files 373

Oracle Trace Import-Database Replay 339
Scalable Hardware Benchmark 278

Schedule 136
Output View 47
Script 48
Overview 237
P
Script Properties 37, 48-50, 54, 56-57, 62,

67, 71-72, 74, 96, 248
Prepare 227
Scripts Overview 105, 214, 288
Preview 113
Server 363
400
Index
Set Number of Users on an Agent 138
Using Run Reports 257
Settings 144, 146-148, 150, 152, 154
SQL 107, 113, 196, 217, 227, 363
What's 17
SQL Scalability Wizard 107, 196, 217
Wizards 101, 105, 142, 171-173, 175
SQL Server 2005 Files 363

Stagger User Startup 140
Start#Entering_the_desired_SQL_
statement 107, 217
Statement 227
Store Procedure Examples 200, 225, 372
Submit Job Wizard 94, 222
System Requirements 376
T
Test 196, 208, 237, 274
Test Results 274
Test Results Overview 237
Time 85
TPC-B Benchmark 279
TPC-C Benchmark 280
TPC-D Benchmark 283
TPC-H Benchmark 285
Transaction 227
Transaction Wizard 224
U
Understanding 21
Understanding Benchmark Factory 20, 22,
25-26
Upgrading Your Database Server 331
User Scenario Wizard 51, 234-236
Using 363, 371

Benchmark Factory User Guide 6-8-1

Enviado por

Dados do documento

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Benchmark Factory User Guide 6-8-1

Enviado por

Direitos autorais:

Formatos disponíveis

Benchmark Factory for

Benchmark Factory for Databases User Guide

Benchmark Factory for Databases

Benchmark Factory Community

New in this Release

New for Release 6.8.1

New for Release 6.8

New For Release 6.7

Understanding Benchmark Factory

About Benchmark Factory

The Benchmark Factory Environment

Integration with Other Quest Software Products

Understanding the Benchmark Factory Environment

Understanding Benchmark Factory Terminology

How Benchmark Factory Works

Create a Script With Benchmark Factory

1. Set Up Benchmark Factory

Benchmark Factory Components

Cutting and Pasting of Run Reports

Testing Server Throughput

Testing SQL Scalability

Testing Real Application Clusters

Benchmark Factory for Databases User Guide

Jobs Queue View

User Scenario Wizard

Profile Creation Wizard

Submit Job Dialog

Cutting and Pasting of Run Reports

Testing Server Throughput

Testing SQL Scalability

Testing Real Application Clusters

Advanced Creation Objects (Object Mapping)

Benchmark Objects Wizard

Changing Scaling Factors

To Change Scale Factors

Execute File Wizard

Benchmark Factory for Databases User Guide

SQL Scalability Wizard

Copy a Load Scenario to a Replay, Mix, Goal, or Scalability Test

Running SQL Preview

Replace Child Latencies

Goal Testing Quick Start

Actions Perfomred in This Quick Start

Use Global Search and Replace

Creating an ODBC Trace File

Real Application Clustering (RAC) Quick Start

Purpose of this Quick Start

Actions Performed in This Quick Start

Save the Selected Job to the Script View

Set the Number of Users on an Agent

Stagger User Startup

Benchmark Factory for Databases User Guide

Running Benchmark Factory with Multiple Agents

Benchmark Factory Console

Benchmark Factory Agent

Running a Job With Multiple Agents

Creating Load Scenarios

Load Scenario Wizard

User Load (Measurement Intervals)

Profile Properties Real-Time Counters Tab

Benchmark Factory for Databases User Guide

Benchmark Objects Wizard

Benchmark Objects Wizard

Data Repository Migration Wizard