1

2

Contents
LNP TEST CASE DESIGN AND LOAD TEST SELECTION ............................................................................3
1 LNP TEST CASE DESIGN ......................................................................................................................3
2 DESIGN OF EXPERIMENT .....................................................................................................................4
3 REFERENCE .....................................................................................................................................9


3

LnP Test Case Design and Load Test
Selection
In LnP testing test case design and load test execution condition setting is very important. Due
to limited resources (Man, Machine and Time), only limited number of LnP test can be
conducted in an eviornment.
1 LnP Test Case Design
Most of the time LnP test cases are not designed separately but picked up from functional test
cases. Certainly this is not a good idea in case of BPEL PM. Why not?

Most of the functional test cases are for boundary systems which are connected using
BPEL processes.
Functional test cases are written from code coverage perspective, essentially covering
boundary conditions.

Due to mentioned reasons, Test cases for LnP exercise should be designed keeping following
consideration in view:

LnP objective (Refer: Error! Reference source not found.)
Load profile (Refer: Error! Reference source not found.)

After designing LnP test cases, next important thing is designing Load Test itself. Therefore
following considerations must be kept in view:

Objective: What is objective of this load test?
Test case selection: Which test cases will be included in this load run?
Test case ratio: What is the ratio of each test case types in this load run?
Load ramp up: What pattern will be followed for load ramp up?
Load ramp down: What pattern will be followed for load ramp down?
Steady Load: What is steady load and for how long (if any)?
Isolation/Integrated: Is this test isolated one or integrated one with boundary systems?
Availability of peripherals: Are all infrastructural needs satisfied? Is this will hit
peripherals upper limit (e.g. bandwidth)?

All most every QA wants to run all possible tests on the system under consideration. But is it
possible. NO. This big no raises another question. If all possible tests cannot be run, then how
to select few? The answer is in statistics Design of Experiment.
4

2 Design of Experiment
Design of Experiment (DoE) technique enables to design a series of experiments (here it is LnP
tests) to gain insight into various factors and their individual and collective effect on a system.
Instead of going into theory part, lets do a simple exercise.
One wants to optimize Data Source Connection pool in a given environment. It is assumed that
except Data Source Connection Pool parameters, any other parameter cannot be changed.
Data Source Connection Pool parameters (Refer: Error! Reference source not found.) are listed
in following table which can be potentially manipulated:
# Parameter Remark
1 Statement cache Possible values: LRU, and
Fixed
2 Statement Cache Size Default: 10
Disabled: 0
3 Connection Testing
Option

3.1 Automatic
3.1.1 Test Frequency Automatic testing is
disabled if set to 0
3.1.2 Test Reserved
Connection
Yes/No
3.1.3 Seconds to
Trust on idle
pool
connection

3.2 Manual Parameters specified in
connection details.
4 Connection creation
retry frequency
Default : 0
0 means connection
creation retry is disabled.
5 Requests to wait for a
connection

5.1 Connection
reserve time
out
Default: 10
Disable: -1
Indefinite: 0
Maximum: 2147483647
5.2 Max. wait
time
Default: MAX_INT
Min/No request will wait: 0
6 XA transaction time
out

7 Automatically recover
leaked connection
Default/Disabled/Minimum:
0
Maximum: 2147483647
5

8 Statement Processing
time
Default/No wait: 0
Maximum: 2147483647
9 Pinned to Thread
10 GridLock connection
11 Single Client Access
Name

12 Logging last resource
13 Number of servers in
the cluster

14 Number of
connections available
in database


Out of fourteen parameters listed above, only following can be manipulated in a given system
due to one or more business, and technical reasons.
# Parameter Is
manipulatable
?
Remark
1 Statement
cache
No Possible values: LRU, and
Fixed
2 Statement
Cache Size
Yes Default: 10
Disabled: 0
3 Connection
Testing
Option
Yes
3.1 Automatic Yes
3.1.
1
Test
Frequency
Yes Automatic testing is
disabled if set to 0
3.1.
2
Test
Reserved
Connectio
n
Yes Yes/No
3.1.
3
Seconds to
Trust on
idle pool
connection
Yes
3.2 Manual No Not used
4 Connection
creation
retry
frequency
Yes Default : 0
0 means connection
creation retry is disabled.
5 Requests to
wait for a
Yes
6

connection
5.1 Connectio
n reserve
time out
Yes Default: 10
Disable: -1
Indefinite: 0
Maximum: 2147483647
5.2 Max. wait
time
Yes Default: MAX_INT
Min/No request will wait: 0
6 XA
transaction
time out
No Not used
7 Automaticall
y recover
leaked
connection
Yes Default/Disabled/Minimum
: 0
Maximum: 2147483647
8 Statement
Processing
time
Yes Default/No wait: 0
Maximum: 2147483647
9 Pinned to
Thread
No Not used
10 GridLock
connection
No Not used
11 Single Client
Access Name
No Not used
12 Logging last
resource
No Not used
13 Number of
servers in the
cluster
No Cannot be changed
14 Number of
connections
available in
database
No Cannot be changed

Above table can be rewritten as:
# ID Parameter Is
manipulatable?
Remark
2 A Statement Cache Size Yes Default: 10
Disabled: 0
3.1.1 B Automatic Connection Testing - Test
Frequency
Yes Automatic testing is disabled
if set to 0
3.1.2 C Automatic Connection Testing - Test
Reserved Connection
Yes Yes/No
3.1.3 D Automatic Connection Testing - Seconds to Yes
7

Trust on idle pool connection
4 E Connection creation retry frequency Yes Default : 0
0 means connection creation
retry is disabled.
5.1 F Requests to wait for a connection -
Connection reserve time out
Yes Default: 10
Disable: -1
Indefinite: 0
Maximum: 2147483647
5.2 G Requests to wait for a connection - Max. wait
time
Yes Default: MAX_INT
Min/No request will wait: 0
7 H Automatically recover leaked connection Yes Default/Disabled/Minimum: 0
Maximum: 2147483647
8 I Statement Processing time Yes Default/No wait: 0
Maximum: 2147483647

Lets list value of parameters in the starting of first test.
ID Parameter Value Remark
A Statement Cache Size 10 Default: 10
Disabled: 0
B Automatic Connection Testing - Test
Frequency
5 Automatic testing is disabled
if set to 0
C Automatic Connection Testing - Test
Reserved Connection
Yes Yes/No
D Automatic Connection Testing - Seconds to
Trust on idle pool connection
10
E Connection creation retry frequency 5 Default : 0
0 means connection creation
retry is disabled.
F Requests to wait for a connection -
Connection reserve time out
1000 Default: 10
Disable: -1
Indefinite: 0
Maximum: 2147483647
G Requests to wait for a connection - Max. wait
time
60 Default: MAX_INT
Min/No request will wait: 0
H Automatically recover leaked connection 1000 Default/Disabled/Minimum: 0
Maximum: 2147483647
I Statement Processing time 600 Default/No wait: 0
Maximum: 2147483647

With wider consultation with technical teams and researching the existing documentation,
delta of each parameter is found from nominal value. Lets list those values:
ID Parameter Value
8

Nominal Min Max
A Statement Cache Size 10 5 15
B Automatic Connection Testing - Test
Frequency
5 3 10
C Automatic Connection Testing - Test
Reserved Connection
Yes No
D Automatic Connection Testing - Seconds to
Trust on idle pool connection
10 5 20
E Connection creation retry frequency 5 3 10
F Requests to wait for a connection -
Connection reserve time out
1000 500 2000
G Requests to wait for a connection - Max. wait
time
60 30 120
H Automatically recover leaked connection 1000 500 2000
I Statement Processing time 600 300 1000

The above listed information can also be depicted as:
Parameter ID Value
Nominal Min Max
A 0 -1 1
B 0 -1 1
C 0 1
D 0 -1 1
E 0 -1 1
F 0 -1 1
G 0 -1 1
H 0 -1 1
I 0 -1 1

If one wish to find out all possible combinations for test, one need to make a truth table. In this
truth table there will be thousands of entries (slightly less than 3*2^9). In a real world, all these
many LnP tests are not possible to perform due to obvious reasons.
Now the question arises, out of 3*2^9 possible combinations which combinations to be tested?
It is always good idea to start with test which has nominal values (say Test-0). Now depending
upon experience of team, expert advice, benchmarks (by analysts or other teams) or previous
experiments; next test should be performed using new parameter values.
Following tests are selected to be performed and result of each test is:
Test # Parameters Results (10K messages per hr)
9

A B C D E F G H I
process completion time in
sec
Test-0 0 0 0 0 0 0 0 0 0 1.000
Test-1 0 0 0 0 1 1 1 1 1 1.001
Test-2 0 0 0 1 1 1 1 1 1 1.201
Test-3 1 1 1 1 1 1 0 0 0 1.301
Test-4 1 1 1 1 1 1 1 0 0 0.9914
Test-5 -1 -1 -1 -1 1 1 1 1 1 1.202
Test-6 -1 -1 -1 -1 -1 -1 1 1 1 1.231
Test-7 0 0 0 -1 -1 -1 -1 -1 -1 1.243
Test-8 1 1 1 1 -1 -1 -1 -1 -1 0.9987
Test-9 -1 -1 -1 -1 1 1 1 1 1 1.012
Test-10 -1 -1 -1 -1 0 0 0 0 0
1.098

From the table, it is very clear that Test-4 is giving best performance under test conditions.

In real life, one needs to do series of LnP tests. One needs to start from bottom of stack to start
tuning the system.
1. Tune operating system
2. Tune JVM
3. Tune WLS
4. Tune JMS ( if in use)
5. Tune database
6. Tune BPEL code ( follow best practices and avoid anti patterns)
7. Now perform integrated LnP test to tune over all, system.
3 Reference

Design of Experiment at Wikipedia:
http://en.wikipedia.org/wiki/Design_of_experiments
Design of Experiment tutorial: https://www.moresteam.com/toolbox/design-of-
experiments.cfm
Statistics Handbook: http://www.itl.nist.gov/div898/handbook/toolaids/pff/e-
Handbook.pdf