08 Software Testing Updated 2018-FOR STUDENTS

Software
Testing
Lecture# 35
DCS, NUCES, Islamabad Spring 2013 2
Why Testing and Analysis?
 Software is never correct no matter

which developing technique is used
 Any software must be verified.
 Software testing and analysis are:
 important to control the quality of the
product (and of the process)
 very (often too) expensive
 difficult and stimulating

 Human intensive
 Engineering, but also social process
 Increasingly complex software systems
 Pervasive in an increasing number of industries

 Errors are part of our daily life
 Humans make errors in their thoughts, actions, and in

the products that might result from their actions
 Errors occur wherever humans are involved in taking

actions and making decisions
 These fundamental facts of human existence make

testing an essential activity

Software Error, Fault, Failure
If not
Defect
removed

 Development took 10 years & $7
billion
 On June 4, 1996, the flight of the
Ariane 5 launcher ended in a
failure
 Only about 37 seconds after
initiation of the flight sequence,
at an altitude of about 3,700 m,
the launcher veered off its flight
path, broke up and exploded
 “Fixed” Ariane 5 – currently
being used

 Source: ARIANE 5 Flight 501 Failure, Report by the Inquiry Board
 A program segment for converting a floating point number to a signed 16
bit integer was executed with an input data value of a 64-bit number.
 The segment was written for Ariane 4 and the conversion was left
unprotected, because it was thought that Ariane 4 will never attain such a
value.
 The subsystem containing this segment was reused for Ariane 5 as it had
been successfully used over years for Ariane 4
 This run time error (out of range, overflow), which arose in both the
active and the backup computers at about the same time, was detected
and both computers shut themselves down.
 This resulted in the total loss of altitude control. This breakup was
detected by an on-board monitor which ignited the explosive charges to
destroy the vehicle in the air.
 Ironically, this program segment was not required by Ariane 5
 Adequate exception handling and redundancy strategies
(real function of a backup system, degraded modes?)
 Clear, complete, documented specifications (e.g.,
preconditions, post-conditions)
 But perhaps more importantly: usage-based testing
(based on operational profiles), in this case actual Ariane
5 trajectories
 Note this was not a complex, computing problem, but a
deficiency of the software engineering practices in place
…

 An F-18 crashed because of a missing exception
condition:
 An if ... then ... block without the else clause that was thought could not
possibly arise.
 In simulation, an F-16 program bug caused the virtual
plane to flip over whenever it crossed the equator, as a
result of a missing minus sign to indicate south latitude.

 In 1986, a man in Texas received between 16,500-25,000
radiations in less than 10 sec, over an area of about 1 cm.
 He lost his left arm, and died of complications 5 months
later.

 Incident: A patient passed away
 Failure: The device applied higher frequency of radiations than what was
safe. Safety range: [1…10,000 Hz]
 Fault: The software controller of the device did not have a conditional block
(if …. else statements) to perform range checking on the frequency of the
radiation to be applied.
 Errors:
 1. The SW developer of the device controller system had forgotten to include a
range checking conditional block on the frequency of the radiation to be
applied.
 2. The device operator was NOT supposed to enter anything outside
[1…10,000 Hz] range.

Testing Strategies
SW testing is one element of a broader topic referred to as
V&V (many SQA activities).
 Verification (correctly implement specific function)
 Defect detection and correction
 Comparison between implementation and the
corresponding specification
 Are we building the product right?
 Validation (SW is traceable to customer requirements)
 Defect prevention
 Provision of sound basis for specific design
decisions
 Are we building the right product?
Example: Checking the printing of receipts is verification and
correct printing (info) is validation.
Types/Strategies of testing
 Code Inspections
 Software (Module) Testing
 Unit Testing
 Functional Testing
 Integration Testing
 Compliance
 Interoperability Testing
 System Testing
 Recovery
 Security
 Environment
 Acceptance Testing
 Regression Testing
 Stress, Security, performance, Load Testing etc

Software Testing
 A better definition: ”the process of running software with
the intent of finding an error”.
 The way most testing works is to input a set of values, then

compare the expected result with the out comes/computed
ones.
 Depending on the type of data input, we can identify three

levels of program correctness possible, probable, and
absolute correctness.

Three Levels of Correctness
Possible correctness
 Obtaining correct output for some arbitrary input (single
set).
 If the outcome is wrong, the program cannot possibly be
correct.
 For example a multiply program: for input 2 and 3, if
result is 6; is possibly correct.
Probable correctness
 Obtaining correct output for a number of carefully selected inputs.
 If all potential problematic areas are checked in this way,
the program is probably correct.
 Try several values, including the obvious “problem” values
such as zero, largest negative, 1, -1 and so on.
Absolute correctness
 can be demonstrated only by a test that involves every
possible combination of inputs.
 Requires huge amount of time; it is therefore not practical
{However for some programs we can prove correctness
mathematically}.
 Imagine the same above test for a 32 bit machine (how
many combinations are possible? (Hundreds of Millions).

 You are to develop a small program that reads three integer values
from an input dialog. The three values represent the lengths of the
sides of a triangle.
 The program displays a message that states whether the triangle is
scalene, isosceles, or equilateral.
 Write a set of test cases—specific sets of data—to properly test the
program
 The set of test data must be handled by the program correctly to be considered a
successful program.
 Remember that
 a scalene triangle is one where no two sides are equal,
 whereas an isosceles triangle has two equal sides, and
 an equilateral triangle has three sides of equal length.
int main()
{
float a,b,c;
cin>>a >> b >>c; For the input values to
if(a<(b+c) && b < (a+c) && c< (a+b)) represent a triangle, they
{ must be integers greater than
cout<<"\nIt is a Triangle.”; 0 where the sum of any two is
if (a==b && b==c) greater than the third.
cout<<"\nIt is a Equilateral Triangle.”;
if(a==b || a==c || b==c)
cout<<"\nIt is a Isosceles Triangle.”;
else
cout<<"\nIt is a Scalene Triangle.”;
What test cases do you
think should be there?
}
else
cout<<"This Triangle is not possible.”;
return 0;
}
Questions to Answer ??
1. Do you have a test case that represents a valid scalene triangle? (Note that
test cases such as 1, 2, 3 and 2, 5, 10 do not warrant a “yes” answer
because there does not exist a triangle having these dimensions.)
2. Do you have a test case that represents a valid equilateral triangle?
3. Do you have a test case that represents a valid isosceles triangle? (Note
that a test case representing 2, 2, 4 would not count because it is not a
valid triangle.)
4. Do you have at least three test cases that represent valid isosceles
triangles such that you have tried all three permutations of two equal sides
(such as, 3, 3, 4; 3, 4, 3; and 4, 3, 3)?
5. Do you have a test case in which one side has a zero value?
6. Do you have a test case in which one side has a negative value?
7. Do you have a test case with three integers greater than zero such that the
sum of two of the numbers is equal to the third? (That is, if the program
said that 1, 2, 3 represents a scalene triangle, it would contain a bug.)

Questions to Answer ?? (2)
8 Do you have at least three test cases in category 7 such that you have tried
all three permutations where the length of one side is equal to the sum of
the lengths of the other two sides (for example, 1, 2, 3; 1, 3, 2; and 3, 1, 2)?
9 Do you have a test case with three integers greater than zero such that the
sum of two of the numbers is less than the third (such as 1, 2, 4 or
12,15,30)?
10 Do you have at least three test cases in category 9 such that you have tried
all three permutations (for example, 1, 2, 4; 1, 4, 2; and 4, 1, 2)?
11 Do you have a test case in which all sides are zero (0, 0, 0)?
12 Do you have at least one test case specifying noninteger values (such as
2.5, 3.5, 5.5)?
13 Do you have at least one test case specifying the wrong number of values
(two rather than three integers, for example)?
14 For each test case did you specify the expected output from the program
in addition to the input values?

 A very simple program
 To do exhaustive testing
 Test with all possible values of a, b, & c (the three sides)
 And their combinations
 32-bit integer, 232 possible values for one integer
 Assuming only integers from 1 to 10, there are 1012 possible values
for a triangle.
 Testing 1000 cases per second, you would need 317 years!
 Test with all the invalid inputs
 e.g., All strings ;)

 If testing a trivial program is so complex, what about
testing large programs of thousands of lines of code, e.g.,
air traffic control software
 Solution?

 If testing a trivial program is so complex, what about
testing large programs of thousands of lines of code, e.g.,
air traffic control software
 Solution?
 Use of sophisticated testing techniques
 Select the test cases that have most chances of triggering a failure
 e.g, boundary values

Testability for self reading
Which S/W is testable? Characteristics of a testable S/W.
 Operability —it operates cleanly

 Observability —the results of each test case are readily
observed
 Controllability —the degree to which testing can be
automated and optimized
 Decomposability —testing can be targeted
 Simplicity —reduce complex architecture and logic to
simplify tests
 Stability —few changes are requested during testing
 Understandability —of the design

"Bugs lurk in corners
Test case and congregate at
design boundaries ..."
Boris Beizer
OBJECTIVE ---- to uncover errors
CRITERIA ----- In a complete manner
CONSTRAINT ------ with a minimum of effort and time
 A successful test -- -- that uncovers an as-yet undiscovered error.

 Minimum number of required tests with 100% functional coverage and
0% redundancy.
 Rich variety of test case design methods
- Cause-effect graphing, Equivalence Class Partitioning, Boundary
Analysis, and vendor specific: client/server, OO test case design.
Possible approaches:
(1) Knowing the specified functions, tests can be conducted to
demonstrate that each function is fully operational (Black Box).
(2) Knowing the internal workings of a product, tests can be conducted
to ensure that “all gears mesh“ (White Box).
Why bother with white box testing?
Black box testing:
 Requirements fulfilled
 Interfaces available and working
But what about
 the internal structure of a component,
 interactions between objects?
 white box testing
FURTHER
 Logical errors and incorrect assumptions are inversely
proportional to the probability that a program path will
be executed.
 Sometime a path executes counterintuitive.
 Typographical errors are random.
Also: if black box testing finds error, locating it is easier
with additional white box testing!
Exhaustive Testing
loop 20
loop 20
There are 10 14 possible paths ! If we execute one

test per millisecond, it would take 3,170 years to
test this program!!

White-Box Testing
... our goal is to ensure that all

statements and conditions have
been executed at least once ...
36
White-Box
 White box testing, sometimes called glass-box and structural testing.
 Various aspects like Statement Coverage Criteria, Edge coverage,
Condition Coverage, Path Coverage are defined mathematically and
test set is designed accordingly.
 There are no algorithms for generating white box test data. However the
checklist might help:
 Has every line of code been executed at least once by test data.
 Have all default paths been traversed at least once.
 Have all significant combinations of multiple conditions been
identified and
 Have all logical decisions exercised on their true and false sides.
 Have all loops executed at their boundaries and within their
operational bounds.
 Have internal data structures validated▪
40
Control flow graph (CFG)
Gives an abstract
representation of the code
Directed Graph
Nodes are blocks of
sequential statements
Edges are transfers of control
 May be labeled with predicate
representing the condition of
control transfer
41
Control Flow Graph - Example
42
Basics of CFG: Patterns
43
Control Flow Coverage
Depending on the (adequacy) criteria to

cover (traverse) CFGs, we can have
different types of control flow coverage:
a) Statement/Node Coverage
b) Edge Coverage
c) Condition Coverage
d) Path Coverage
Discussed in detail next…
44
(1) Statement/Node Coverage
1) Hypothesis: Faults cannot be discovered if the

statements containing them are not executed
2) Statement coverage criteria: Equivalent to
covering all nodes in CFG
3) Executing a statement is a weak guarantee of
correctness, but easy to achieve
4) Several inputs may execute the same
statements
45
Statement coverage
Execute every statement in the

program at least once
For path: {a,c,e)
(A, B, X) := (2, 0, 3) or
(A, B, X) := (2, 0, any X)
46
Incompleteness of Statement/Node
Coverage
 Though often used, statement coverage is a

weak guarantee of fault detection
47
Statement coverage
Problems:
 Line 2: if the requirement was || rather than &&
 Line 5: no difference if x > 1 condition was
correct or even missing

 Didn’t test the path {a, b, d} in which x is
unchanged
 Same test (a=2, b=0) tests all statements but
not all conditions.
48
Decision (or branch) Coverage
 Select a test set T such that,

by executing P for each test
case t in T, each edge of P’s
control flow graph is
traversed at least once
 We need to exercise all

conditions that traverse the
control flow of the program
with true and false values
 Also known as branch or

edge coverage
49
Decision Coverage
 If there is an error for x >= 0, it will be discovered
by decision coverage *
 If (A or B)
 TF and FF will toggle the decision *
50
Decision coverage
You must write enough test
cases that each decision
has a T and a F outcome at
least once.
 ace & abd or acd & abe
 (2, 0, 4) & (1, 1, 1)
 (3, 0, 3) & (2, 1, 1)
 ace & abe
 (2, 0, X) & (2, 1, X)
Problem
 If the condition x > 1 had a fault,
e.g., x < 1, the mistake would
not be detected
51
Test cases for Decision Coverage
 We choose a test set with two test cases:

1. One table with 0 items and,
2. A table with 3 items, the second element being the desired
one
 For the second test case, the “while” loop body is
executed twice, once executing the “if-then” branch
 The edge coverage criterion is fulfilled and the error
is NOT discovered by above test set
...
while (not found) and counter < number_of_items loop
 The reason for the above problem?
 Not all possible values of the constituents of the condition in
the while loop have been exercised. counter <
number_of_items is not evaluated to False.
52
Condition Coverage
 We need to further strengthen the edge coverage

criterion
 Condition Coverage (CC) Criterion:
 Select a test set T such that, by executing P for each
element in T, each edge of P’s control flow graph is
traversed, and all possible values of the constituents of
compound conditions are exercised at least once
 If (A or B)
 TF/FT and FF will toggle the conditions
53
Condition coverage
You must write enough

test cases that each
condition has a T and a F
outcome at least once
A > 1, A <= 1, B = 0, B != 0
A = 2, A != 2, X > 1, X <= 1
54
Problem with Condition Coverage
 May not cover decisions
 Example
A. If (A && B)
1) A: True, B: False
2) A: False, B: True
B. If (A || B)
1) A: True, B: False
2) A: False, B: True
*Any problem with above 4 TCs?

In case A True value of Decision is not achieved
In case B False value of Decision is not achieved
55
Path Coverage
 Path Coverage Criterion: Select a

test set T such that, by executing P
for each test case t in T, all paths
leading from the initial to the final
node of P’s control flow graph are
traversed
 In practice the number of paths is
too large, if not infinite
 (e.g., when we have loops)
 Some paths are infeasible
 e.g., not practical given the system’s
business logic
Path Coverage – Example
Path Coverage (T1’s coverage)
Path Coverage (T2’s coverage)
TC21 TC22
Path Coverage – Example
Path Coverage – Dealing with Loops
 In practice, however, the number of paths can be too
large, if not infinite (e.g., when we have loops) →
Impractical
 A pragmatic heuristic: Look for conditions that
execute loops
 Zero times
 A maximum number of times
 A average number of times (statistical criterion)
 For example, in the array search algorithm
 Skipping the loop (the table is empty)
 Executing the loop once or twice and then finding the element
 Searching the entire table without finding the desired element
Path Coverage: Loops
Another Example
Path Coverage - Example
Control Flow Coverage: Reachability
 Not all statements are usually reachable in real-
world programs
 It is not always possible to decide automatically
if a statement is reachable and the percentage of
reachable statements
 When one does not reach a 100% coverage, it is
therefore difficult to determine the reason
 Tools are needed to support this activity but it
cannot be fully automated
 Research focuses on search algorithms to help
automate coverage
 Control flow testing is, in general, more
applicable for testing small pieces of code
Cyclomatic Complexity
 Cyclomatic complexity provides a quantitative
measure of the logical complexity of a program
– It is the number of tests that must be conducted
to assure that all statements have been executed
at least once
 Cyclomatic complexity, V(G) is given by:
1 Node
Edge
(1) V(G) = E - N + 2 2,3
where E is the number of edges and N 6 R2 4,5
number of nodes (11-9 +2 = 4) 7 R3 8
9 R1
(2) V(G) = P + 1, where P is the number
of predicate nodes (3+1 = 4)
10 Region
(3) V(G) = number of regions (4)
11 R4
65
Independent Paths
 The value for cyclomatic complexity defines the

number of independent paths in the basis set of a
program
 An independent path is any path through the
program that introduces at least one new set of
processing statements or a new condition
 An independent path must move along at least one
edge that has not been traversed before the path is
defined
 A set of independent paths for a flow graph
composes a basis set.
66
Deriving Test Cases
Using the design or code, draw a
corresponding flow graph
you don't need a flow chart,
1 Node
Edge
Determine the cyclomatic complexity (even
without developing flow graph: count each 2,3
simple logical test (compound tests as 2 or 6 4,5
R
more) + 1 7 R3 8 2
9 R1
Determine a basis set of paths
Prepare test cases that will force execution of 10 Region
paths in the basis set
11 R4
basis path testing should be applied to
critical modules
67
Basis Path Testing
1
derive the
independent paths:
2 Since V(G) = 4,
there are four paths
3
4
5 6 Path 1: 1,2,3,6,7,8
Path 2: 1,2,3,5,7,8
Path 3: 1,2,4,7,8
7
Path 4: 1,2,4,7,2,4,...7,8
8
derive test cases to exercise
these paths.
68
Control Structure Testing: Loop
Nested
Loops
Simple Concatenated
loop Unstructured
Loops Loops
69
Loop Testing: Simple Loops
Minimum conditions—Simple Loops
1. skip the loop entirely

2. only one pass through the loop
3. two passes through the loop
4. m passes through the loop m < n*
5. (n-1), n, and (n+1) passes through the loop
*where n is the maximum number of allowable

passes
70
Loop Testing: Nested Loops
Nested Loops
(1) Start at the innermost loop. Set all outer loops to their minimum
iteration parameter values.
(2) Test the min+1, typical, max-1 and max for the innermost loop (while
holding the outer loops at their minimum values).
(3) Move out one loop and set it up as in step 2, holding all other loops at
typical values.
Continue this step until the outermost loop has been tested.
Concatenated Loops
If the loops are independent of one another
then treat each as a simple loop
else
treat as nested* loops
endif
* for example, the final loop counter value of loop 1
is used to initialize loop 2
For Other control structures DIY
71
Black-Box Testing
requirements
output
input events
NU-Islamabad Spring 2013 72

Black Box Testing
 Types of errors regarding functional requirements of
software:
-- Incorrect or missing functions
-- Interface errors
-- Error in data structure & external data base access
-- Performance errors
-- Initialization & termination errors
 No functional requirements NO Black Box Testing.
 Demonstrates that each function is fully operational.
 Uncovers different kind of errors than white box testing.
 Performed later in the testing process.

 Black box techniques derive a set of test cases
that satisfy the following criteria:
(1) Test cases reduce the number of
additional test cases that must be designed to
achieve reasonable testing
(2) Test cases that tell us something about the
presence or absence of classes of errors,
rather than errors associated only with the
specific test at hand.
 Black box techniques can supplement the test
cases generated by white box.
How to Design Test Cases?
 Equivalence class partitioning

 Boundary value analysis
 Cause / effect graphing (for
combinations of input
conditions)
 Error Guessing

Equivalence Class Partitioning
Two considerations:
1. each test case should invoke as many different input
conditions as possible in order to minimize the total
number of test cases necessary.
2. one should try to partition the input domain of a program
into a finite number of equivalence classes.
 Equivalence partitioning uses the idea of equivalence classes.

 An equivalence class is a set of data which as for
specification is concerned will be treated identically
(equivalently). *
 Objective is to identify those classes of data, which will cause
the module to respond in a different manner from other
classes.
 This is done by reading the specification and creating a list of
all characteristics of programs (e.g. must be numeric, two
integers are input).
Equivalence Class Testing
 Motivation:
 we would like to have a sense of complete testing and we
would hope to avoid test redundancy
 Equivalence classes:
 partitions of the input set in which input data have the same
effect on the program (e.g., the result in the same output)
 Entire input set is covered: completeness
 Disjoint classes: to avoid redundancy
 Test cases: one element of each equivalence class
 But equivalence classes have to be chosen wisely
 Guessing the likely system behavior is needed
77
Example
 Travel service offers discounts to
travelers based on their age.
 0-4 years 100%
 5-15 years 50%
 16-64 years 0%
 64 years and older 25%
 Equivalence classes for age
 0, 1, 2, 3, 4
 5, 6, 7, …15
 16, 17, 18, …64
 65, 66, 67, …120
Similarly for destinations
 e.g., destination can be grouped
based on regions (that have
same fair)
Nothing special for name
78
Identifying Equivalence classes
 A key concept in the identification of classes is
negation, i.e. If a characteristic is identified as an
equivalence class, then one should immediately
negate the characteristic in order to find
examples of classes which should cause the
module to do something different such as
“generate an error message”.
 Partitioning each input condition into two or
more groups.
 Two types of equivalence classes are identified:
1. Valid Equivalence Classes
2. Invalid Equivalence Classes

Defining equivalence classes
Input condition is a range: one valid and two

invalid classes are defined
Input condition requires specific value: one
valid and two invalid classes are defined
Input condition is boolean: one valid and one
invalid class are defined
 Then define one test case for each

equivalence class
Equivalence classes Examples
Input or Output Valid Equivalence Invalid Equivalence

Event Classes Classes
Enter a 1–9 < 1, > 9
non-zero digit Letters and other non-
numeric characters
Enter the first First character is a capital First character is not a

letter of a name letter letter
First character is a lower
case letter
Draw a line From 1 to 4 inches long No line

Max. 4 inches Longer than 4 inches
Not a line (a curve)

Characteristic Valid equivalence Invalid equivalence
class class
First char must be alphabetic Letter Non-letter
Next three numerics All numeric One char non-

numeric
Range 100-500 In range (i) Above range
(ii) Below range
Equivalence class table for customer-acc-number
Exercise: Create similar table for your roll number

Weak/Strong Equivalence Classes
 For an example SUT, suppose there are three input variables

from three domains: A, B, C
 A = A1 ∪ A2 ∪ A3 ∪ … ∪ Am where ai∈A
 B = B1 ∪ B2 ∪ B3 ∪ … ∪ Bn where bi∈Bi
 C = C1 ∪ C2 ∪ C3 ∪ … ∪ Co where ci∈Ci
 Weak Equivalence Class Testing:
 Choosing one variable value from each equivalence class (one ai, bi, and
ci) such that all classes are covered.
 # of test cases?
 max (|A|, |B|, |C|)
 Strong Equivalence Class Testing:
 based on the Cartesian product of the partition subsets (A x B x C),
 i.e., testing all interactions of all equivalence classes.
 # of test cases?
 |A| x |B| x |C|
83
Black-Box Testing

Boundary-value Analysis
Boundary-value analysis differs from Equivalence Partitioning in two
respects:
1. Rather than selecting any element to represent an equivalence
class, boundary-value analysis requires that one or more
elements be selected such that each edge of the equivalence
class is subjected to a test.
2. Rather than focusing attention on the input conditions (input
space), test cases are also derived by considering the result
space (i.e., output equivalence classes)
 It will be more exacting to create test cases for number = 99, 100,
500, 501, rather than numbers = 50, 250, 900 for previous example.
 if a module uses a file of records, how will program react if there are no
records on the file; also if a transaction file is used for updating a
master file; have all permutations of EOF conditions been considered.
 If a module is passed an array, what if it contains zero elements? when it
contains maximum number of elements, and so on a pointer to an array
accessing out side an array-boundary.
Errors at the boundaries
 Experience indicates that programmers make

mistakes in processing values at and near the
boundaries of equivalence classes.
 For example, suppose that method M is required to compute a
function f1 when x≤ 0 is true and function f2 otherwise.
 However, M has an error due to which it computes f1 for x<0
and f2 otherwise.
 This fault is revealed when M is tested against x=0,
but not if the input test set is, for example, {-4, 7}
derived using equivalence partitioning
 In this example, the value x=0, lies at the boundary
of the equivalence classes x≤0 and x>0.
86
Example
 Function findPrice() has two parameters: an item

code must be in the range 99..999 and quantity in
the range 1..100
 Equivalence classes for code:
 E1: Values less than 99
 E2: Values in the range
 E3: Values greater than 999
 Equivalence classes for qty:
 E4: Values less than 1.
 E5: Values in the range.
 E6: Values greater than 100
87
Example – Identifying boundaries
 Equivalence classes and boundaries for findPrice

 Boundaries are indicated with an x.
 Points near the boundary are marked *. {Class Exercise}
88
Construct Test sets
 Test selection based on the boundary value

analysis technique requires that tests must
include, for each variable, values at and around the
boundaries
 Consider the following test set:
89
Error-Guessing
Some people design the test cases by

intuition and experience.
The basic idea is to enumerate a list of

possible errors and then write test
cases based on the list.

Testing Phase
What content should be included in a software test plan?
- Testing activities and schedule

- Testing tasks and assignments
- Selected test strategy and test models
- Test methods and criteria
- Required test tools and environment
- Problem tracking and reporting
- Test cost estimation

Test Execution
 using manual approach
 using an automatic approach
 using a semi-automatic approach
Basic activities in test execution:
 Select a test case
 Set up the pre-conditions for a test case
 Set up test data
 Run a test case following its procedure
 Track the test operations and results
 Monitor the post-conditions of a test case & expected results
 Verify the test results and report the problems if there is any
 Record each test execution

Example: Windows Calculator
R-001:
The users should be able to add two numbers and view their result on the
display.
Use Case: UC01 Add Two Numbers
Actors: User
Purpose: Add two numbers and view their result
Overview: The user inputs two numbers and (then adds them and)
checks the result, displayed on the screen.
Type: Primary, Real
Cross References: R-001
Typical Course of Events

Actor Action System Response
1. The actor opens the calculator. The keypad and
display screen appears.
2. The actor input the first number by clicking on the 3. The digit is displayed on the screen.
keypad or using keyboard.
4. The actor clicks or presses the “+” key.
5.The actor then adds the second number as (2). 6. The pressed digit is displayed on the screen.
7. The actor clicks the “=” key. 8. The sum of the two digits is displayed on the screen.

Test Case
Test Case ID: T-101 Test Item: Add Numbers
Wrote By: (tester name) Junaid Documented Date: 26th April 2005
Test Type: Manual Test Suite#: NA
Product Name: Windows Calculator Release and Version No.: V 1.0
Test case description:
Add any two Numbers
Operation procedure:
Open Calculator
Press “1”
Press “+”
Press “2”
Press “=”
Pre-conditions: Post-conditions:
Calculator Opened Result Displayed
Inputs data and/or events: Expected output data and/or events:
1+2= 3
Required test scripts (for auto): NA
Cross References: (Requirements or Use Cases) R-001, UC-01
Bug Report
Problem ID: B-101 current software name: Windows Calculator
Release and Version No.: V 1.0
Test type: Manual Reported by: Junaid
Reported date: 26th April 2005 Test case ID: T-101
Subsystem (or module name): Calculation Feature Name (or
Subject): Add Numbers
Problem type (REQ, Design, Coding,): Coding
Problem severity (Fatal, Major, Minor,): Major
Problem summary and detailed description:
On adding two numbers the result is not correct.
Cause analysis: NA
How to reproduce? Why Required:
Attachments: Nil

08 Software Testing Updated 2018-FOR STUDENTS

Enviado por

Dados do documento

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

08 Software Testing Updated 2018-FOR STUDENTS

Enviado por

Direitos autorais:

Formatos disponíveis

Software

 Software is never correct no matter

DCS, NUCES, Islamabad Spring 2013 3

DCS, NUCES, Islamabad Spring 2013 4

 Humans make errors in their thoughts, actions, and in

 Errors occur wherever humans are involved in taking

 These fundamental facts of human existence make

DCS, NUCES, Islamabad Spring 2013 5

DCS, NUCES, Islamabad Spring 2013 6

DCS, NUCES, Islamabad Spring 2013 8

DCS, NUCES, Islamabad Spring 2013 10

DCS, NUCES, Islamabad Spring 2013 11

DCS, NUCES, Islamabad Spring 2013 12

DCS, NUCES, Islamabad Spring 2013 13

DCS, NUCES, Islamabad Spring 2013 16

 The way most testing works is to input a set of values, then

 Depending on the type of data input, we can identify three

DCS, NUCES, Islamabad Spring 2013 18

DCS, NUCES, Islamabad Spring 2013 20

DCS, NUCES, Islamabad Spring 2013 24

DCS, NUCES, Islamabad Spring 2013 25

DCS, NUCES, Islamabad Spring 2013 26

DCS, NUCES, Islamabad Spring 2013 27

DCS, NUCES, Islamabad Spring 2013 28

 Operability —it operates cleanly

DCS, NUCES, Islamabad Spring 2013 29

 A successful test -- -- that uncovers an as-yet undiscovered error.

There are 10 14 possible paths ! If we execute one

DCS, NUCES, Islamabad Spring 2013 33

... our goal is to ensure that all

Depending on the (adequacy) criteria to

1) Hypothesis: Faults cannot be discovered if the

Execute every statement in the

 Though often used, statement coverage is a

 Line 5: no difference if x > 1 condition was

correct or even missing

not all conditions.

 Select a test set T such that,

 We need to exercise all

 Also known as branch or

 We choose a test set with two test cases:

 We need to further strengthen the edge coverage

You must write enough

*Any problem with above 4 TCs?

 Path Coverage Criterion: Select a

 The value for cyclomatic complexity defines the

Minimum conditions—Simple Loops

1. skip the loop entirely

*where n is the maximum number of allowable

NU-Islamabad Spring 2013 72

NU-Islamabad Spring 2013 73

 Equivalence class partitioning

NU-Islamabad Spring 2013 75

 Equivalence partitioning uses the idea of equivalence classes.

2. Invalid Equivalence Classes

Input condition is a range: one valid and two

 Then define one test case for each

Input or Output Valid Equivalence Invalid Equivalence

Enter the first First character is a capital First character is not a

Draw a line From 1 to 4 inches long No line

NU-Islamabad Spring 2013 81

Next three numerics All numeric One char non-

Equivalence class table for customer-acc-number