SOFTWARE TESTING

Fundamentals of testing
Topics
Fundamentals of testing
Role of testing
Static testing
Dynamic testing
Test management
Testing tools
Testing object oriented software

2
In god we
trust,
everything
else we
test
3
Software testing
Testing is the process of
executing a program with
the intent of finding
errors.
-Glen Myers
4
Software testing
Testing is questioning a
product in order to evaluate
it
-Cem Kaner
5
What is software testing?
Software testing is a process of analyzing or
operating software for the purpose of finding
bugs.

Testing is the process of identifying defects, where
a defect is any variance between actual and
expected results.

Software testing is the process used to help to
identify the correctness, completeness, security
and quality of developed computer software.

6
What is software testing?
Testing is generally described as a group of
procedures carried out to evaluate some
aspect of a piece of software.

Testing can be described as a process used
for revealing defects in software, and for
establishing that the software has attained a
specific degree of quality with respect to
selected attributes.
7
Phases in a tester's mental life(0)
Phase 0: (Until 1956: Debugging Oriented)
There is no difference between testing and
debugging.
Phase 0 thinking was the norm in early days of
software development till testing emerged as a
discipline.
8
Phases in a tester's mental life(1)
Phase 1: (1957-1978: Demonstration Oriented)
The purpose of testing here is to show that
software works.
Highlighted during the late 1970s.
This failed because the probability of showing
that software works 'decreases' as testing
increases.
i.e. The more you test, the more likely you'll find
a bug.

9
Phases in a tester's mental life(2)
Phase 2: (1979-1982: Destruction Oriented)
The purpose of testing is to show that software
doesn't work.
This also failed because the software will never
get released as you will find one bug or the other.
Also, a bug corrected may also lead to another
bug.
10
Phases in a tester's mental life(3)
Phase 3: (1983-1987: Evaluation Oriented)
The purpose of testing is not to prove anything but
to reduce the perceived risk of not working to an
acceptable value (Statistical Quality Control).
Notion is that testing does improve the product to
the extent that testing catches bugs and to the
extent that those bugs are fixed.
The product is released when the confidence on
that product is high enough. (Note: This is applied to
large software products with millions of code and
years of use.)



11
Phases in a tester's mental life(4)
Phase 4: (1988-2000: Prevention Oriented)
Testability is the factor considered here.
One reason is to reduce the labour of testing.
Other reason is to check the testable and non-
testable code.
Testable code has fewer bugs than the code
that's hard to test.
Identifying the testing techniques to test the
code is the main key here.

12
Why do we test software?
To demonstrate that the software is
doing what it is supposed to do as
well as the software is not doing what
it is not supposed to do.
e.g. medicine
13
Sales representative / Engineer:
This car has the best possible
transmission and brake , and
accelerate from 0 to 80mph in 20
seconds!
Customer: Well, that may be
true, but unfortunately it
accelerates (even faster) when I
press the brake pedal!

Testing should focus on finding
defects before customers find them.
14
Dijkstras doctrine
Consider a problem that is supposed to accept
six character code and ensure that the first
character is numeric and rest of the characters
are alphanumeric. How many combinations of
input data should we test the program
exhaustively?

Testing can only prove the presence of
defects, never their absence.
15
The Cat and The Saint
Why one tests is
as important as
What to test and
How to test
16
Test the tests first!
Audiologist and patient
Doctor (from 30 feet): What is your name?
..
Doctor (from 20 feet): What is your name?
..
Doctor (from 10 feet): What is your name?
Patient: For the third time, let me repeat
My name is Sheela!
Test the tests first- a defective test is
more dangerous than a defective product.
17
Fundamental objective
The fundamental
objective of software
testing is to find defects,
as early as possible and
get them fix.
18
Testing ballpoint pen
Does the pen write in the right color,
with the right line thickness?
Is the logo on the pen according to
company standards?
Is it safe to chew on the pen?
Does the click-mechanism still work
after 100 000 clicks?
Does it still write after a car has run
over it?

What is expected from this pen?
Intended use!!
19
Goal: develop software to meet its intended use!
But: human beings make mistake!





Product of any engineering activity must be
verified against its requirements throughout its
development.
20
Verifying bridge = verifying design,
construction, process,

Software must be verified in much the same
spirit. In this lecture, however, we will identify
that verifying software is perhaps more difficult
than verifying other engineering products.

21
Why software has bugs?
Ambiguous/ Unclear requirements
Increased complexity of software
Programming errors
Poor documentation/ Knowledge transfer
Communication gap
Inability to manage change
Time pressure/ Deadlines
22
Why software has bugs?
23
Software development
24
Error, Fault, Failure
25
Error
People make errors. A good synonym is
mistake.
When people make mistakes while coding, we
call these mistakes bugs.
Errors tend to propagate; a requirements error
may be magnified during design and amplified
still more during coding.
26
Fault
A fault is the result of an error.
It is more precise to say that a fault is the
representation of an error, where
representation is the mode of expression, such
as narrative text, data flow diagrams, hierarchy
charts, source code, and so on.
Defect is a good synonym for fault, as is bug.
Faults can be elusive.

27
Fault
When a designer makes an error of omission, the
resulting fault is that something is missing that
should be present in the representation.

We might speak of faults of commission and faults
of omission.
A fault of commission occurs when we enter
something into a representation that is incorrect.
Faults of omission occur when we fail to enter correct
information. Of these two types, faults of omission are
more difficult to detect and resolve.

28
Failure
A failure occurs when a fault executes.
Two points arise here: one is that failures only
occur in an executable representation, which
is usually taken to be source code, or more
precisely, loaded object; the second point is
that this definition relates failures only to faults
of commission.
How can we deal with failures that correspond
to faults of omission?
29
Program behavior
30
Types of faults
Algorithmic: division by zero
Computation & Precision: order of op
Documentation: doc - code
Stress/Overload: data structure size ( dimensions of
tables, size of buffers)
Capacity/Boundary: x devices, y parallel tasks, z interrupts
Timing/Coordination: real-time systems
Throughout/Performance: speed in req.
Recovery: power failure
Hardware & System Software: modem
Standards & Procedure: organizational standard; difficult
for programmers to follow each other.
31
The policeman and the bridge
Prevention is better
than cure you may
be able to expand
your horizon much
farther.
32
Testing vs. Debugging
Testing Debugging
Testing starts with known
conditions, uses predefined
procedures and has
predictable outcomes.
Debugging starts from
possibly unknown initial
conditions and the end can
not be predicted except
statistically.
Testing can and should be
planned, designed and
scheduled.
Procedure and duration of
debugging cannot be so
constrained.
Testing is a demonstration of
error or apparent correctness.
Debugging is a deductive
process.
Testing proves a
programmer's failure.
Debugging is the
programmer's justification.
33
Testing vs. Debugging
Testing Debugging
Testing, as executes, should
strive to be predictable, dull,
constrained, rigid and
inhuman.
Debugging demands intuitive
leaps, experimentation and
freedom.
Much testing can be done
without design knowledge.
Debugging is impossible
without detailed design
knowledge.
Testing can often be done by
an outsider.
Debugging must be done by
an insider.
Much of test execution and
design can be automated.
Automated debugging is still
a dream.
34
What is defect?
The software doesnt do something that the
product specification says it should do.
The software does something that the product
specifications say it shouldnt do.
The software does something that the product
specification doesnt mention.
The software doesnt do something that the
product specification doesnt mention but
should do.
Difficult to understand, hard to use, slow etc.
35
Software testing process
Test planning
Test design
Test implementation (in case of automation)
Test execution
Test analysis
Postmortem reviews
36
Test initiation criteria
Timing: as soon as we have software
requirements (baseline).
Objective
To trap requirements-related defects as early as
they can be identified.
Approach
Test requirements
37
Test completion criteria
Following factors are considered
Deadlines; e.g. release deadlines, testing
deadlines
Test cases completed with certain percentage
passed
Coverage of code, functionality, or
requirements reaches a specified point
Bug rate falls below a certain level; or
Beta or alpha testing period ends
38
Participants in testing
Customer
Users
Developers
Includes individual/group which gather
requirements, design, build, change and maintain
software
Testers
Senior management
Auditors

39
Challenges in testing
Testing considered late in the project
Requirements not testable
Integrate after all components have been
developed
Test progress hard to measure
Complete testing is not possible
40
Economics of testing
Customer
Dissatisfaction
Number of
defects
Under testing
Duration
Quantit
y
Optimum
testing
Cost of testing
Over testing
41
The Psychology of testing
42
People make mistakes, but they do not like to
admit them!
Developer test
Blindness to ones own errors
Independent testing team
Reporting of failures
Mutual comprehension
General principles of testing
43
1. Testing shows the presence of defects, not their
absence
2. Exhaustive testing is not possible
3. Testing activities should start as early as
possible
4. Defects tend to cluster together
5. The pesticide paradox
6. Test is context dependent
7. The fallacy of assuming that no failures means a
useful system

Quality, QA and QC
44
Quality is meeting the requirements expected of
the software, consistently and predictably.
Expected behavior is characterized by as set of
test cases. Each test case is characterized by
The environment under which the test case is to be
executed
Inputs that should be provided for that test case
How these inputs should get processed?
How changes should be produced in the internal state
on environment; and
What output should be produced?
Quality, QA and QC
45
The actual behavior of a given software for a
given test case, under given inputs, in a given
environment, and in a given internal state is
characterized by
How these inputs are actually get processed?
What changes are actually produced in the internal
state or environment; and
What outputs are actually produced.

If actual and expected behavior are identical then
test case is passed otherwise the given software
has defect on that test case

Quality Control
46
Build the product
Test it for expected behavior after it is built
If expected behavior is not same as the actual behavior
of product, fixes the product as is necessary
Rebuild the product

Above steps are repeated till expected behavior of
product matches the actual behavior for scenarios
tested.
QC is defect detection and defect correction oriented
QC works on product rather than process
Quality Assurance
47
Attempts defect prevention by concentrating on
producing the product rather than on defect
detection/ correction after the product is built.

Review the design before product is built
Correct he design errors
Production of better code (coding standards to be
followed)
QA continued throughout the life of the product
hence it is a staff function (Everybody's
responsibility)
QA vs. QC
48
Quality Assurance Quality Control
Concentrate on process of
producing the products
Concentrates on specific
products
Defect prevention oriented Defect detection and
correction oriented
Usually done throughout
the life cycle
Usually done after the
product is built
This is a staff function This is a line function
Examples: Reviews and
audits
Example: Software testing
at various levels
Knows as verification Known as validation
/testing
Bug life cycle
49