Escolar Documentos
Profissional Documentos
Cultura Documentos
- What is RISK
- Factors determining damage
- Factors determining probability
- A simple method to calculate risk
- Risk management in test projects:
Risks before, during and after the test
- Some more stuff, if there is time
Risk based testing
Slide no. 1
Slide no. 2
Strategy
Objective: Find the most important defects as early
as possible at the lowest price
No risk -> no test
Business based decision
Slide no. 3
What is risk?
RISK
Damage
Probability of
failure
Usage frequency
Lack of quality
Risk:= You dont know what will happen but you do know the probabilities
Uncertainty = You dont even know the probabilities.
Slide no. 4
Risk definition
Damage
financial, loss of (faith of) clients, damage to corporate identity
impact on other functions or systems
detection and repair time
Probability of failure
globally = (estimated) size * complexity
in detail = knowledge of development project (just before
testing)
Jukka Talvio von F-Secure: It shall be the duty of managers to make decisions and the
duty of engineers to make them informed ones.
Risk based testing
Slide no. 5
Risk
Strategy
Risk
Identification
Testing,
Inspection etc.
Risk
Assessment
Matrix: Cost
and Probability
Risk
Mitigation
Test Metrics
Risk
Reporting
Risk
Prediction
Slide no. 6
Risk analysis
Applicable on the level of
system
subsystem
individual function or module (e.g. insert new entry into phone
database)
Fundamental problems:
Difficult to measure
Failure to account for risk compensation (people compensate
for greater safety by taking more risks)
Slide no. 7
Risk analysis
Risk analysis should lead to a limited number of classes of
approximately equal risks
functional defects
bad performance
bad usability
low maintainability
...
Slide no. 8
R( f ) = P( f ) *C( f )
z
Slide no. 9
Top-20
Extra Testing:
- Extra Test by product specialist
- automated regression test
- ...
3
Slide no. 10
Slide no. 11
Slide no. 12
Complex areas
Changed areas
Number of people involved
Turnover
New technology, solutions,
methods
- New tools
Time pressure
Areas which needed optimizing
Areas with many defects before
Geographical spread
History of prior use
Local factors
Slide no. 13
Do not forget
Can we test ONLY PART of the product?
Other versions later?
Slide no. 14
Spreadsheet
http://home.c2i.net/schaefer/testing/Riskcalc.xls
Slide no. 15
Example
Area to test
Usage
frequency
Visibility
Complexity
Geography
Turnover
Weight
10
Function A
1125
Function A
performanc
e
Function B
1530
368
FB
usability
377
Function C
572
Function D
240
SUM
Slide no. 16
Slide no. 17
Slide no. 18
A shorter method:
Determine the relative importance of
quality characteristics
(They depend on value and possible damage).
Functionality
50
Reliability
20
Usability
20
Efficiency
Maintainability
Portability
0
2004 Hans Schaefer
Slide no. 19
Reliability
30
Usability
40
Paper review,
Usability lab
Efficiency
10
No test
Flexibility
(maintain)
20
Design review
Monitoring of
repairs
Slide no. 20
Slide no. 21
Slide no. 22
Slide no. 23
Slide no. 24
Slide no. 25
Slide no. 26
Analysis of causes
If you have many defects with the same cause category, think about
improving your way of working!
Typical for unit testing:
Logic
Computation
Interfacing
Data handling
Input data problem
Documentation
Change
Slide no. 27
Slide no. 28
Delays
-> Alternative plans, more parallel work
Lack of knowledge
-> Test of earlier versions
Slide no. 29
Slide no. 30
Slide no. 31
Slide no. 32
Slide no. 33
Test reporting,
risks and benefits
addresses
Testing
Risks
th
re
at
en
demonstrates
Benefits
informs about
Project
status
Slide no. 34
Residual Risks
Start
Planned
end
all risks
open at
the start
residual risks
of releasing
TODAY
Slide no. 35
Benefits
Start
Planned
end
all benefits
unachieved at
the start
residual
benefits not yet
realized if
releasing
TODAY
Slide no. 36
References
IEEE Standard 1044-2002: Standard Classification for Software Anomalies
IEEE Standard 1044.1-2002: Guide to Classification for Software Anomalies
Soon to come: IEEE Std. 16085 Standard for Software Engineering Software Life Cycle Processes - Risk Management
You find them at sales@ieee.org
Rex Black, Managing the Testing Process, John Wiley, 2002. (includes CD with a test priority
spreadsheet)
Hall, Payson: A Calculated Gamble. In STQE Magazine No 1 +2 / 2003.
Stamatis, D.H., Failure Mode and Effect Analysis: FMEA from Theory to Execution, Amer Society
for Quality, 1995.
Schaefer, Hans: Strategies for Prioritizing Test, STAR WEST 1998.
http://home.c2i.net/schaefer/testing/risktest.doc
James Bach, Risk Based Testing, STQEMagazine, Vol1, No. 6,
www.stqemagazine.com/featured.asp?stamp=1129125440
Felix Redmill in Professional Tester, April 2003. www.professional-tester.com
Tom DeMarco and Tim Lister, "Waltzing with Bears: Managing Risk on Software Projects, 2003.
www.riskbasedtesting.com
Risk based testing
Slide no. 37
Questions?
Slide no. 38