Planning and Monitoring the

Process
Learning objectives

Understand the purposes of planning and monitoring

Distinguish strategies from plans, and understand their relation
Understand the role of risks in planning
Understand the potential role of tools in monitoring a quality process
Understand team organiation as an integral part of planning
!hat are Planning and Monitoring"
Planning#
$ %cheduling activities &'hat steps" in 'hat order"(
$ )llocating resources &'ho 'ill do it"(
$ Devising unambiguous milestones for monitoring
Monitoring# *udging progress against the plan
$ +o' are 'e doing"
) good plan must have visibility #
$ )bility to monitor each step, and to make objective judgments of
progress
$ ,ounter 'ishful thinking and denial
-uality and Process
-uality process# %et of activities and responsibilities
$ focused primarily on ensuring adequate dependability
$ concerned 'ith project schedule or 'ith product usability
) frame'ork for
$ selecting and arranging activities
$ considering interactions and trade.offs
/ollo's the overall soft'are process in 'hich it is embedded
$ 01ample# 'aterfall soft'are process $$2 34 model5# unit testing
starts 'ith implementation and finishes before integration
$ 01ample# 6P and agile methods $$2 emphasis on unit testing and
rapid iteration for acceptance testing by customers
Customer Requirements
01ample Process# ,lean room
Specification
Function Usage
Incremental
Development
Planning
Statistical test case
generation
Usage specifications
Formal Design
Correctness Verification
Functional specifications
Statistical testing
Source
code
Test cases
Quality Certification odel
TTF statistics
Interfail times
Improvement Feed!ac"
Customer Requirements
01ample Process# ,lean room
Specification
Function Usage
Incremental
Development
Planning
Statistical test case
generation
Usage specifications
Formal Design
Correctness Verification
Functional specifications
Statistical testing
Source code
Test cases
Quality Certification odel
TTF statistics
Interfail times
Improvement Feed!ac"
Activities and
responsibilities
focused on quality
Integrated into an
overall development
process
01ample Process# 01treme Programming
&6P(
#enerate User
Stories
Create Unit
Tests
Pair
Programming
$ unit testing
Create
%cceptance
Tests
Incremental
Release
pass
&e't version
Revie()
Refine)
prioriti*
e
%cceptance
Testing
Passed all
unit tests
Passed all unit
tests
Failed acceptance test
7verall 7rganiation of a -uality Process

8ey principle of quality planning

$
the cost of detecting and repairing a fault increases as
a function of time between committing an error and
detecting the resultant faults

therefore 999
$
an efficient quality plan includes matched sets of
intermediate validation and verification activities that
detect most faults within a short time of their
introduction

and 999
$
4:4 steps depend on the intermediate 'ork products
and on their anticipated defects
4erification %teps for ;ntermediate )rtifacts
;nternal consistency checks
$ compliance 'ith structuring rules that define 3'ell.formed5
artifacts of that type
$ a point of leverage# define syntactic and semantic rules thoroughly
and precisely enough that many common errors result in detectable
violations
01ternal consistency checks
$ consistency 'ith related artifacts
$ 7ften# conformance to a 3prior5 or 3higher.level5 specification
<eneration of correctness conjectures
$ ,orrectness conjectures# lay the ground'ork for e1ternal
consistency checks of other 'ork products
$ 7ften# motivate refinement of the current product
Strategy Plan
Scope Organization Project
Structure
and
content
based on
Organization
structure,
experience and
policy over several
projects
Standard
structure
prescribed in
strategy
Evolves Slowly, with
organization and
policy changes
Quickly,
adapting to
project needs
=est and )nalysis %trategy
Lessons of past e1perience
$ an organiational asset built and refined over time
>ody of e1plicit kno'ledge
$ more valuable than islands of individual competence
$ amenable to improvement
$ reduces vulnerability to organiational change &e9g9, loss of
key individuals(
0ssential for
$ avoiding recurring errors
$ maintaining consistency of the process
$ increasing development efficiency
,onsiderations in /itting a %trategy
to an 7rganiation
%tructure and sie
$ e1ample
Distinct quality groups in large organiations, overlapping of
roles in smaller organiations
greater reliance on documents in large than small
organiations
7verall process
$ e1ample
,lean room requires statistical testing and forbids unit testing
$ fits 'ith tight, formal specs and emphasis on reliability
6P prescribes 3test first5 and pair programming
$ fits 'ith fluid specifications and rapid evolution
)pplication domain
$ e1ample
0lements of a %trategy
,ommon quality requirements that apply to all or most products
$ unambiguous definition and measures
%et of documents normally produced during the quality process
$ contents and relationships
)ctivities prescribed by the overall process
$ standard tools and practices
<uidelines for project staffing and assignment of roles and
responsibilities
=est and )nalysis Plan
)ns'er the follo'ing questions#
!hat quality activities 'ill be carried out"
!hat are the dependencies among the quality activities and bet'een
quality and other development activities"
!hat resources are needed and ho' 'ill they be allocated"
+o' 'ill both the process and the product be monitored"
Main 0lements of a Plan
;tems and features to be verified
$ %cope and target of the plan
)ctivities and resources
$ ,onstraints imposed by resources on activities
)pproaches to be follo'ed
$ Methods and tools
,riteria for evaluating results
-uality <oals
01pressed as properties satisfied by the product
$ must include metrics to be monitored during the project
$ example: before entering acceptance testing, the product must pass
comprehensive system testing 'ith no critical or severe failures
$ not all details are available in the early stages of development
;nitial plan
$ based on incomplete information
$ incrementally refined
=ask %chedule
;nitially based on
$ quality strategy
$ past e1perience
>reaks large tasks into subtasks
$ refine as process advances
;ncludes dependencies
$ among quality activities
$ bet'een quality and development activities
<uidelines and objectives#
$ schedule activities for steady effort and continuous progress and
evaluation 'ithout delaying development activities
$ schedule activities as early as possible
$ increase process visibility &ho' do 'e kno' 'e?re on track"(
=est 01ecution
Example Risks
01ecution costs higher than
planned
%carce resources available for
testing
Control Strategies
Minimie parts that require full
system to be e1ecuted
;nspect architecture to assess
and improve testability

;ncrease intermediate feedback

;nvest in scaffolding
@equirements
Example Risk

+igh assurance critical

requirements increase
e1pense and
uncertainty
Control Strategies
,ompare planned testing effort 'ith
former projects 'ith similar criticality
level to avoid underestimating testing
effort
>alance test and analysis
;solate critical parts, concerns and
properties
,ontingency Plan
Part of the initial plan
$ !hat could go 'rong" +o' 'ill 'e kno', and ho' 'ill 'e recover"
0volves 'ith the plan
Derives from risk analysis
$ 0ssential to consider risks e1plicitly and in detail
Defines actions in response to bad ne's
$ Plan > at the ready &the sooner, the better(
Preliminary
plan
Preliminary
plan
First
release
First
release
Second
release
Second
release
+mergency
plan
+mergency
plan
Final
plan
Final
plan
,
0volution of the Plan
Process Monitoring
;dentify deviations from the quality plan as early as possible and take
corrective action
Depends on a plan that is
$ realistic
$ 'ell organied
$ sufficiently detailed 'ith clear, unambiguous milestones and
criteria
) process is visible to the e1tent that it can be effectively monitored
f
a
u
l
t
s
Builds
0valuate )ggregated Data by )nalogy
Process ;mprovement
Monitoring and improvement 'ithin a project or across
multiple projects#
7rthogonal Defect ,lassification &7D,(
:@oot ,ause )nalysis &@,)(