Training Review Technique

Training
Review Technique
for Object Reviews
2 / Hubert Gast / 14.Feb.2008 Continental AG
Agenda
Benefits
Review Types
Review Planning, Review Execution
Hints for effective Reviews
Guidelines and best practices
Focus of this Training are Object Reviews (reviewing Documents
according to P730103, not Milestone Reviews acc. to P730128) !
Benefits
Earlier detection of defects
less risk in late project phases
cheaper defect removal
much lower maintenance costs
better quality of documents, esp. basic documents like System Design, Functional Spec
comparable quality level of all reviewed documents in project
guaranteed appliance of project wide Guidelines & Rules
Know-how sharing within project
Disadvantage: Time spent for Review (Admin, Preparation, Meeting) is lost if Review
results does not lead to Document update
Benefit of Reviews: Development Costs
0
1
2
3
4
5
6
Analysis Design Coding Mod test Sys test Use
Normalized Cost for Defect Detection and Removal
Require
Design
Code
the earlier defects are detected, the cheaper is the defect removal
Without
reviews
Using
reviews
P
e
o
p
l
e

R
e
s
o
u
r
c
e
Design Code Testing Ship
Schedule
Source: M. Fagan, IBM 1986
In the 1st half of a project
reviews can achieve 20-30% of
the development effort
Planning/
Requirements
Benefits: Resources with and without reviews
Review types, main goals
Note: terms and their meaning vary in literature - this is our definition in PDMK
Intensive Inspection: thorough check of a document, reader, by preparation and
meeting ( find defects)
Inspection: thorough check of a document by preparation and meeting
( find defects)
Reading: no meeting, only collecting issues found during reading
( find defects)
4 Eye Review: meeting, no preparation, informally collect issues
( find defects e.g. after changes)
Walkthrough: "walk through" document in a meeting
( understanding, find defects, know-how)
Workshop: discussion about document contents in a meeting
( decision making)
f
o
r
m
a
l

l
e
v
e
l
Review types: pro & contra
Intensive +: most effective finding major defects of a document
Inspection: +: additional items are often found during meeting
-: most time consuming review method (reader)
Inspection: +: very effective finding major defects of a document
+: additional items are often found during meeting
-: takes more time than reading
Reading: +: efficient if all participants take time for checking
+: independent of location of participants
-: no additional issues found which would come up in logging meeting
4 Eye Review: +: efficient, few participants, informal logging
-: not central documents, only for minor changes
Workshop: +: structured discussion and logging of issues, results, solutions
-: not very efficient for finding defects
Walkthrough:+: deep understanding for all of difficult parts of a document
-: takes much time, not efficient for finding defects
new Review type: Intensive Inspection
(additional items to Inspection in orange). According to P730103d, this review type must be used for very complex documents
of high importance (central documents like SW System Design etc.)
In the Intensive Inspection the following roles must be used:
Moderator
Author (must not be the same person as the Moderator)
Reader: is reviewer, with the special task to explain the review object in his own words
Reviewer with special understanding of the review object, e.g. designer, developer, tester
It follows a defined multistage process with the following steps:
Planning: check of the review object against entry criteria, select reviewer, distribute invitation.
Introduction: Active introduction of the review object (topics, structure) by the author. This can be a meeting separate from
the review meeting.
Preparation: every reviewer checks the review object, especially following his special role. (if available, he uses the
checklist).
Inspection Meeting: check, if the reviewer are sufficiently prepared, the reader explains the review object in his words,
discussion of open topics, different understanding, the Moderator records all comments/findings and open topics. Decide
upon re-inspection.
Error analysis (Development and Review Process Improvement): The causes for errors allowed by the development
process should be determined and described in the Review Protocol. Problems encountered or improvement suggestions
in the use of the Intensive Inspection method should be described in the Review Protocol.
Re-Work/Update of the review object.
Acceptance: Moderator checks, if the author as the responsible has corrected all findings, has clarified all open items and
initiated the handling of findings in external documents.
new Review type: Four Eyes Inspection
The Four Eyes Inspection is an informal inspection with at least two participants: author
and expert.
No preparation is necessary.
The use of the Review Minutes Template is not mandatory, also a handwritten list or a
commented papercopy of the document can serve as Review Minutes. Nevertheless the
author is responsible to file the Review Minutes, even if there's no need to use the
configuration management therefor.
it is advised either to file the papercopies in a central 4-Eye-Review binder or even
better- check out the document, change it online while doing the 4-Eye-Review and
check it in again after the 4-Eye-Review with an History Log entry like "after 4-Eye-
Review"
The review result (statistical data) is entered into the Review List.
As this review type is very informal, it is usable only for documents without central
importance or to verify minor changes.
Review Planning - Aim
Which Document is reviewed when, howand by
whom?
select documents for review
set review date soon after document should be available
choose appropriate review type
name the review participants
enter all this data in Review List
Review Planning Process
see PDMK:
Methods Quality Management: Object Review Planning
Object Reviews are also done on System level and in ME + EE
common Object Review List
Review Planning
plan reviews according to PDMK / SMK tables by:
phase
document with priority (tailoring!)
recommended reviewers
recommended Review Type
see PDMK: Methods Quality Management: Choosing Review Type, Object
and Participants, Object Review Plan
see SMK: Methods Quality Management: Software Object Review Plan
Review Planning: Review List
see PDMK: Documents Quality Management: Review List
see Review List Tab "Guidelines" for fill out instructions
the planned review is entered into the Review List with:
review object, document type, date, reviewers, moderator
enter all data thoroughly and completely, as this list is not only the
"Review Database" for the project, but also serves as data source for
annually Review Process improvements
Review Planning: Review List usage
Review Status:
See PDMK: Methods Quality Management: Object Review List Usage
The Review Status is set to PLA after the review has been planned, then it changes to TDO after
invitation, followed by REV after the review meeting (or reading completion).
DNE is the last state if the review has been completed successfully.
If a review has to be deferred, its state changes to DEF, until a new review date is assigned. This state
change should only be done, if a review date has to be shifted for more than 2 weeks. If a review is
postponed only for some days (e.g. review participant not available), the review state can remain in
TDO, just the updated Planned Review Date is added in the Review List, the first Planned Review Date
is set in brackets behind the updated one.
Cancelling a review can be done in the states PLA, TDO, DEF or REV and is documented by setting its
state to REJ (rejected) in the Review List.
Review Execution
how a review is carried
out:
see PDMK: Methods
Quality Management:
Object Review Execution
Review Execution: Review Minutes
see PDMK: Documents Quality Management: Object Review Minutes
Fill out title page, footer and Overview section completely and thoroughly
enter all comments in the list, so that the document author understands them
assign one of the types F,I,D,O,K to each comment, see PDMK: Methods Quality
Management: Object Review Minutes
after each item is commented by the author, the moderator can spot check the
implementation and set the Review State to DNE
Review Minutes have to be checked in (demanded by P730103)
Review Execution: Review Minutes: "D" issues
Benchmark Subproject: Process Optimization - Cost Reduction Measure #1 demanded from
01.04.2007 on:
Leave out all D (Improvement of presentation) entries in Object Review
Minutes and Lists
PDMK Method: Quality Management: Object Review Minutes:
ncontinue to bring up documentation issues (D points) in the review meeting
nbad readability remains a violation of review entry criteria postpone review
nD points are only listed in Review Minutes if document may be read by customer, for all other
documents D issues are not recorded any more
nfor all non-customer relevant documents, D count remains empty in Review Minutes and
Review List
ndocument author should note and implement important D issues on his own responsibility to
improve readability of the document, however implementation of those is not checked by Review
Process
PDMK Template Object Review Minutes:
footnote 2: ... D=Improvement of presentation (readability), ...: no change!
PDMK Template Object Review List:
empty D count fields are not colored any more (was indicator for missing entry)
Review Execution: Test Specifications
only review the "Test Objectives" when reviewing Test Specifications
the Test Objectives should give a good representation of what is tested
items forgotten in the test will show up quickly when focusing on Test Objectives
the details of each single test case are not so important and likely to be correct
(otherwise test fails), so checking those is not needed and not efficient
this is a "cost reduction measure", but already has been shown to raise
effectiveness in the past (before 01.April 2007)
Hints for effective Reviews
Use reviews to check all important documents in a project
more detailed planning of reviews: what / who / when?
set a focus: what should be checked in the review?
Split document if it is too big!
choose most effective review type
do a Walkthrough of critical parts during a document Inspection
select right participants (document owner, experienced people)
build up review experience for all project members (newbies!)
apply best practices based on lessons learned from earlier reviews (use laptop, beamer, BeyondCo.)
use checklists where they make sense
QA people are not needed for each review
at least 50% of all reviews should be made without QA support
QA does spot-checks of review planning, protocol and follow-up
effective Reviews: Planning: Choice of objects
late changes: short time to delivery or SOP, 'quick & dirty
complex changes, new features with influence throughout many other documents
high change request rate, many changes since last review (see MR DB)
new developer (internal or external)
modules with many warnings generated by automated code analysis (see QA C results)
complex modules or interfaces, with great influence on the system (see Understand C
results, product metrics)
modules using data from more than 1 interrupt or task level, and 1 level can interrupt
another in a pre-emptive way: data consistency/ shared memory access problem
more frequent or severe problem known in the project
effective Reviews: Planning: size of objects
Plan Review Meeting no longer than 2 hours, include breaks
Consider optimum checking rate for a deep and efficient exploration of about 100-140 NLOC/h or
3-5 pages/h (300 words of content per page) (Only a rough guideline)
Divide documents (tasks) into suitable chunks:
assign different (parts of) checklists to roles
concentrate on changes, or other special parts of documents
changes should be marked with changes bars, use a diff tool
check many modules, but for only 1 checklist item or problem, a more frequent or expensive
one reported in a late development phase
use more than 1 inspection or checking phase
A good design with small modules is a very great help - Design for Testability! (see Understand C /
product metrics)
15
12
9
6
3
20 40 60 80 100
D
e
f
e
c
t

d
e
n
s
i
t
y

(
d
e
f
e
c
t
s
/
p
a
g
e
)
Inspection rate (pages/hour)
Source: Tom Gilb, Denise Leigh: Software Inspection p 334,
230 inspections of Sema Group (GB)
you will find more defects when inspecting pages longer
not more than 20 pages should be inspected per hour,
or defects are not found!
one review should not last longer than 2 hours
=> if a document has more than 40 pages,
chunk document in parts and inspect each
part separately!
effective Reviews: Planning: size of objects
effective Reviews: Planning: Choice of participants
Persons who are responsible for the object (module, document) or others where a good
quality of the object is in their own interest, e.g. users of objects,
Experienced designers and integrators, who have the overview over the design, the
system and its requirements. Might be a bottleneck!
Do not change the whole review team between reviews concerning 1 module, e.g.
between specification - design - test specification - code.
Participants can also be persons from other departments, e.g. from system testing or
from the customer. It can be useful to have some more perspectives to the object, e.g. a
user or legal perspective.
New people which are not yet routine-blinded may bring in new ideas or raise interesting
questions.
effective Reviews: Experience Data
The following statistical Experience Data has been determined by yearly analysis of our
review data:
the average duration of the Logging Meeting is 2 hours
the average Review Effort is about 8 hours
(Review Effort = # of participants*Logging meeting time + preparation time +
Administration time)
the average number of Review participants is around 3
80
40
23
8
0
1 2 3 4 5
Experience as Checker: Also other
documents produced by Gary became better
after participating several inspections.
Gary was document
author at points 1 to 5
Issues Identified
Source: Douglas Aircraft, 1988,
private report to Tom Gilb
effective Reviews: Personal Learning Curve
Extra motivation (Kick-off)
Moderator
should encourage beginners and shy persons, ask for help and give tips.
Beginners should start small, not with all rules, but with important.
should ask - not wait - for suggestions for improvement
Communicate!
Known problems or open items should be solved before a review takes place!
If there are major problems encountered all participants have to be informed
immediately so that the review procedure can be aborted as quickly as possible!
If a participant cannot join the team, the moderator should contact his/her boss to
change priority.
The 'hot seat'
Documents are checked, not the author:
'choice of words', e.g. here you made another mistake: ...
no big boss
No improvement suggestions to the author during logging meeting, but
afterwards
Planning of review participants should avoid possible personal friction between
author and checkers: checkers should dare to raise items
Consider: Everyone will have this role!
Checking
Focus on majors = economical important defects, which cannot be found by
tools - e.g. static code analysis - nor module test. Checklists should reflect this.
In reality this is difficult, many minors are found much more often. Though it is a
good feeling to find a high number of issues, it is of course much better to find 1
major instead of 100 minors only.
Minors like syntax errors in a comment should be sent to the author beforehand,
without an extra logging.
The checking rate used as a guideline for planning reviews and the planned
duration is no law: stop checking when run dry
choose best environment, with no interrupts like telephone calls possible,
set a goal: 70%majors!
Logging meeting
Try to express the content of a logging item in a few precise words.
The author should watch the log (real time log validation), or should express the
content of a logging item in his own words. The author is normally the editor, so he
must understand what is logged.
Use the auto-save function when logging to a sheet on a PC !
'No' discussion, just log it!
If an item cannot be solved in a short time, or a question cannot be answered
quickly, the discussion should be deferred to a 3rd hour, only with the people
involved.
Use Beamer and Laptop to show document and protocol
Thank you for your attention
are there further questions?

Training Review Technique

Enviado por

Dados do documento

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Training Review Technique

Enviado por

Direitos autorais:

Formatos disponíveis

Training

Você também pode gostar