SCAMPI Appraisals

John Kennedy
The MITRE Corporation
21 March 2007
 Agenda
9 „ What is a SCAMPI Appraisal?
„ Resource Requirements
„ Schedule
„ Scope
„ Interviews
„ Discussion
 Definitions
„ Appraisal
„ An examination of one or more processes by a
trained team of professionals using a model as
a basis for determining strengths and
weaknesses
„ SCAMPI
„ Standard CMMI Appraisal Method for Process
Improvement
 Appraisal Principles

„ Start with a Process Reference Model

„ Use a Defined Appraisal Method
„ Involve Senior Management as
Appraisal Sponsors
„ Observe Non-Attribution
„ Approach the Appraisal Collaboratively
„ Focus on Action
 Basic Appraisal Precepts
„ Non-Attribution
„ Nothing seen or heard during an appraisal will be
attributed to a person or project
„ Confidentiality
„ All artifacts provided to the team during the
appraisal will either be returned to the appraised
organization, destroyed, or deleted
„ Appraisal findings are the property of the appraisal
sponsor(s) and will not be divulged or used outside
the context of the appraisal without the specific
permission of the sponsor(s)
 Appraisal Method Classes
Characteristics Class A Class B Class C
Amount of objective High Medium Low
evidence gathered
Ratings generated Yes No No
Resource needs High Medium Low
(relative)
Team size (relative) Large Medium Small
Appraisal Team Leader Lead Lead Lead
Requirements Appraiser Appraiser Appraiser
or Team or Team
Leader Leader
CMMI-SE/SW/IPPD/SS v1.1
Process Areas (Staged)
Level Focus Process Areas
Continuous Quality
Causal Analysis and Resolution (CAR)
process Productivity
5 Optimizing Organizational Innovation & Deployment (OID)
improvement

4 Quantitatively Quantitative Quantitative Project Management (QPM)

Managed management Organizational Process Performance (OPP)

Organizational Process Focus (OPF)

Organizational Process Definition (OPD)
Organizational Training (OT)
Integrated Project Management (IPM)
Process Risk Management (RSKM)
3 Defined standardization Decision Analysis and Resolution (DAR)
Requirements Development (RD)
Technical Solution (TS)
Product Integration (PI)
Product Verification (VER)
Validation (VAL)
Integrated Teaming (IT)
Organizational Environment for Integration (OEI)
Integrated Supplier Management (ISM)

Requirements Management (REQM)

Basic Project Planning (PP)
2 Managed
project Project Monitoring and Control (PMC)
management Measurement and Analysis (MA)
Process and Product Quality Assurance (PPQA)
Configuration Management (CM)
Supplier Agreement Management (SAM)
Risk
1 Performed Rework
 CMMI SE/SW/SS/IPPD
(Continuous Representation)
Process
Management
• Organizational Process Focus
• Organizational Process
Definition
• Organizational Training
• Organizational Process
Performance
• Organizational Innovation
and Deployment
CMMI
Project
Management Engineering Support
• Project Planning • Requirements Management • Configuration Mgmt.
• Project Monitoring and • Requirements Development • Process and Product
Control • Technical Solution Quality Assurance
• Supplier Agreement Mgmt. • Product Integration • Measurement &
• Integrated Project Mgmt. • Verification Analysis
• Integrated Supplier Management • Validation • Decision Analysis and
• Risk Management Resolution
• Quantitative Project Mgmt. • Causal Analysis and
• Integrated Teaming Resolution
• Organizational
Environment for
Integration
Systems & Software Engineering Process Areas (PAs)
Supplier Sourcing PA
Integrated Product and Process Development PAs
 CMMI-DEV v1.2
Level Focus Process Areas
Continuous
Process Organizational Innovation and Deployment
5 Optimizing Causal Analysis and Resolution
Improvement
4 Quantitatively Quantitative Organizational Process Performance
Managed Management Quantitative Project Management

Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
3 Defined Process Organizational Process Definition +IPPD
Standardization Organizational Training
Integrated Project Management +IPPD
Risk Management
Decision Analysis and Resolution

Requirements Management
Project Planning
Basic Project Monitoring and Control
2 Managed Project Supplier Agreement Management
Management Measurement and Analysis
Process and Product Quality Assurance
Configuration Management
Risk
Rework
1 Initial
Structure of the CMMI®
Staged Representation
Maturity Levels

Process Area 1 Process Area 2 Process Area n

Specific Generic
Goals Goals

Specific Generic
Practices Practices

Adopted from a chart by the Software Engineering

Institute, Carnegie Mellon University
 Reasons for Appraisals And
Assessments
„ Generating Maturity Level or Capability Level
Ratings (A)
„ Source Selection (B)
„ Status of Process Implementation (B)
„ Evaluation for Award or Incentive Fee (B)
„ Internal Snapshot, Mini-Assessment (B/C)
„ Education, Gap Analysis (B/C)
„ Contract Monitoring (B/C)
„ Risk Management (B/C)
„ Planning an Improvement Strategy (C)
 Source Selection SCAMPI
„ Timely and Objective Information
„ Zeroes in on the Critical Contractor
Capabilities
„ Identifies Strengths
„ Weaknesses
„ Areas of Risk to Program Success
„ Information for Planning and Contract Negotiation
„ Basis for award Fee?
„ CMMI Level 3 or not?
 Risk Management SCAMPI
1. Identify the Process Areas (PAs) Most
Critical to an Organization’s Success
2. Conduct a SCAMPI B of Those PAs to
Identify
a. Strengths
b. Opportunities to Improve (Gaps between
the CMMI and What Is Being Done)
c. Planning Information
 Agenda
„ What is a SCAMPI Appraisal?
9„ Resource Requirements
„ Schedule
„ Scope
„ Interviews
„ Discussion
 Resource Requirements - 1
„ Certified Lead Appraiser or Team Leader
„ Qualified Team Members
„ Training Requirements
„ Introduction to CMMI
„ SCAMPI Team Training
„ Team Knowledge/Experience Requirements
„ 5 Yrs Engineering, 3 Yrs Management
„ Domain Knowledge is necessary and At Least Half of
the team should have appraisal experience
„ Logistic Support at the Appraisal Site
 Resource Requirements - 2
„ Travel
„ Resource Requirements depend on:
„ Mode of Evidence Review (Discovery or
Verification)
„ Scope (Slide 22)
„ Other Sponsor Information Requirements
„ Scope Determines:
„ Team Size and Composition
„ Time Required
 Agenda
„ What is a SCAMPI Appraisal?
„ Resource Requirements
9„ Schedule
„ Scope
„ Interviews
„ Discussion
 Typical Appraisal Activities
„ Planning and Preparation
„ Appraisal Input and Plan
„ Team Selection and Training
„ Readiness Review
„ Conduct Appraisal
„ Document Review
„ Interviews
„ Collation of Findings
„ Preliminary Findings Presentation
„ Report Results
„ Final Findings Presentation
„ Executive Level Outbrief
„ Appraisal Disclosure Statement
 Generic SCAMPI C Timeline

Conduct Report
Phase Plan and Prepare for Appraisal
Appraisal Results

Plan Appraisal PIID Development On Site Required

Activities Select and Train Team Readiness Review Activities Docs to SEI

Appraisal
Artifact Preliminary
Plan Logistics
Appraisal PIID Development Review and Recorded
Deliverable(s) Trained Planning
Input Preliminary Artifact Review Readiness Final Appraisal
Appraisal PIID
Review Findings
Team Planning

Hours required:

Team Leader

ATM

ATM

ATM
 Agenda
„ What is a SCAMPI Appraisal?
„ Resource Requirements
„ Schedule
9„ Scope
„ Interviews
„ Discussion
 Preliminary Steps
With the Appraisal Sponsor:
„ Review and discuss program work

requirements in terms of processes

„ Decide which Process Areas are most

critical to the success of the program and

should be appraised
„ Other important Program Information

Needs
 What is Scope?
„ Scope is
„ The Number of Process Areas to be Appraised and
„ Other important Program Information Needs
„ Scope should include
„ Process Areas most critical to the success of the
program
„ Whatever else the sponsor wants, e.g.,
„ Strengths and Weaknesses

„ Observations

„ Recommendations for Implementation

„ Process Improvement Plan

„ Scope Determines:
„ Appraisal Team Size and Composition
„ Time Required
 Scope: “You Must Decide”
„ Business and Technical Objectives
„ Program Risks
„ Where Failure Is Unacceptable
„ What Is Negotiable/Not-Negotiable
„ Dependencies and Constraints
„ Historical Information
„ Other Program Information Needs
 )
g ce
in ien
um er
ss xp
(A E
ze nd
Si s , a

# PAs To Be Appraised
am on
Te cati
ifi
al
Qu
Team Size

Quality of Findings
Level of Detail and
 Agenda
„ What is a SCAMPI Appraisal?
„ Resource Requirements
„ Schedule
„ Scope
9„ Interviews
„ Discussion
 Role of Interviews
„ Draw on existing expertise and knowledge of
resident management, engineering, and
support professionals
„ Supplement knowledge gained from review of
organizational and project documents
„ Start the thinking process about what it would
really take to further improve the
organization's development capabilities
„ Obtain buy-in of discussion participants in
assessment outcome
 Types of Interviews
„ Project Leader interviews
„ If multiple projects are being appraised in an
organization, one session for each project
„ Functional area interviews
„ Management group or groups
„ Engineering group or groups
„ Support group or groups
„ Process group or groups
„ Organizational Roles
„ Senior/Middle Managers
„ Project Leaders
„ Practitioners
 Format of Interviews
„ Who will be there?
„ For some interviews, the entire team will be
present; for others, a team sub-set will be present
„ Follow-up interviews may be with a team sub-set
„ What to bring with you?
„ You may bring a few items that are particularly
helpful in your daily work, or that typify what you
produce
„ What kind of questions will be asked?
„ There may be some broad, general questions;
however, interview questions will generally be
narrowly focused and rather detailed
 About Interviews

„ It is OK to say
„ I don’t know.
„ We are not at that stage yet
„ The only bad answer is a disingenuous
answer
 Agenda
„ What is a SCAMPI Appraisal?
„ Resource Requirements
„ Schedule
„ Scope
„ Interviews
9„ Discussion
Y’all got any
questions?
The World Famous
SPIN
Win-A-Mallet
Trivia Quiz
 Rules
„ Short answers
„ Trade answer sheets and grade your
neighbor
„ Top 4 go to the tie-breaker rounds
„ Top 2 go to the final tie-breaker
Ready?
 Questions
1. Name the seven Level 2 Process areas in the
Staged Presentation of CMMI (1 pt per correct
answer)
2. The Battle of Little Round Top was fought in
what state? (1 point)
3. How does any organization get a Maturity
Level 3 Rating? (2 points)
4. What are the 3 phases of a SCAMPI
Appraisal?
5. Name the 3 hard prerequisites for being a
SCAMPI Appraisal Team Member
6. Name This Person
 Answers
1. REQM, PP, PMC, CM, SAM, MA, and PPQA (1 pt per
correct answer)
2. Pennsylvania (1)
3. Successfully pass a SCAMPI Class A Appraisal (1)
4. Plan and Prepare, Execute, Report Findings (1
pt/correct answer)
5. Intro to CMMI Course, 5 years of Engineering
Experience and 3 years of Management Experience
(2 pts/correct answer)
6. Larry Birkhead, Former boyfriend of Anna Nicole
Smith
„ +2 point for “Don’t know” or “Don’t care”
„ - 2 points for a correct answer

Perfect Score = 20
Tie Breakers – Round One
 Tie Breakers Round 1
1. What three things comprise the scope
of an appraisal
2. What is the sunset date for CMMI v1.1?
3. Where is Chapultepec Castle?
 Tie-Breaker Answers
1. First question:
„ The projects or instantiations to be
appraised
„ The process areas to be appraised and to
what level
„ Additional products requested by the
sponsor
2. 31 August 2007
3. Mexico (The Halls of Montezuma)
 Final Tie-Breaker
Visual
(First one to answer
wins the mallet)
Who is this person?
 Prince Harry
Third in Line to the Throne
QE-2’s Grandson