303 PG 7120 - 2C

Procedures and Guidelines
DIRECTIVE NO. 303-PG-7120.2.1C APPROVED BY Signature:

EFFECTIVE DATE: January 23, 2006 NAME: Esmond B. Marvray
EXPIRATION DATE: January 23, 2011 TITLE: Chief, Assurance Management Office
Responsible Office: Code 303 / Assurance Management Office

Title: Procedure for Developing and Implementing Software Quality Programs
PREFACE
P.1 PURPOSE
This procedure provides direction to Software Quality (SQ) personnel who are responsible for
developing and implementing Software Quality programs for all GSFC developed or acquired software
products and systems. Additional work instructions provide applicable procedures, step-by-step
instructions, and checklists for performing SQ process and product assessments throughout the software
development life cycle.
P.2 APPLICABILITY
This procedure applies to software created and acquired by or for NASA, including Government off-
the-shelf (GOTS) software, modified off-the-shelf (MOTS) software, and commercial off-the-
shelf (COTS) software when included in a NASA system. NASA systems include test beds,
ground support systems, flight systems, and software research projects that support or perform
our scientific missions.
P.3 AUTHORITY
This procedure adheres to the NASA Software Assurance Standard for planning and performing the
process and product quality assurance activities.
NASA-STD-8739.8, NASA Software Assurance Standard

NASA-STD-8719.13 Software Safety Standard
NPD 2820.1, NASA Software Policies
GPG 7120.2, Project Management
P.4 REFERENCES
a. NPD 7120.4, NASA Program/Project Management
b. NPR 7150.2, NASA Software Engineering Requirements
c. NPG 7120.5, NASA Program and Project Management Processes and Requirements
CHECK THE GSFC DIRECTIVES MANAGEMENT SYSTEM AT

http://gdms.gsfc.nasa.gov/gdms TO VERIFY THAT THIS IS THE CORRECT VERSION PRIOR TO USE.
GSFC Form 3-18 (10/01)

d. GPG 7120.4, Risk Management
e. GPG 8700.4, Integrated Independent Reviews
f. GPG 8700.6, Engineering Peer Reviews
g. 300-PG-7120.2.1, Mission Assurance Guidelines (MAG) Implementation
h. 300-PG-7120.2.2, Mission Assurance Guidelines (MAG) for Tailoring to the Needs of GSFC
Projects
i. 303-PG-1060.1.1, Systems Assurance Manager Reporting
j. 303-WI-7120.1.2, Software Quality Assessment and Reporting Process
k. IEEE 610.12, IEEE Standard Glossary of Software Engineering Terminology
l. IEEE STD 730-2002, IEEE Standard for Software Quality Assurance Plans
m. SMAP-GB-A201, Software Assurance Guidebook
n. SMAP-GB-A301, Software Quality Assurance Audits Guidebook
o. GSFC Software Assurance Web Site: http://sw-assurance.gsfc.nasa.gov
p. OSSMA Software Quality Assurance Data Management Plan
P.5 CANCELLATION
303-PG-7120.2.1B, Procedure for Developing and Implementing Software Quality Programs
P.6 SAFETY
The Systems Assurance Manager (SAM) will assure that Safety-critical systems that include software
are evaluated for software’s contribution to the safety of the system during the concept phase, and prior
to the start, or in the early phases, of the acquisition or planning for the given software. Unless the
evaluation proves that the software is not safety critical, the NASA Software Safety Standard 8719.13 is
to be followed.
P.7 TRAINING
SQ personnel shall have fundamental knowledge in the following areas/disciplines through prior
experience, training, or certification in methodologies, processes, and standards:
GSFC Form 3-18 (10/01)

• Software Quality Assurance
• Audits and Reviews
• Risk Management
• Configuration Management
• Software Safety
• Contracts/Contractor Surveillance
• CMMI
• ISO 9001
• Project-specific Training
• ISD Software Engineering Discussions
It is the responsibility of the SQ personnel to acquire the necessary skills or knowledge in each of the
above disciplines. SQ personnel are responsible for completing an SQ Training log that specifies the
type of training and/or on-the-job experience that has been completed, along with the source of the
training, and the date of completion.
P.8 RECORDS
Record Title Record Custodian Retention

SQ Assessment Report Software Quality Engineering *NRRS 8/36.5C1 – Handle
Repository Database (SQERD) as permanent pending
retention approval
Completed Checklists Software Quality Engineering *NRRS 8/36.5C1 – Handle

Repository Database (SQERD) as permanent pending
retention approval
*NRRS – NASA Record Retention Schedules (NPG 1441.1)
SQ personnel shall assure that all assessments performed on their projects have the resultant artifact
information entered correctly and accurately in the SQERD. All records of assessment reports, findings
and observations will be maintained electronically. SQ personnel shall maintain SQ folders that contain
project work products or links to products (e.g., project Service Order, SQE Activity Schedule, project-
related documents). For more details on SQ records, their location, and data retention, reference the
Office of Systems Safety and Mission Assurance (OSSMA) Software Quality Assurance Data
Management Plan.
P.9 METRICS

GSFC Form 3-18 (10/01)

The Lead Software Quality Engineer (SQE) shall monitor, analyze, and control the process and product
software quality assurance activities in OSSMA based on metrics gathered across GSFC projects.
Potential process improvement or corrective action may be applied in areas such as resources assigned
to projects, process and/or product assessments planned, and/or training provided to SQ personnel.
The following standard metrics are the minimum planned metrics that will be collected, reported, and
maintained in the area of software quality assurance:
• SQ effort and funds expended (Planned vs. Actual)

• Number of SQ Assessments (Planned vs. Actual)
• Number of SQ Assessment Findings or noncompliances (Open vs. Closed)
• Number of SQ Assessment Observations
• Number of Risks identified as a result of an SQ Assessment
Additional Project metrics may also be collected, reported, and maintained, as required by the SAM.
Sample metrics include:
• Number of Peer Reviews (Planned vs. Actual)

• Number of Open vs. Closed Action Items from peer reviews
• Number of Open vs. Closed Software Problem Reports, with aging and trending over a specified
time frame
• Number of Open vs. Closed IV&V issues (via the Facility’s Project Issue Tracking System
(PITS))
DEFINITIONS
a. Audit – An independent examination of work processes or set of work products to assess compliance
with specifications, standards, contractual agreements, or other criteria. [IEEE 610.12]
b. Configuration Management (CM) – A discipline applying technical and administrative direction and
surveillance to identify and document the functional and physical characteristics of a configuration
item, control changes to those characteristics, record and report change processing and
implementation status, and verify compliance with specified requirements. [IEEE 610.12]
c. Engineering Peer Review (EPR) – A focused, in-depth technical review that supports the evolving
design and development of a product subsystem or discipline area [GPG 8700.6]. The purpose of an
EPR is to add value and reduce risk through expert knowledge infusion, confirmation of approach,
and specific recommendations. An EPR provides a penetrating examination of design, analysis,
integration, test and operational details, drawings, processes and data.

GSFC Form 3-18 (10/01)

d. Finding – Non-compliance to a requirement, procedure, standard, or specification.
e. Integrated Independent Review (IIR) – One of a series of system-level reviews conducted at critical
project/product milestones in accordance with GPG 8700.4. IIRs build upon the results of a robust
set of engineering peer reviews. IIR examples include System Concept review (SCR), Critical
Design Review (CDR), and Mission Operations Review (MOR).
f. Observation - A statement of fact (positive or negative) based on objective evidence.
g. Process Assessment – A systematic examination to determine whether a software process is being

performed in accordance with documented plans, procedures, etc.
h. Product Assessment – A systematic examination to determine whether a software product meets

specified requirements and standards.
i. Product Design Lead (PDL) – The manager or leader with overall responsibility for managing the
design activity, managing the technical and organizational interfaces identified during design
planning, and where required, forming and leading the Product Design Team (PDT). The term
refers to flight project managers, principal investigators, mission managers, instrument managers,
software managers, lead engineers, etc.
j. Safety Critical Software – Software that resides in a safety-critical system that is a potential hazard
cause or contributor, supports a hazard control or mitigation, controls safety-critical functions, or
detects and reports 1) fault trends that indicate a potential hazard and /or 2) failures which lead to a
hazardous condition. Analysis of the system should consider software that processes hazardous
commands or data, is required to put or keep the system in a safe state, provides information upon
which a safety decision is made, is part of a safety subsystem, or that can adversely affect system
safety by its failure or anomalous behavior.
k. Software Assurance (SA) – The planned and systematic set of activities that ensure that software life
cycle processes and products conform to requirements, standards, and procedures. [IEEE 610.12]
For NASA this includes the disciplines of Software Quality (SQ), Software Safety, Software
Reliability, Software Verification and Validation (V&V), and Independent Verification and
Validation (IV&V).
l. Software Quality (SQ) – The discipline of software quality is a planned and systematic set of
activities to ensure quality is built into the software. It consists of software quality assurance,
software quality control, and software quality engineering.
m. Software Quality Assurance (SQA) - The function of software quality that assures that the standards,
processes, and procedures are appropriate for the project and are correctly implemented.

GSFC Form 3-18 (10/01)

n. Software Quality Control – The function of software quality that checks that the project follows its
standards, processes, and procedures, and that the project produces the required internal and external
(deliverable) products.
o. Software Quality Engineering – The function of software quality that assures that quality is built
into the software by performing analyses, trade studies, and investigations on the requirements,
design, code, and verification processes and results to assure that reliability, maintainability, and
other quality factors are met.
p. Software Quality (SQ) Personnel - Personnel responsible for providing SQ support to the Systems
Assurance Manager. This includes software quality engineers, Defense Contract Management
Agency (DCMA) specialists, or support provided under the Supplier Assurance Contract (SAC).
Note: The Systems Assurance Manager may also perform the duties of a software quality person.
q. Software Requirements Traceability Matrix (SRTM) – A tool developed and maintained by software
engineering that traces software requirements back to system requirements and forward to design,
code, test procedures, and test results.
r. Systems Assurance Manager (SAM) – Code 300 personnel responsible for supporting the PDL in
the coordination of the definition and implementation of a Project Systems Safety and Mission
Assurance Program (SSMAP).
s. Validation – Confirmation by examination and provision of objective evidence that the particular
requirements for a specific intended use are fulfilled.
t. Verification - Confirmation by examination and provision of objective evidence that specified

requirements have been fulfilled.

GSFC Form 3-18 (10/01)
DIRECTIVE NO. Page 7 of 20
303-PG-7120.2.1C
EFFECTIVE DATE: January 23, 2006
EXPIRATION DATE: January 23, 2011
PROCEDURES
1. OVERVIEW
Software Quality (SQ) is defined as a planned and systematic approach for evaluating the quality of and
adherence to software product standards, processes, and procedures. It entails reviewing all software
development products and related processes to ensure that they meet a predefined set of requirements,
standards, and procedures. SQ is an integral part of the software development activities, beginning in
the formulation phase of the project and continuing through all phases of the project (i.e., concept,
requirements, design, implementation, verification, validation, and operation and maintenance).
2. ROLES and RESPONSIBILITIES
Software Quality (SQ) is part of the larger software assurance (SA) program, which is ultimately part of
the overall mission assurance program and as such, shall be developed and implemented in conjunction
with the mission assurance program. The Systems Assurance Manager (SAM) shall ensure that SA is
established and fully implemented. This includes defining SA requirements and requesting adequate
resources for performing process and product assessments throughout the development life cycle.
Beginning in the formulation phase, the SAM shall develop the Mission Assurance Requirements
(MAR) for the project. The MAR shall identify all software assurance requirements, including any
required software assurance processes and products for the mission. In response to the MAR and the
developer’s Software Management Plan (SMP)/Product Plan, Software Quality personnel shall have the
primary responsibility with the concurrence of the SAM and Project Manager to develop a Software
Quality Assurance Plan (SQAP) that identifies and details the software assurance activities that will be
performed throughout the entire software life cycle. NOTE: For the purpose of this procedure,
Software Quality personnel develop an SQAP for the project/mission from the acquirer perspective.
The developer, also known as the provider, is responsible for developing an SQAP from the developer’s
perspective.
3. REPORTING
The Systems Assurance Manager (SAM), as well as software quality personnel, maintains a level of
independence from the project and the developer(s). Software quality personnel are assigned to a
project and provide Project Management with visibility into the processes and quality of the product. In
addition to the required Office of Systems Safety and Mission Assurance (OSSMA) weekly and
monthly reporting deliverables as stated in 303-PG-1060.1.1, SQ personnel shall document and report
software quality assessments, assessment results, issues, and lessons learned to the project and
appropriate process and product owners (i.e., stakeholders), as stated in the SQ Assessment/Reporting
Process work instruction.

GSFC Form 3-18 (10/01)
303-PG-7120.2.1C
4. DEVELOPING THE SOFTWARE QUALITY ASSURANCE PLAN

The project Software Quality Assurance Plan (SQAP) is developed in the early stages of, and in parallel
with, the overall project planning effort. The SQAP provides a foundation for implementing an
effective Software Quality implementation program and defines the approach that will be used by the
SAM and SQ personnel to monitor and assess software development processes and products. The
SQAP shall be reviewed and approved by the SAM and Project Management. The format of the SQAP
shall follow the NASA Software Assurance Standard and IEEE STD 730-2002, the IEEE Standard for
Software Quality Assurance Plans.
SQ personnel shall develop the SQAP in relationship to the Software Management Plan (SMP) and/or
other developer deliverables. SQ personnel shall identify software quality activities (i.e., process and
product assessments) that shall be performed throughout the software development life cycle. In
addition, the SQ shall identify the interdependencies with other disciplines (e.g., configuration
management, risk management, test management, and lessons learned) and the SQ metrics that will be
collected, analyzed, and reported. SQ personnel are also required to develop and maintain an SQ
Activity Schedule that is aligned with the SQAP, as well as the project’s milestones. Schedule updates
shall be made on a monthly basis and made available, upon request.
Note: Sample software quality assurance plans, checklists, and reports can be found on GSFC’s
Software Assurance web site, http://sw-assurance.gsfc.nasa.gov, and a link is available from the
Office of Systems Safety and Mission Assurance (OSSMA) web site.
Table 4.0-1, SQ Activities across the Software Development Phases, is an “At-A-Glance”

reference for those SQ activities that are to be performed during each development phase. This
table should be used as a planning tool to determine the activities and resources that will be
needed to implement the SQAP for the project. Tailoring is acceptable commensurate with the
scope of the development effort, software size, complexity, criticality, and level of risk associated
with the software system(s). However, some level of SQ is required for all GSFC missions and
the presence of IV&V does not preclude the requirements for SQ.
Reference the NASA Software Assurance Standard NASA-STD-8739.8 for guidelines on

Software Quality tailoring.

GSFC Form 3-18 (10/01)
303-PG-7120.2.1C
Table 4.0-1 SQ Activities across the Software Development Phases
Software Development Phases

Integration & Test Acceptance Test Operations &
SQ Activities
Concept Requirements Design Implementation

(Verification) (Validation) Maintenance
Reviews  Assess System  Assess Software  Assess preliminary  Assess peer  Witness test execution  Witness test  Assess software
Concept Review Specification and critical design reviews (e.g., code  Assess Test execution enhancement build
(SCR) Review (SSR) reviews (i.e., PDR walkthroughs or Readiness Review  Assess AR or reviews
 Review Lessons  Capture/Review LL and CDR) inspections) (TRR) ORR  Capture/Review LL
Learned (LL)  Assess peer  Capture/Review LL  Capture/Review LL  Assess MOR
reviews and FOR
 Capture/Review LL  Capture/Review
LL
Software  Develop Software  Assess Software  Assess Software  Assess code for  Assess development  Assess  Assess updated
Deliverables Quality (SQ) task and Management Plan Design compliance to records development software
resource allocation (SMP) documentation standards  Assess Test Reports records documentation
forecast  Assess Software  Assess initial  Assess and test artifacts  Assess Test  Review SA Plan(s)
 Develop SW Requirements development development  Assess/verify SRTM Reports and test to address O&M
assurance Spec’s (SRSs) and records (e.g., records  Assess final User’s artifacts phase changes
requirements for ICD’s development  Assess final Test Guides  Assess/verify  Maintain SQ
SOW/MAR  Assess/Approve folders) Plans and  Maintain SQ Schedule SRTM Schedule
 Assess Quality SQAPs  Assess Test Plans procedures  Assess
Manual and Quality  Develop/Approve and procedures  Assess updates to ADP/VDDs
Management System Acquirer SQAP  Assess updates to SRTM  Assess/Approve
 Assess System  Assess S/W SRTM and  Assess preliminary Software
Requirements requirements allocation to S/W User’s Guides Maintenance
Specification traceability matrix components and  Maintain SQ Plan
 Develop SQ Activity (SRTM) design Schedule  Maintain SQ
Schedule  Maintain SQ  Maintain SQ Schedule
Schedule Schedule
Configuration  Assess  Assess CMP  Assess CMP  Assess CMP  Assess CMP  Assess CMP
Management Configuration compliance compliance compliance compliance compliance
Management Plan  Perform baseline  Perform baseline  Perform baseline  Assess FCA and  Review operational
(CMP) audits audits audits PCA artifacts baseline
 Assess CMP  Participate in CCBs  Participate in CCBs  Participate in CCBs  Participate in  Participate in CCBs
compliance CCBs
 Participate in CCBs
Software  Review and track  Review and track  Assess, analyze &  Assess, analyze &  Assess, analyze  Assess, analyze &
Problem action items action items trend SPRs trend SPRs & trend SPRs trend SPRs
Reporting  Assess Software  Review and track  Review and track  Review and track  Review and track
(SPR) and Problem Reporting action items action items action items action items
Corrective (SPR) system
Action (CA)
GSFC Form 3-18 (10/01)
303-PG-7120.2.1C
Integration & Test Acceptance Test Operations &

SQ Activities
Concept Requirements Design Implementation

(Verification) (Validation) Maintenance
Risk  Assess Project’s  Identify, review,  Identify, review, and  Identify, review, and  Identify, review, and  Identify, review,  Identify, review, and
Management Continuous Risk and assess assess software assess software assess software and assess assess software
Mgmt Approach software related related risks related risks related risks software related related risks
 Assess/Approve Risk risks risks
Mgmt Plan
IV&V  Review IV&V  Review IV&V  Review IV&V  Review IV&V monthly  Review IV&V  Review IV&V
Project Plans, and monthly reports monthly reports reports and TIMs monthly reports monthly reports
technical issues and TIMs and TIMs and TIMs and TIMs
(i.e., TIMs)
All acronyms are defined within the content of Section 4.0
Key Terms Used: Participate: to be a contributing member with defined roles and responsibilities
Perform: to lead or conduct a prescribed activity
Assess: to evaluate processes/products and provide assessment results and recommendations
Review: to extract for informational purposes and use as potential input into SQ activity planning
The above table is a continuation of Table 4.0-1 SQ Activities across the Software Development Phases

GSFC Form 3-18 (10/01)
303-PG-7120.2.1C
4.1 Overview of Software Development Activities
The following overview provides a high level description for each of the traditional software
development phases with emphasis on SQ activities. The use of phases to describe SQ activities is
applicable to any software development methodology or life cycle model.
4.1.1 System Concept Phase

During the system concept phase, the software concept is developed, the feasibility of the software
system is evaluated, and the acquisition strategy is developed. The SAM develops a service order and
secures resources to perform software quality. If the software system is to be acquired, a procurement
package, including software assurance requirements, is prepared and a contract is awarded. After
contract award, SQ personnel assess the developer’s quality management system to assure that standards
and procedures are in place as required by the mission assurance requirements. If a system is determined
to be safety-critical (e.g., through a preliminary hazard analysis), the use of software within that system
shall be analyzed. Software safety analyses are performed first to determine if the software is safety-
critical, and later to evaluate how well the software safety requirements are defined, designed, and
implemented in the system. This phase ends with a System Concept Review (SCR).
4.1.2 Software Requirements Phase
During the software requirements phase, the developer analyzes the system requirements to generate
the software requirements. Test planning is begun, with a method for verifying each requirement
identified and included in a preliminary test plan. Software requirements are mapped back to system
requirements and safety critical software requirements are uniquely identified and included in a
preliminary software requirements traceability matrix. Methods, standards, and procedures are detailed
and put in place. This phase ends with a requirements review, at which time the requirements are
agreed to between the acquirer and the developer and placed under configuration management control.
SQ personnel reviews and assesses each of the developer’s documents and the maturity of the
processes, plans, and procedures that have been established. Key software deliverables during this
phase include the developer’s final Software Management Plan (SMP), Software Requirements
Specification, Software Quality Assurance Plan (SQAP), and Software Configuration Management
Plan (CMP). These documents are typically separate documents, but may be included in the SMP for
smaller projects.
If IV&V is warranted, the SAM and/or SQ personnel establish a working relationship with IV&V that
fosters open communication and exchange of software assurance data.
4.1.3 Software Design Phase

The software system is typically designed in two phases. The architectural design (or preliminary
design) results in a high level design of the system, where requirements are fully allocated to software
components. During the detailed design phase, the architectural design is expanded to the lowest level
and the detailed design is baselined at the conclusion of the critical design review (CDR). Interface
control documents are completed and test plans revised and all design documents are placed under
configuration control.
GSFC Form 3-18 (10/01)
303-PG-7120.2.1C
During the design phases, SQ focuses on the developer’s progress and documentation of the software
design (e.g., attends design reviews/walkthroughs). If software development folders (or other similar
documentation) are used, they should be initiated early in the design phase and frequently assessed for
accuracy and compliance to required content. SQ also assures that all requirements have been allocated
to software components and that configuration management mechanisms are in place and effectively
controlling the requirements, software design, software documentation, and action items.
4.1.4 Software Implementation Phase

During the implementation phase, the software is coded and unit tested. This is a critical phase in the
software development life cycle whereby compliance to standards and procedures is imperative. SQ
provides management visibility into the development processes and quality of the product through
participation in code walkthroughs or inspections and assessments of configuration control processes,
software development records, and updates to the software requirements traceability matrix. SQ also
analyzes and trends software problem reports, monitors and tracks action items from system reviews
and peer reviews, and monitors risks. At the end of the phase, required products should be ready for
delivery, subject to modification during integration and test. Final test plans and procedures are
completed, along with preliminary user’s guides, and reviewed by SQ.
4.1.5 Software Integration and Test Phase

The objectives of the integration and test phase are to integrate the software units into a completed
subsystem or system, discover and correct any nonconformances or software problem reports, and
demonstrate that the software system meets its requirements. A test readiness review (TRR) concludes
this phase, at which time the developer provides evidence that the software system is ready for
acceptance testing. SQ assesses the development records, test reports, and test artifacts to substantiate
the readiness of the software for final delivery. In addition, SQ continues to monitor and assess the
developer’s configuration management system, to analyze and trend software problem reports, and to
review the accuracy of the requirements traceability matrix. Final user’s guides should be completed
prior to acceptance test and reviewed by SQ.
4.1.6 Software Acceptance Test Phase

During the acceptance test phase, formal acceptance procedures are executed to demonstrate that the
system meets customer requirements and that the right product was developed. SQ continues to focus
on test activities and documentation, configuration management, software and hardware baseline
management, software problem reports, and overall readiness of the system. This phase concludes with
an acceptance review (AR).
4.1.7 Operation and Maintenance Phase

During this phase, the software is baselined and used in its intended environment. Software corrections
and modifications are made to sustain/enhance its operational capabilities and to upgrade its capacity to
support its users. SQ continues to assess updated software documentation, changes to the operational
baseline, configuration management controls, and the software problem reporting system. The level of
SQ should be commensurate with the extent and criticality of changes being made to the software.
When long term sustaining engineering is required, the acquirer should ensure that a yearly assessment
GSFC Form 3-18 (10/01)
303-PG-7120.2.1C
is performed to assure a stable and mature software system. The SQAP should also be updated to
reflect the required Operation and Maintenance activities.
4.2 SQ Product Assessments
SQ personnel shall conduct all product assessments in accordance with the Software Quality Assessment
and Reporting Process work instruction, 303-WI-7120.1.2. SQ assessments of software development
products are based on the products defined in the developer’s Software Management Plan (SMP). The
SMP deliverables, contractual deliverables, and the identified reviews form the basis for the SQ
assessment criteria.
Sections 4.2.1 through 4.2.5 summarize software product assessments typically performed during the
development and maintenance of software.
4.2.1 System/Subsystem Reviews
SQ personnel shall assess the supporting data packages from system level/software subsystem reviews
to assure that:
a. Review packages are being developed according to the specified criteria as required by the
Systems Review Office
b. Content is complete, accurate and of sufficient level of detail
c. Requests for Action are captured, reviewed, and tracked to closure
Note: System level/software subsystem reviews include, but are not limited to, Preliminary Design
Review (PDR), Critical Design Review (CDR), Operations Readiness Review (ORR), etc.
4.2.2 Peer Reviews
SQ personnel shall assess Peer Reviews data packages to assure that:

a. Review packages are being developed according to the requirements in GPG 8700.6, Peer
Reviews
b. Content is complete, accurate, and of sufficient level of detail
c. Requests for Action are captured, reviewed, and tracked to closure
4.2.3 Document Reviews
SQ personnel shall conduct product assessments on documents used to plan, develop, test, and maintain
software (e.g., Software Management Plan (SMP), Configuration Management Plan (CMP), Software
Requirements Specification, Risk Management Plan). Software documents are assessed to ensure that:
a. Technical documents are being developed in accordance with contractual documents and
specified standards
b. Content within documents are a complete and accurate representation of the specified standard
or required format and will satisfy software requirements
c. Level of detail is sufficient and consistent throughout
GSFC Form 3-18 (10/01)
303-PG-7120.2.1C
d. Requirements traceability between formal technical documents exists

e. Proper document versioning appears on document
4.2.4 Software Development Records (e.g., folders /logs)
SQ personnel shall assess software development records (e.g., software development folders) to assure
that software work products developed during the life cycle phases are being maintained and that
changes are recorded promptly. Software development records may contain, but are not limited to,
software design data, code, test results, risks, etc.
4.2.5 Software Configuration Management
SQ personnel shall assure that software configuration management (SCM) products are generated per
the CMP or the SMP. SCM products are assessed to assure product integrity throughout the life cycle.
The SQ assessment shall include, but is not limited to, configuration items, configuration baselines,
change control records, and configuration audit records.

GSFC Form 3-18 (10/01)
303-PG-7120.2.1C
4.3 SQ Process Assessments
SQ personnel shall conduct all process assessments in accordance with the Software Quality Assessment
and Reporting Process work instruction, 303-WI-7120.1.2. SQ assessments of the software
development processes are based on the processes defined in the developer’s Software Management
Plan (SMP). The SMP and its identified project procedures form the basis for the SQ assessment
criteria.
Sections 4.3.1 through 4.3.10 summarize software process evaluations typically performed during the
development and maintenance of software.
4.3.1 System/Subsystem Reviews
SQ personnel shall assess the process used to conduct reviews to determine if technical and systems
management experts are in attendance, correct information is presented, entry and exit criteria are met,
appropriate documents are identified for update, and that a system is in place to capture, review and
track action items to closure.
4.3.2 Peer Reviews
SQ personnel shall assess the process used to conduct peer reviews to determine if the appropriate
technical engineers are in attendance, correct information is presented, appropriate documents are
identified for update, and that a system is in place to capture, review and track action items to closure.
4.3.3 Project Planning, Monitoring and Control
SQ personnel shall assess the processes for project planning, monitoring and control as described in the
Project Plan.
SQ personnel shall assure that:

a. Project planning parameter estimates are established and maintained
b. The project plan is established and maintained
c. Commitments to the project plan are established and maintained
d. Actual performance and progress of the project are monitored against the project plan
e. Corrective actions are managed to closure when project’s performance or results deviate
significantly from the plan
4.3.4 Requirements Management
SQ personnel shall assess the processes for requirements management as described in the developer’s
SMP or CMP.
SQ personnel shall assure that:

a. Processes are in place for managing and maintaining the requirements baseline throughout the
software development life cycle, including requirements identified as software safety critical
b. A Software Requirements Traceability Matrix has been developed and routinely updated to
reflect changes in status
GSFC Form 3-18 (10/01)
303-PG-7120.2.1C
c. Requirements changes are communicated across the project teams
4.3.5 Software Configuration Management (SCM)

SQ personnel shall assure that SCM processes are implemented per the CMP or the SMP. The SQ
assessment shall include SCM change control, software code management, software license
management, configuration audits, baseline management, etc.
SQ personnel shall assure that the SCM processes address the following:
a. Configuration Identification
b. Configuration Control
c. Configuration Status Accounting
d. Configuration Audits and Reviews
e. Interface Control
f. Subcontractor/vendor control (if applicable)
4.3.6 Test Management
SQ personnel shall assure that the test management processes are being implemented per the SMP
and /or Test Plan(s). This includes all types of testing of software system components as described in
the test plan, specifically during integration testing (verification) and acceptance testing (validation).
SQ shall monitor testing efforts to assure that test schedules are adhered to and maintained to reflect an
accurate progression of the testing activities. Test monitoring may be scheduled or performed at
random. The purpose of the test management assessment is to assure that:
a. Tests are planned and documented

b. Tests are conducted using approved test procedures and appropriate test tools
c. CM processes are implemented to maintain the integrity of the test environment
d. Assumptions, constraints, and results are accurately recorded
e. Anomalies are identified, documented, addressed, and tracked to closure
f. Requirements are satisfied
4.3.7 Software Problem Reporting and Corrective Action
SQ personnel shall assess the software change control process to assure it is being implemented per the
CMP or SMP. SQ personnel shall assess change control as it relates to those changes identified as
necessary using discrepancy reports (DRs). DRs identify changes that have been requested due to
deficiencies found during testing, installation, or operations. DRs are evaluated as to their criticality to
the system operation.
4.3.8 Risk Management
SQ personnel shall assess the project’s risk management process against the documented risk
management plan and GPG 7120.4.
4.3.9 Lessons Learned

GSFC Form 3-18 (10/01)
303-PG-7120.2.1C
SQ personnel shall assess the project’s application of lessons learned to assure that project personnel
capture, review, identify, and implement process improvements throughout the development life cycle.
For example, the Program/Project Manager shall report the extent to which he or she applied lessons
learned at each major milestone. Reference NPG 7120.5 for specific requirements.
4.3.10 Supplier Agreement Management (SAM)
SQ personnel shall assess the project’s supplier agreement management process against the documented
process. At a minimum, SQ personnel shall assess that all supplier agreements have been established
and maintained. SQ personnel shall also assure that agreements are satisfied by both the project and the
supplier.

GSFC Form 3-18 (10/01)
303-PG-7120.2.1C
CHANGE HISTORY LOG
Revision Effective Date Description of Changes
Baseline 11/17/2003 Initial Release
Modified title of PG, added configuration control number to the

NASA Software Policies NPD, added 3 new references to P.5,
modified recommended training in P.7, modified definitions for
“finding” and “observation”, added an SQ Reporting Form to P.8,
A 7/27/2004 deleted option to include the SAP in the Project’s Surveillance
Plan, provided updates to Table 4.0-1, added references to the
Software Quality Assessment Process 303-WI-7120.1.1 in
Sections 4.2 and 4.3, deleted the Software Safety Plan as a product
in Table 4.2-1, and updated naming convention to several software
quality work instructions in Tables 4.2-1 and 4.3-1.
B 01/25/2005 Removed non-requirements and revised to add clarity regarding
requirements
C 01/23/06 Added references to NPR 7150.2, the Software Safety Standard,
the SQERD Database, and the OSSMA Software Quality
Assurance Data Management Plan. Added new text to Section
P.6, Safety. Revised P.7, Training, to mirror the training
requirements in the updated SQAP template. Modified the
Metrics requirements in P.9. Added 2 additional process
assessment types to Section 4.3, specifically Project Planning,
Monitoring and Control, and Supplier Agreement Management.
Deleted the Process Data Flow Diagram.

GSFC Form 3-18 (10/01)
303-PG-7120.2.1C
Acronyms
AR – Acceptance Review
CCB – Configuration Control Board
CDR – Critical Design Review
CM – Configuration Management
CMMI – Capability Maturity Model Integration
CMP – Configuration Management Plan
COTS – Commercial Off-the-Shelf
DCMA – Defense Contract Management Agency
DR – Discrepancy Report
EPR – Engineering Peer Review
FCA – Functional Configuration Audit
FOR – Flight Operations Review
FSW – Flight Software
GOTS – Government Off-the-Shelf
GSFC- Goddard Space Flight Center
I&T – Integration & Test
ICD – Interface Control Document
IIR – Integrated Independent Review
ISD – Information Systems Division
IV&V – Independent Verification & Validation
LL – Lessons Learned
MAG – Mission Assurance Guidelines
MAR – Mission Assurance Plan
MOR – Mission Operations Review
MOTS – Modified Off-the-Shelf
NRRS – NASA Record Retention Schedules
ORR – Operational Readiness Review
OSSMA – Office of Systems Safety & Mission Assurance
PCA – Physical Configuration Audit
PDL – Product Design Lead
PDR – Preliminary Design Review
PITS – Project Issue Tracking System
SA – Software Assurance
SAC – Supplier Assurance Contract
SAM – Systems Assurance Manager
SCR – System Concept Review
SDD – Software Design Description
SDF – Software Development Folder
SMP – Software Management Plan
SPR – Software Problem Report
SQ – Software Quality
SQA – Software Quality Assurance
GSFC Form 3-18 (10/01)
303-PG-7120.2.1C
SQAP – Software Quality Assurance Plan

SQE – Software Quality Engineer
SQERD – Software Quality Engineering Repository Database
SRS – Software Requirements Specification
SRTM – Software Requirements Traceability Matrix
SSMAP – Systems Safety & Mission Assurance Program
TIM – Technical Issue Memorandum
TRR – Test Readiness Review
VDD – Verification Description Document
VOSSMA – Virtual Office of Systems Safety & Mission Assurance

GSFC Form 3-18 (10/01)

303 PG 7120 - 2C

Enviado por

Dados do documento

Descrição original:

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

303 PG 7120 - 2C

Enviado por

Direitos autorais:

Formatos disponíveis

Procedures and Guidelines

DIRECTIVE NO. 303-PG-7120.2.1C APPROVED BY Signature:

Responsible Office: Code 303 / Assurance Management Office

NASA-STD-8739.8, NASA Software Assurance Standard

a. NPD 7120.4, NASA Program/Project Management

b. NPR 7150.2, NASA Software Engineering Requirements

CHECK THE GSFC DIRECTIVES MANAGEMENT SYSTEM AT

DIRECTIVE NO. 303-PG-7120.2.1C APPROVED BY Signature:

e. GPG 8700.4, Integrated Independent Reviews

f. GPG 8700.6, Engineering Peer Reviews

g. 300-PG-7120.2.1, Mission Assurance Guidelines (MAG) Implementation

i. 303-PG-1060.1.1, Systems Assurance Manager Reporting

j. 303-WI-7120.1.2, Software Quality Assessment and Reporting Process

k. IEEE 610.12, IEEE Standard Glossary of Software Engineering Terminology

m. SMAP-GB-A201, Software Assurance Guidebook

n. SMAP-GB-A301, Software Quality Assurance Audits Guidebook

o. GSFC Software Assurance Web Site: http://sw-assurance.gsfc.nasa.gov

p. OSSMA Software Quality Assurance Data Management Plan

303-PG-7120.2.1B, Procedure for Developing and Implementing Software Quality Programs

DIRECTIVE NO. 303-PG-7120.2.1C APPROVED BY Signature:

Record Title Record Custodian Retention

Completed Checklists Software Quality Engineering *NRRS 8/36.5C1 – Handle

CHECK THE GSFC DIRECTIVES MANAGEMENT SYSTEM AT

DIRECTIVE NO. 303-PG-7120.2.1C APPROVED BY Signature:

• SQ effort and funds expended (Planned vs. Actual)

• Number of Peer Reviews (Planned vs. Actual)

CHECK THE GSFC DIRECTIVES MANAGEMENT SYSTEM AT

DIRECTIVE NO. 303-PG-7120.2.1C APPROVED BY Signature:

f. Observation - A statement of fact (positive or negative) based on objective evidence.

g. Process Assessment – A systematic examination to determine whether a software process is being

h. Product Assessment – A systematic examination to determine whether a software product meets

CHECK THE GSFC DIRECTIVES MANAGEMENT SYSTEM AT

DIRECTIVE NO. 303-PG-7120.2.1C APPROVED BY Signature:

t. Verification - Confirmation by examination and provision of objective evidence that specified

CHECK THE GSFC DIRECTIVES MANAGEMENT SYSTEM AT

2. ROLES and RESPONSIBILITIES

CHECK THE GSFC DIRECTIVES MANAGEMENT SYSTEM AT

4. DEVELOPING THE SOFTWARE QUALITY ASSURANCE PLAN

Table 4.0-1, SQ Activities across the Software Development Phases, is an “At-A-Glance”

Reference the NASA Software Assurance Standard NASA-STD-8739.8 for guidelines on

CHECK THE GSFC DIRECTIVES MANAGEMENT SYSTEM AT

Table 4.0-1 SQ Activities across the Software Development Phases

Software Development Phases

Concept Requirements Design Implementation

Integration & Test Acceptance Test Operations &

Concept Requirements Design Implementation

All acronyms are defined within the content of Section 4.0

CHECK THE GSFC DIRECTIVES MANAGEMENT SYSTEM AT

4.1 Overview of Software Development Activities

4.1.1 System Concept Phase

4.1.3 Software Design Phase

4.1.4 Software Implementation Phase

4.1.5 Software Integration and Test Phase

4.1.6 Software Acceptance Test Phase

4.1.7 Operation and Maintenance Phase

4.2 SQ Product Assessments

4.2.1 System/Subsystem Reviews

4.2.2 Peer Reviews

SQ personnel shall assess Peer Reviews data packages to assure that:

4.2.3 Document Reviews

d. Requirements traceability between formal technical documents exists

4.2.4 Software Development Records (e.g., folders /logs)