Escolar Documentos
Profissional Documentos
Cultura Documentos
PREFACE
P.1 PURPOSE
This procedure provides direction to Software Quality (SQ) personnel who are responsible for
developing and implementing Software Quality programs for all GSFC developed or acquired software
products and systems. Additional work instructions provide applicable procedures, step-by-step
instructions, and checklists for performing SQ process and product assessments throughout the software
development life cycle.
P.2 APPLICABILITY
This procedure applies to software created and acquired by or for NASA, including Government off-
the-shelf (GOTS) software, modified off-the-shelf (MOTS) software, and commercial off-the-
shelf (COTS) software when included in a NASA system. NASA systems include test beds,
ground support systems, flight systems, and software research projects that support or perform
our scientific missions.
P.3 AUTHORITY
This procedure adheres to the NASA Software Assurance Standard for planning and performing the
process and product quality assurance activities.
P.4 REFERENCES
c. NPG 7120.5, NASA Program and Project Management Processes and Requirements
h. 300-PG-7120.2.2, Mission Assurance Guidelines (MAG) for Tailoring to the Needs of GSFC
Projects
l. IEEE STD 730-2002, IEEE Standard for Software Quality Assurance Plans
P.5 CANCELLATION
P.6 SAFETY
The Systems Assurance Manager (SAM) will assure that Safety-critical systems that include software
are evaluated for software’s contribution to the safety of the system during the concept phase, and prior
to the start, or in the early phases, of the acquisition or planning for the given software. Unless the
evaluation proves that the software is not safety critical, the NASA Software Safety Standard 8719.13 is
to be followed.
P.7 TRAINING
SQ personnel shall have fundamental knowledge in the following areas/disciplines through prior
experience, training, or certification in methodologies, processes, and standards:
CHECK THE GSFC DIRECTIVES MANAGEMENT SYSTEM AT
http://gdms.gsfc.nasa.gov/gdms TO VERIFY THAT THIS IS THE CORRECT VERSION PRIOR TO USE.
GSFC Form 3-18 (10/01)
Procedures and Guidelines
P.8 RECORDS
SQ personnel shall assure that all assessments performed on their projects have the resultant artifact
information entered correctly and accurately in the SQERD. All records of assessment reports, findings
and observations will be maintained electronically. SQ personnel shall maintain SQ folders that contain
project work products or links to products (e.g., project Service Order, SQE Activity Schedule, project-
related documents). For more details on SQ records, their location, and data retention, reference the
Office of Systems Safety and Mission Assurance (OSSMA) Software Quality Assurance Data
Management Plan.
P.9 METRICS
The following standard metrics are the minimum planned metrics that will be collected, reported, and
maintained in the area of software quality assurance:
DEFINITIONS
a. Audit – An independent examination of work processes or set of work products to assess compliance
with specifications, standards, contractual agreements, or other criteria. [IEEE 610.12]
b. Configuration Management (CM) – A discipline applying technical and administrative direction and
surveillance to identify and document the functional and physical characteristics of a configuration
item, control changes to those characteristics, record and report change processing and
implementation status, and verify compliance with specified requirements. [IEEE 610.12]
c. Engineering Peer Review (EPR) – A focused, in-depth technical review that supports the evolving
design and development of a product subsystem or discipline area [GPG 8700.6]. The purpose of an
EPR is to add value and reduce risk through expert knowledge infusion, confirmation of approach,
and specific recommendations. An EPR provides a penetrating examination of design, analysis,
integration, test and operational details, drawings, processes and data.
e. Integrated Independent Review (IIR) – One of a series of system-level reviews conducted at critical
project/product milestones in accordance with GPG 8700.4. IIRs build upon the results of a robust
set of engineering peer reviews. IIR examples include System Concept review (SCR), Critical
Design Review (CDR), and Mission Operations Review (MOR).
i. Product Design Lead (PDL) – The manager or leader with overall responsibility for managing the
design activity, managing the technical and organizational interfaces identified during design
planning, and where required, forming and leading the Product Design Team (PDT). The term
refers to flight project managers, principal investigators, mission managers, instrument managers,
software managers, lead engineers, etc.
j. Safety Critical Software – Software that resides in a safety-critical system that is a potential hazard
cause or contributor, supports a hazard control or mitigation, controls safety-critical functions, or
detects and reports 1) fault trends that indicate a potential hazard and /or 2) failures which lead to a
hazardous condition. Analysis of the system should consider software that processes hazardous
commands or data, is required to put or keep the system in a safe state, provides information upon
which a safety decision is made, is part of a safety subsystem, or that can adversely affect system
safety by its failure or anomalous behavior.
k. Software Assurance (SA) – The planned and systematic set of activities that ensure that software life
cycle processes and products conform to requirements, standards, and procedures. [IEEE 610.12]
For NASA this includes the disciplines of Software Quality (SQ), Software Safety, Software
Reliability, Software Verification and Validation (V&V), and Independent Verification and
Validation (IV&V).
l. Software Quality (SQ) – The discipline of software quality is a planned and systematic set of
activities to ensure quality is built into the software. It consists of software quality assurance,
software quality control, and software quality engineering.
m. Software Quality Assurance (SQA) - The function of software quality that assures that the standards,
processes, and procedures are appropriate for the project and are correctly implemented.
o. Software Quality Engineering – The function of software quality that assures that quality is built
into the software by performing analyses, trade studies, and investigations on the requirements,
design, code, and verification processes and results to assure that reliability, maintainability, and
other quality factors are met.
p. Software Quality (SQ) Personnel - Personnel responsible for providing SQ support to the Systems
Assurance Manager. This includes software quality engineers, Defense Contract Management
Agency (DCMA) specialists, or support provided under the Supplier Assurance Contract (SAC).
Note: The Systems Assurance Manager may also perform the duties of a software quality person.
q. Software Requirements Traceability Matrix (SRTM) – A tool developed and maintained by software
engineering that traces software requirements back to system requirements and forward to design,
code, test procedures, and test results.
r. Systems Assurance Manager (SAM) – Code 300 personnel responsible for supporting the PDL in
the coordination of the definition and implementation of a Project Systems Safety and Mission
Assurance Program (SSMAP).
s. Validation – Confirmation by examination and provision of objective evidence that the particular
requirements for a specific intended use are fulfilled.
PROCEDURES
1. OVERVIEW
Software Quality (SQ) is defined as a planned and systematic approach for evaluating the quality of and
adherence to software product standards, processes, and procedures. It entails reviewing all software
development products and related processes to ensure that they meet a predefined set of requirements,
standards, and procedures. SQ is an integral part of the software development activities, beginning in
the formulation phase of the project and continuing through all phases of the project (i.e., concept,
requirements, design, implementation, verification, validation, and operation and maintenance).
Software Quality (SQ) is part of the larger software assurance (SA) program, which is ultimately part of
the overall mission assurance program and as such, shall be developed and implemented in conjunction
with the mission assurance program. The Systems Assurance Manager (SAM) shall ensure that SA is
established and fully implemented. This includes defining SA requirements and requesting adequate
resources for performing process and product assessments throughout the development life cycle.
Beginning in the formulation phase, the SAM shall develop the Mission Assurance Requirements
(MAR) for the project. The MAR shall identify all software assurance requirements, including any
required software assurance processes and products for the mission. In response to the MAR and the
developer’s Software Management Plan (SMP)/Product Plan, Software Quality personnel shall have the
primary responsibility with the concurrence of the SAM and Project Manager to develop a Software
Quality Assurance Plan (SQAP) that identifies and details the software assurance activities that will be
performed throughout the entire software life cycle. NOTE: For the purpose of this procedure,
Software Quality personnel develop an SQAP for the project/mission from the acquirer perspective.
The developer, also known as the provider, is responsible for developing an SQAP from the developer’s
perspective.
3. REPORTING
The Systems Assurance Manager (SAM), as well as software quality personnel, maintains a level of
independence from the project and the developer(s). Software quality personnel are assigned to a
project and provide Project Management with visibility into the processes and quality of the product. In
addition to the required Office of Systems Safety and Mission Assurance (OSSMA) weekly and
monthly reporting deliverables as stated in 303-PG-1060.1.1, SQ personnel shall document and report
software quality assessments, assessment results, issues, and lessons learned to the project and
appropriate process and product owners (i.e., stakeholders), as stated in the SQ Assessment/Reporting
Process work instruction.
SQ personnel shall develop the SQAP in relationship to the Software Management Plan (SMP) and/or
other developer deliverables. SQ personnel shall identify software quality activities (i.e., process and
product assessments) that shall be performed throughout the software development life cycle. In
addition, the SQ shall identify the interdependencies with other disciplines (e.g., configuration
management, risk management, test management, and lessons learned) and the SQ metrics that will be
collected, analyzed, and reported. SQ personnel are also required to develop and maintain an SQ
Activity Schedule that is aligned with the SQAP, as well as the project’s milestones. Schedule updates
shall be made on a monthly basis and made available, upon request.
Note: Sample software quality assurance plans, checklists, and reports can be found on GSFC’s
Software Assurance web site, http://sw-assurance.gsfc.nasa.gov, and a link is available from the
Office of Systems Safety and Mission Assurance (OSSMA) web site.
Key Terms Used: Participate: to be a contributing member with defined roles and responsibilities
Perform: to lead or conduct a prescribed activity
Assess: to evaluate processes/products and provide assessment results and recommendations
Review: to extract for informational purposes and use as potential input into SQ activity planning
The above table is a continuation of Table 4.0-1 SQ Activities across the Software Development Phases
The following overview provides a high level description for each of the traditional software
development phases with emphasis on SQ activities. The use of phases to describe SQ activities is
applicable to any software development methodology or life cycle model.
SQ personnel reviews and assesses each of the developer’s documents and the maturity of the
processes, plans, and procedures that have been established. Key software deliverables during this
phase include the developer’s final Software Management Plan (SMP), Software Requirements
Specification, Software Quality Assurance Plan (SQAP), and Software Configuration Management
Plan (CMP). These documents are typically separate documents, but may be included in the SMP for
smaller projects.
If IV&V is warranted, the SAM and/or SQ personnel establish a working relationship with IV&V that
fosters open communication and exchange of software assurance data.
During the design phases, SQ focuses on the developer’s progress and documentation of the software
design (e.g., attends design reviews/walkthroughs). If software development folders (or other similar
documentation) are used, they should be initiated early in the design phase and frequently assessed for
accuracy and compliance to required content. SQ also assures that all requirements have been allocated
to software components and that configuration management mechanisms are in place and effectively
controlling the requirements, software design, software documentation, and action items.
is performed to assure a stable and mature software system. The SQAP should also be updated to
reflect the required Operation and Maintenance activities.
SQ personnel shall conduct all product assessments in accordance with the Software Quality Assessment
and Reporting Process work instruction, 303-WI-7120.1.2. SQ assessments of software development
products are based on the products defined in the developer’s Software Management Plan (SMP). The
SMP deliverables, contractual deliverables, and the identified reviews form the basis for the SQ
assessment criteria.
Sections 4.2.1 through 4.2.5 summarize software product assessments typically performed during the
development and maintenance of software.
SQ personnel shall assess the supporting data packages from system level/software subsystem reviews
to assure that:
a. Review packages are being developed according to the specified criteria as required by the
Systems Review Office
b. Content is complete, accurate and of sufficient level of detail
c. Requests for Action are captured, reviewed, and tracked to closure
Note: System level/software subsystem reviews include, but are not limited to, Preliminary Design
Review (PDR), Critical Design Review (CDR), Operations Readiness Review (ORR), etc.
SQ personnel shall conduct product assessments on documents used to plan, develop, test, and maintain
software (e.g., Software Management Plan (SMP), Configuration Management Plan (CMP), Software
Requirements Specification, Risk Management Plan). Software documents are assessed to ensure that:
a. Technical documents are being developed in accordance with contractual documents and
specified standards
b. Content within documents are a complete and accurate representation of the specified standard
or required format and will satisfy software requirements
c. Level of detail is sufficient and consistent throughout
CHECK THE GSFC DIRECTIVES MANAGEMENT SYSTEM AT
http://gdms.gsfc.nasa.gov/gdms TO VERIFY THAT THIS IS THE CORRECT VERSION PRIOR TO USE.
GSFC Form 3-18 (10/01)
DIRECTIVE NO. Page 14 of 20
303-PG-7120.2.1C
EFFECTIVE DATE: January 23, 2006
EXPIRATION DATE: January 23, 2011
SQ personnel shall assess software development records (e.g., software development folders) to assure
that software work products developed during the life cycle phases are being maintained and that
changes are recorded promptly. Software development records may contain, but are not limited to,
software design data, code, test results, risks, etc.
SQ personnel shall assure that software configuration management (SCM) products are generated per
the CMP or the SMP. SCM products are assessed to assure product integrity throughout the life cycle.
The SQ assessment shall include, but is not limited to, configuration items, configuration baselines,
change control records, and configuration audit records.
SQ personnel shall conduct all process assessments in accordance with the Software Quality Assessment
and Reporting Process work instruction, 303-WI-7120.1.2. SQ assessments of the software
development processes are based on the processes defined in the developer’s Software Management
Plan (SMP). The SMP and its identified project procedures form the basis for the SQ assessment
criteria.
Sections 4.3.1 through 4.3.10 summarize software process evaluations typically performed during the
development and maintenance of software.
4.3.1 System/Subsystem Reviews
SQ personnel shall assess the process used to conduct reviews to determine if technical and systems
management experts are in attendance, correct information is presented, entry and exit criteria are met,
appropriate documents are identified for update, and that a system is in place to capture, review and
track action items to closure.
4.3.2 Peer Reviews
SQ personnel shall assess the process used to conduct peer reviews to determine if the appropriate
technical engineers are in attendance, correct information is presented, appropriate documents are
identified for update, and that a system is in place to capture, review and track action items to closure.
SQ personnel shall assess the processes for project planning, monitoring and control as described in the
Project Plan.
SQ personnel shall assure that the SCM processes address the following:
a. Configuration Identification
b. Configuration Control
c. Configuration Status Accounting
d. Configuration Audits and Reviews
e. Interface Control
f. Subcontractor/vendor control (if applicable)
4.3.6 Test Management
SQ personnel shall assure that the test management processes are being implemented per the SMP
and /or Test Plan(s). This includes all types of testing of software system components as described in
the test plan, specifically during integration testing (verification) and acceptance testing (validation).
SQ shall monitor testing efforts to assure that test schedules are adhered to and maintained to reflect an
accurate progression of the testing activities. Test monitoring may be scheduled or performed at
random. The purpose of the test management assessment is to assure that:
SQ personnel shall assess the project’s application of lessons learned to assure that project personnel
capture, review, identify, and implement process improvements throughout the development life cycle.
For example, the Program/Project Manager shall report the extent to which he or she applied lessons
learned at each major milestone. Reference NPG 7120.5 for specific requirements.
SQ personnel shall assess the project’s supplier agreement management process against the documented
process. At a minimum, SQ personnel shall assess that all supplier agreements have been established
and maintained. SQ personnel shall also assure that agreements are satisfied by both the project and the
supplier.
Acronyms
AR – Acceptance Review
CCB – Configuration Control Board
CDR – Critical Design Review
CM – Configuration Management
CMMI – Capability Maturity Model Integration
CMP – Configuration Management Plan
COTS – Commercial Off-the-Shelf
DCMA – Defense Contract Management Agency
DR – Discrepancy Report
EPR – Engineering Peer Review
FCA – Functional Configuration Audit
FOR – Flight Operations Review
FSW – Flight Software
GOTS – Government Off-the-Shelf
GSFC- Goddard Space Flight Center
I&T – Integration & Test
ICD – Interface Control Document
IIR – Integrated Independent Review
ISD – Information Systems Division
IV&V – Independent Verification & Validation
LL – Lessons Learned
MAG – Mission Assurance Guidelines
MAR – Mission Assurance Plan
MOR – Mission Operations Review
MOTS – Modified Off-the-Shelf
NRRS – NASA Record Retention Schedules
ORR – Operational Readiness Review
OSSMA – Office of Systems Safety & Mission Assurance
PCA – Physical Configuration Audit
PDL – Product Design Lead
PDR – Preliminary Design Review
PITS – Project Issue Tracking System
SA – Software Assurance
SAC – Supplier Assurance Contract
SAM – Systems Assurance Manager
SCR – System Concept Review
SDD – Software Design Description
SDF – Software Development Folder
SMP – Software Management Plan
SPR – Software Problem Report
SQ – Software Quality
SQA – Software Quality Assurance
CHECK THE GSFC DIRECTIVES MANAGEMENT SYSTEM AT
http://gdms.gsfc.nasa.gov/gdms TO VERIFY THAT THIS IS THE CORRECT VERSION PRIOR TO USE.
GSFC Form 3-18 (10/01)
DIRECTIVE NO. Page 20 of 20
303-PG-7120.2.1C
EFFECTIVE DATE: January 23, 2006
EXPIRATION DATE: January 23, 2011