Escolar Documentos
Profissional Documentos
Cultura Documentos
Software Assurance: The planned and systematic set of activities that ensure that
software life cycle processes and products conform to requirements, standards, and
procedures [IEEE 610.12 IEEE Standard Glossary of Software Engineering
Terminology]. For NASA this includes the disciplines of Software Quality (functions of
Software Quality Engineering, Software Quality Assurance, Software Quality Control),
Software Safety, Software Reliability, Software Verification and Validation, and IV&V.
Software Quality: The discipline of software quality is a planned and systematic set of
activities to ensure quality is built into the software. It consists of software quality
assurance, software quality control, and software quality engineering. As an attribute,
software quality is (1) the degree to which a system, component, or process meets
specified requirements. (2) The degree to which a system, component, or process meets
customer or user needs or expectations [IEEE 610.12 IEEE Standard Glossary of
Software Engineering Terminology].
Software Quality Assurance: The function of software quality that assures that the
standards, processes, and procedures are appropriate for the project and are correctly
implemented.
Software Quality Control: The function of software quality that checks that the project
follows its standards, processes, and procedures, and that the project produces the
required internal and external (deliverable) products.
Software Quality Engineering: The function of software quality that assures that quality
is built into the software by performing analyses, trade studies, and investigations on the
requirements, design, code and verification processes and results to assure that reliability,
maintainability, and other quality factors are met.
Testing levels or phases should be applied against the application under test when the
previous phase of testing is deemed to be complete . or .complete enough.. Any defects
detected during any level or phase of testing need to be recorded and acted on
appropriately.
Design Review
"The objective of Design Reviews is to verify all documented design criteria before
development begins." The design deliverable or deliverables to be reviewed should be
complete within themselves. The environment of the review should be a professional
examination of the deliverable with the focus being the deliverable not the author (or
authors). The review must ensure each design deliverable for: completeness, correctness,
and fit (both within the business model, and system architecture).
Design reviews should be conducted by: system matter experts, testers, developers, and
system architects to ensure all aspects of the design are reviewed.
Unit Test
"The objective of unit test is to test every line of code in a component or module." The
unit of code to be tested can be tested independent of all other units. The environment of
the test should be isolated to the immediate development environment and have little, if
any, impact on other units being developed at the same time. The test data can be
fictitious and does not have to bear any relationship to .real world. business events. The
test data need only consist of what is required to ensure that the component and
component interfaces conform to the system architecture. The unit test must ensure each
component: compiles, executes, interfaces, and passes control from the unit under test to
the next component in the process according to the process model.
The developer in conjunction with a peer should conduct unit test to ensure the
component is stable enough to be released into the product stream.
"The objective of function test is to measure the quality of the functional (business)
components of the system." Tests verify that the system behaves correctly from the user /
business perspective and functions according to the requirements, models, storyboards, or
any other design paradigm used to specify the application. The function test must
determine if each component or business event: performs in accordance to the
specifications, responds correctly to all conditions that may be presented by incoming
events / data, moves data correctly from one business event to the next (including data
stores), and that business events are initiated in the order required to meet the business
objectives of the system.
System Test
"The objective of system test is to measure the effectiveness and efficiency of the system in
the "real-world" environment." System tests are based on business processes (workflows)
and performance criteria rather than processing conditions. The system test must
determine if the deployed system: satisfies the operational and technical performance
criteria, satisfies the business requirements of the System Owner / Users / Business
Analyst, integrates properly with operations (business processes, work procedures, user
guides), and that the business objectives for building the system were attained.
There are many aspects to System testing the most common are:
Security Testing: The tester designs test case scenarios that attempt to subvert or
bypass security.
Stress Testing: The tester attempts to stress or load an aspect of the system to the
point of failure; the goal being to determine weak points in the system architecture.
Performance Testing: The tester designs test case scenarios to determine if the
system meets the stated performance criteria (i.e. A Login request shall be responded
to in 1 second or less under a typical daily load of 1000 requests per minute.)
Install (Roll-out) Testing: The tester designs test case scenarios to determine if the
installation procedures lead to an invalid or incorrect installation.
Recovery Testing: The tester designs test case scenarios to determine if the system
meets the stated fail-over and recovery requirements.
"The objective of User Acceptance test is for the user community to measure the
effectiveness and efficiency of the system in the "real-world" environment.". User
Acceptance test is based on User Acceptance criteria, which can include aspects of
Function and System test. The User Acceptance test must determine if the deployed
system: meets the end Users expectations, supports all operational requirements (both
recorded and non-recorded), and fulfills the business objectives (both recorded and non-
recorded) for the system.
User Acceptance test should be conducted by the end users of the system and monitored
by an independent testing organization. The Users must ensure the system is stable and
meets the minimum quality criteria before proceeding to system deployment (roll-out).
Testing Roles
As in any organization or organized endeavor there are Roles that must be fulfilled within
any testing organization. The requirement for any given role depends on the size,
complexity, goals, and maturity of the testing organization. These are roles, so it is quite
possible that one person could fulfill many roles within the testing organization.
The Role of Test Lead / Manager is to effectively lead the testing team. To fulfill this role
the Lead must understand the discipline of testing and how to effectively implement a
testing process while fulfilling the traditional leadership roles of a manager. What does
this mean? The manager must manage and implement or maintain an effective testing
process.
Test Architect
The Role of the Test Architect is to formulate an integrated test architecture that supports
the testing process and leverages the available testing infrastructure. To fulfill this role
the Test Architect must have a clear understanding of the short-term and long-term goals
of the organization, the resources (both hard and soft) available to the organization, and a
clear vision on how to most effectively deploy these assets to form an integrated test
architecture.
The Role of the Test Designer / Tester is to: design and document test cases, execute
tests, record test results, document defects, and perform test coverage analysis. To fulfill
this role the designer must be able to apply the most appropriate testing techniques to test
The Role of the Test Automation Engineer to is to create automated test case scripts that
perform the tests as designed by the Test Designer. To fulfill this role the Test
Automation Engineer must develop and maintain an effective test automation
infrastructure using the tools and techniques available to the testing organization. The
Test Automation Engineer must work in concert with the Test Designer to ensure the
appropriate automation solution is being deployed.
The Role of the Test Methodologist is to provide the test organization with resources on
testing methodologies. To fulfill this role the Methodologist works with Quality
Assurance to facilitate continuous quality improvement within the testing methodology
and the testing organization as a whole. To this end the methodologist: evaluates the test
strategy, provides testing frameworks and templates, and ensures effective
implementation of the appropriate testing techniques.
Testing Techniques
Overtime the IT industry and the testing discipline have developed several techniques for
analyzing and testing applications.
Black-box Tests
Black-box tests are derived from an understanding of the purpose of the code; knowledge
on or about the actual internal program structure is not required when using this
approach. The risk involved with this type of approach is that .hidden. (functions
unknown to the tester) will not be tested and may not been even exercised.
White-box tests are derived from an intimate understanding of the purpose of the code
and the code itself; this allows the tester to test .hidden. (undocumented functionality)
within the body of the code. The challenge with any white-box testing is to find testers
that are comfortable with reading and understanding code.
Regression testing is not a testing technique or test phase; it is the reuse of existing tests
to test previously implemented functionality--it is included here only for clarification.
Equivalence Partitioning
Boundary-value Analysis
Error Guessing
Error Guessing involves making an itemized list of the errors expected to occur in a
particular area of the system and then designing a set of test cases to check for these
expected errors. Error Guessing is more testing art than testing science but can be very
effective given a tester familiar with the history of the system.
Output Forcing
Output Forcing involves making a set of test cases designed to produce a particular
output from the system. The focus here is on creating the desired output not on the input
that initiated the system response.
A number of different types of automated and manual testing tools are required to support
an automated testing framework.
Test Design Tools. Tools that are used to plan software testing activities. These tools are
used to create test artifacts that drive later testing activities.
Static Analysis Tools. Tools that analyze programs without machines executing them.
Inspections and walkthroughs are examples of static testing tools.
Dynamic Analysis Tools. Tools that involve executing the software in order to test it.
Load and Performance Tools. Tools that simulate different user load conditions for
automated stress and volume testing.
Non-GUI Test Drivers and Test Managers. Tools that automate test execution of
applications that do not allow tester interaction via a GUI.
Other Test Implementation Tools. Miscellaneous tools that assist test implementation.
We include the MS Office Suite of tools here.
Test Evaluation Tools. Tools that are used to evaluate the quality of a testing effort.