Testing Software

Testing Fundamentals

Why do you test? What happens if you dont test? What two things are you testing? Test coverage should find as many errors as possible in a system A test is considered successful if it can detect a previously undetected error

Testing categories

Black-box testing (Quality of Conformance)

A.k.a. Behavior Testing Input and output parameters are known Tests are conducted to determine if the functions produce the required output based on input

White-box testing (Quality of Design)

Internal operations are known and tests are conducted to ensure that the internal operations conform to specifications.

White-Box Testing

Unit testing done by the developer attempting to reach 100% unit test coverage Basis-path testing: Ensures that each statement in the software is tested Condition testing: Ensures that every possible condition is tested (a>b, a<b, a=b, a and b, a or b, a and not b, not a) Loop Testing and Error-handing paths Dynamic testing with test cases

Black-Box Testing

Integration, Validation, and System Testing Errors in performance Missing or incorrect functions Errors in user interface Functionality testing Effects on certain combinations of data (monkey-testing)

Integration Testing

Tests multiple components at once under a single test case Tests the interfaces between the components Tests connectivity between the components Integration tests can be white- or black-box
All unit tests and most integration tests are created by the developer. Examples and differences between these two types of tests?

Integration Testing Types

Top-down: Start at the top of the hierarchy and move down.

Depth-first: follow each branch to all of its leaves before moving on to the next branch Breadth-first: cover layers in the hierarchy

Bottom-up: integrate components at the lowest levels and build up

Regression testing: conducted to detect errors in code that was not changed, but may be affected by code that was changed. Smoke testing: designing sets of test cases that check for all possible errors

Validation Testing

Done by test team that likely includes end users to validate Quality of Conformance Black-Box testing method that utilizes a test plan to:

Execute an exhaustive number of test cases defined with specific input that should yield specific output Record the results Track defects

System Testing

Black-Box Testing Must be planned in advance with developed test cases and/or test scripts Security Testing: tests to help determine integrity = {(1 security) * (1 threat)} Performance Testing: user response time Stress Testing: strong test cases with executed continuously for a long time while monitoring CPU utilization

Testing Cycles

In an effort to catch and fix all possible defects, it is necessary to conduct tests scientifically in cycles No code is to be changed in the test environment until the end of a test cycle Each subsequent cycle must include tests that passed in the previous cycle

Test Case Development

The key to testing is writing good test cases Unit testing: create a series of test cases that will execute every line of code. Integration testing: create a series of test cases that will utilize every interface Validation testing: Think of every possible business scenario and write a test case System testing: think of every bad thing that could happen and write a test case

Test Case Specification

For user validation tests, the tester must understand the systems requirements. Analysts make great test case spec writer! Test case specs (a.k.a. test scripts) are documented and a large number of people are organized to implement them. End users make great test implementers! Implementation = validation + verification

Automated Testing Tools

Two types of testing tools

Quality of Design

Testing tools such as MS Web Stress Tool and LoadRunner that run test scripts which ensure that the system works as under certain conditions. Testing tools that take in requirements and generate test cases based on the requirements

Quality of Conformance

Other resources

Test automation frameworks x

http://safsdev.sourceforge.net/FRAMESDataDrivenTestAutomationFrameworks.htm