When is testing sufficient?

Test coverage criteria II

Agreeing presentation topics deadline is today

We will learn
Data flow coverage criteria

Comparison of criteria
Excercise

Last time
Control Flow criteria

Black box

Equivalence partitions Boundary values Statement coverage Branch coverage Path coverage Condition, multiple condition coverage

White box

White box structural coverage

Based on program formalisation (~program)

Graph ~ FSM (Finite State Machine) ~ EFSM (Extended FSM) ~ State machine diagram, activity diagram ~ ...

Assignment and comparison operators

Assignment

= Assignment (x = a + b)
== Equal (a == b) != Not Equal (a != b) >, <, >=, <=

Comparison

Data flow coverage - definitions

Definition defining a variable: x=a+2;

Use using the variable value

For computation a=x+2 (c-use, computation use) In the predicate x==0 (p-use, predicate use)
x=1 x=2 X!=1 x==1

x++ (st x=x+1)

Y=x+1

Definitions
x is defined on the edge ed, used on the edge

eu On the path ed, e1, ... , en, eu x is not redefined on sub-path e1, ..., en. If the previous holds, then the sub-path e1, ..., en is definition-clear path for x In other words, x is not redefined on this subpath, x holds the same value

All-defs (for x) coverage

If x is defined on the edge i, then all-defs

coverage is reached, when tests execute at least one path from i to some use of x In other words: all-defs criterion requires, that every definition of x has at least one use (predicate or computational)
x=1 k=x+2 x<3 k=k+2

x=2

k=x+1

y=0

y=y+1

All-defs: {1, 3, 4}, {1, 2, 3} (shortest)

2 6

All-uses (for x) coverage

Also reach coverage or du-pairs coverage

If variable x is defined on the edge i, then all-

uses criterion requires a test set, where tests excecute at least one path from every i to every edge, where the assigned value of x is used In other words: at least one path from each definition of x to every use of this definition

All-uses (for x) coverage (2)

All-uses: {1, 3, 4}, {1,3,4,5}, {1, 2, 3}, {1, 2, 3,

4}, {1, 2, 3, 4, 5}
x=1
1 3

k=x+2
4

x<3 k=k+2
5

x=2

k=x+1

y=0

y=y+1

All-p-uses/All-c-uses (for x) coverage

All-p-uses at least one path from defining x

to every use of this definition in a predicate All-c-uses at least one path from defining x to every use of this definition in computation

NB! a=x-1, not x=a+1

x=1 k=x+2 3 4 x<3 k=k+2 5

All-uses = all-p-uses + all-c-uses

1

x=2

k=x+1

y=0

y=y+1

All-p-uses: {1,3,4,5} , {1,2,3,4,5} All-c-uses: {1,3,4,}, {1,2,3}, {1,2,3,4}

All-du-paths (for x) coverage

If x is defined on the edge i, then all-du-paths

criterion requires tests to excecute all paths from every definition of x to every use of each definition In other words: all possible paths form defininig x to its use
x=1 1 3 k=x+2 4 x<3 k=k+2 5

x=2

k=x+1

y=0

y=y+1

All-du-paths: {1, 3, 4}, {1,3,4,5}, {1,3,6,4,5}, {1, 2, 3}, {1, 2, 3, 4}, {1, 2, 3, 4, 5}, {1, 2, 3, 6, 4, 5}

Strength of criteria
All-paths Stronger All-du-pairs All-uses All-c-uses All-defs All-p-uses

All edges (branch coverage)

Weaker

All nodes (statement coverage)

Related issues
Which tests (paths) should be excecuted to

achieve certain coverage? (test sequence generation problem) Which paths (program statements) can not be excecuted? (problem of finding infeasible paths dead code) How to find data, that allows to excecute tests for certain coverage? (test data generation problem)

Exercise
Find paths for x that

satisfy the following criteria:

x=read(x) y=read(y)

max=x

3
x>y

x<y

4
max = y

All-defs: All-uses: All-p-uses: All-c-uses: All-du-paths:

Exercise
Find paths for x that

satisfy the following criteria:

x=read(x) y=read(y)

max=x

3
x>y

x<y

4
max = y

All-defs: {1,2,3} All-uses:{1,2,3}, {1,2,3,4}, {1,2,3,5}, All-p-uses: {1,2,3,4}, {1,2,3,5} All-c-uses: {1,2,3} All-du-paths = all-uses + {1,2,3,4,5}