Software Model Checking: Willem Visser

2002
Software Model Checking
Willem Visser
Research Institute for Advanced Computer Science

NASA Ames Research Center
wvisser@email.arc.nasa.gov
Overview
2002
• Introduction to Model Checking
– Hardware and Software Model Checking
• Program Model Checking
– Major Trends
• Abstraction
• Improved model checking technology
– A Brief History
• SPIN
• Hand-translations
• State-less model checking
• Semi-automated translations
• Fully automated translations
– Current Trends
• Custom-made model checkers for programs
• SLAM
• JPF
• Summary
• NASA Case Studies - Remote Agent, DEOS and Mars Rover
• Future of Software Model Checking
24 September 2002 © Willem Visser 2002 2

Model Checking
2002 The Intuition
• Calculate whether a system satisfies a certain
behavioral property:
– Is the system deadlock free?
– Whenever a packet is sent will it eventually be received?
• So it is like testing? No, major difference:
– Look at all possible behaviors of a system
• Automatic, if the system is finite-state
– Potential for being a push-button technology
– Almost no expert knowledge required
• How do we describe the system?
• How do we express the properties?
Labeled State Graph
2002 Kripke Structure
Each state represents all

variable values and
x ~p location counters
k p
Each transition
y ~p
represents an execution
step in the system ~p The labels represent
hh
predicates in each state
e.g. (x = 5)
z ~p
K = ({p,~p},{x,y,z,k,h},R,{x},L)

Property Specifications
2002
• Temporal Logic
– Express properties of event orderings in time
– e.g. “Always” when a packet is sent it will “Eventually” be received
• Linear Time • Branching Time

– Every moment has a unique successor – Every moment has several successors
– Infinite sequences (words) – Infinite tree
– Linear Time Temporal Logic (LTL) – Computation Tree Logic (CTL)

Safety and Liveness
2002
• Safety properties
– Invariants, deadlocks, reachability, etc.
– Can be checked on finite traces
– “something bad never happens”
• Liveness Properties
– Fairness, response, etc.
– Infinite traces
– “something good will eventually happen”

Mutual Exclusion Example
2002
• Two process mutual exclusion with shared semaphore
• Each process has three states
• Non-critical (N)
• Trying (T)
• Critical (C)
• Semaphore can be available (S0) or taken (S1)
• Initially both processes are in the Non-critical state and
the semaphore is available --- N1 N2 S0
N1  T1 N2  T2
T1  S0  C1  S1 || T2  S0  C2  S1
C2  N2  S0
C1
24 September 2002  N1  S0 © Willem Visser 2002 7
2002
N1N2S0
T1N2S0 N1T2S0
C1N2S1 T1T2S0 N1C2S1
C1T2S1 T1C2S1
K ╞ AG EF (N1  N2  S0)
No matter where you are there is
always a way to get to the initial state
2002
N1N2S0
T1N2S0 N1T2S0
C1T2S1 T1C2S1
K ╞ AG EF (N1  N2  S0)

2002
N1N2S0
T1N2S0 N1T2S0
C1T2S1 T1C2S1
K ╞ AG EF (N1  N2  S0)

2002
N1N2S0
T1N2S0 N1T2S0
C1T2S1 T1C2S1
K ╞ AG EF (N1  N2  S0)

2002
N1N2S0
T1N2S0 N1T2S0
C1T2S1 T1C2S1
K ╞ AG EF (N1  N2  S0)

2002
N1N2S0
T1N2S0 N1T2S0
C1T2S1 T1C2S1
K ╞ AG EF (N1  N2  S0)

Model Checking
2002
• Given a Kripke structure M = (S,R,L) that represents a
finite-state concurrent system and a temporal logic
formula f expressing some desired specification, find the
set of states in S that satisfy f:
{ s  S | M,s ╞ f }
• Normally, some states of the concurrent system are

designated as initial states. The system satisfies the
specification provided all the initial states are in the set.
We often write: M ╞ f

Explicit vs. Symbolic
2002 Model Checking
• Explicit State • Symbolic
– states are enumerated on-the-fly – Sets of states are manipulated at a time
– Forwards analysis – Typically a backwards analysis
– Stores visited states in a hashtable – Transition relation encoded by (some
variant of) Binary Decision Diagrams
(BDDs) or as a satisfiability problem
• Characteristics • Characteristics
– Memory intensive – Can handle very large state spaces
– Good for finding concurrency errors – Not as good for asynchronous systems
– Short execution paths are better, but – Cannot deal well with long execution
long execution paths can also be traces
handled
– Can handle dynamic creation of – Works best with a static transition
objects/threads relation, hence doesn’t deal well with
dynamic creation of objects/threads
– Mostly used in software – Mostly used in hardware

Overview
2002
– Hardware Model Checking
– Software Model Checking
• Case Studies

Hardware Model Checking
2002
• BDD-based model checking was the enabling
technology
– Hardware is typically synchronous and regular, hence the
transition relation can be encoded efficiently
– Execution paths are typically very short
• The Intel Pentium bug, was the “disaster” that got
model checking on the map in the hardware industry
– What is it going to take in the software world?
• Intel, IBM, Motorola, etc. now employ hundreds of
model checking experts
2002
• Although the early papers on model checking
focused on software, not much happened until 1997
• Until 1997 most work was on software designs
– Since catching bugs early is more cost-effective
– Problem is that everybody use a different design
notation, and although bugs were found the field never
really moved beyond some compelling case-studies
– Reality is that people write code first, rather than design
• The field took off when the seemingly harder
problem of analyzing actual source code was first
attempted
Program Model Checking
2002
• Why is program analysis with a model
checker so much more interesting?
– Designs are hard to come by, but buggy
programs are everywhere!
– Testing is inadequate for complex software
(concurrency, pointers, objects, etc.)
– Static program analysis was already an
established field, mostly in compiler
optimization, but also in verification.

The Trends in
2002 Program Model Checking
Most model checkers cannot deal with the features of

modern programming languages
• Bringing programs to model checking

– Abstraction (including translation)
• Bringing model checking to programs
– Improve model checking to directly deal with
programs as input

Overview
2002
– Major Trends
• Abstraction
• Improved model checking technology
– A Brief History
– Current Trends
• Case Studies
2002 Enabling Technology
Abstraction
Program Model Checker
void add(Object o) {
Input
buffer[head] = o;
head = (head+1)%size;
}
Object take() {
…
tail=(tail+1)%size;
return buffer[tail];
}
Infinite state Finite state

Abstraction
2002
• Model checkers don’t take real “programs” as input
• Model checkers typically work on finite state systems
• Abstraction therefore solves two problems
– It allows model checkers to analyze a notation they couldn’t deal
with before, and,
– Cuts the state space size to something manageable
• Abstraction comes in three flavors
– Over-approximations, i.e. more behaviors are added to the
abstracted system than are present in the original
– Under-approximations, i.e. less behaviors are present in the
abstracted system than are present in the original
– Precise abstractions, i.e. the same behaviors are present in the
abstracted and original program

Under-Approximation
2002 “Meat-Axe” Abstraction
• Remove parts of the program deemed “irrelevant” to the
property being checked
– Limit input values to 0..10 rather than all integer values
– Queue size 3 instead of unbounded, etc.
• The abstraction of choice in the early days of program
model checking – used during the translation of code to a
model checker’s input language
• Typically manual, with no guarantee that the right
behaviors are removed.
• Precise abstraction, w.r.t. the property being checked, may
be obtained if the behaviors being removed are indeed not
influencing the property
– Program slicing is an example of an automated
under-approximation that will lead to a precise abstraction w.r.t. the
property being checked
Over-Approximations
2002 Abstract Interpretation
• Maps sets of states in the concrete program to one state in the abstract
program
– Reduces the number of states, but increases the number of possible
transitions, and hence the number of behaviors
– Can in rare cases lead to a precise abstraction
• Type-based abstractions
– Replace int by Signs abstraction {neg,pos,zero}
• Predicate abstraction
– Replace predicates in the program by boolean variables, and replace each
instruction that modifies the predicate with a corresponding instruction
that modifies the boolean.
• Automated (conservative) abstraction
• Eliminating spurious errors is the big problem
– Abstract program has more behaviors, therefore when an error is found in
the abstract program, is that also an error in the original program?
– Most research focuses on this problem, and its counter-part the
elimination of spurious errors, often called abstraction refinement

Bringing
2002 Model Checking to Programs
• Allow model checkers to take modern
programming languages as input, or, notations that
are of similar expressive power
• Major hurdle is how to encode the state of the
system efficiently
• Alternatively state-less model checking
– No state encoding or storing
• Almost exclusively explicit-state model checking
• Abstraction can still be used as well
– Source to source abstractions

Overview
2002
– Major Trends
– A Brief History
• SPIN
– Partial-order reductions
– VeriSoft
– Current Trends
• Case Studies

The Early Years
2002
• Hand-translation with ad-hoc abstractions
– 1980 through mid 1990s
• Semi-automated, table-driven translations
– 1998
• Automated translations still with
ad-hoc abstractions
– 1997-1999
• State-less model checking for C
– VeriSoft 1997
Overview
2002
– Major Trends
– A Brief History
• SPIN
– VeriSoft
– Current Trends
• Case Studies

SPIN Model Checker
2002
• Kripke structures are described as “programs” in the
PROMELA language
– Kripke structure is generated on-the-fly during model checking
• Automata based model checker
– Translates LTL formula to Büchi automaton
• By far the most popular model checker
– 10th SPIN Workshop to be held with ICSE – May 2003
• Relevant theoretical papers can be found here
– http://netlib.bell-labs.com/netlib/spin/whatispin.html
• Ideal for software model checking due to expressiveness of
the PROMELA language
– Close to a real programming language
• Gerard Holzmann won the ACM software award for SPIN

Hand-Translation
2002
abstraction
Verification model
translation
Program
• Hand translation of program to model checker’s input notation

• “Meat-axe” approach to abstraction
• Labor intensive and error-prone

Hand-Translation
2002 Examples
• Remote Agent – Havelund,Penix,Lowry 1997
– http://ase.arc.nasa.gov/havelund
– Translation from Lisp to Promela (most effort)
– Heavy abstraction
– 3 man months
• DEOS – Penix, Visser, et al. 1998/1999
– http://ase.arc.nasa.gov/visser
– C++ to Promela (most effort in environment generation)
– Limited abstraction - programmers produced sliced system
– 3 man months
Overview
2002
– Major Trends
– A Brief History
• SPIN
– VeriSoft
– Current Trends
• Case Studies

State-less Model Checking
2002
• No state storing, hence no state matching
• Must limit search-depth to ensure termination
• Enabling technology is partial-order reduction
– Avoid revisiting some states
• Annotate code to allow verifier to detect “important” statements
– Communications and nondeterministic choice
– Model checker will try all interleavings of the these statements, but will
consider all other statements “atomic”, i.e. the last statement in any group
will be an important one.
• Examples include
– VeriSoft
• http://www1.bell-labs.com/project/verisoft/
– Rivet
• http://sdg.lcs.mit.edu/rivet.html
– See also work by Scott Stoller
• http://www.cs.sunysb.edu/~stoller/

Partial-Order Reductions
2002
• Reduce the number of interleavings of
independent concurrent transitions
• x := 1 || y := 1 where initially x = y = 0
00 00 00
x := 1 y := 1 x := 1 y := 1 x := 1
10 01 10 01 10
y := 1 x := 1 y := 1 y := 1
11 11 11
No Reductions Transitions Reduced States Reduced

Partial Order Reductions
2002 Basic Ideas
• Independence
– Independent transitions cannot disable nor enable each other
– Enabled independent transitions are commutative
• Partial-order reductions only apply during the
on-the-fly construction of the Kripke structure
• Based on a selective search principle
– Compute a subset of enabled transitions in a state to execute
• Sleep sets (Reduce transitions)
• Persistent sets (Reduce states)

Persistent Set Reductions
2002
• A subset of the enabled transitions is called persistent,
when any transition can be executed from outside the
set and it will not interact or affect the transitions inside
the set
– Use the static structure of the system to determine what goes
into the persistent set
– Note, all enabled transitions are trivially persistent
• Only execute transitions in the persistent set
• Persistent set algorithm is used within SPIN
• See papers by Godefroid and Peled
VeriSoft
2002
• The first model checker that could T1 || T2
handle programs directly T1 T2
– C programs running on Unix
• Relies on partial-order reductions to T2 T1
limit the number of times a state is

revisited
– Persistent sets T1 || T2 T1 || T2
• Reduce states visited T1
T1 T2
– Sleep sets
• Reduce transitions executed
T2 T2 T1
• Paths must be replayed from the
Persistent Sleep
initial state to try new branches Sets Sets
– No check-pointing

State-less Example
2002
T1 T3
T2 T4
T3 T1
T3 T2 T4 T1
T4 T2
T1 ; T2 || T3 ; T4
• DFS with state storage – executes 12 transitions

• DFS state-less – executes 24 transitions

State-less Example
2002
T1 T3
T2 T4
T3 T1
T3 T2 T4 T1
T4 T2
T1 ; T2 || T3 ; T4

• DFS state-less with partial-order reduction
• Sleep sets - executes 8 transitions (all states)

State-less Example
2002
T1 T3
T2 T4
T3 T1
T3 T2 T4 T1
T4 T2
T1 ; T2 || T3 ; T4

• DFS state-less with partial-order reduction
• Sleep sets - executes 8 transitions (all states)
• Persistent sets – executes 4 transitions (only 5 states)
Overview
2002
– Major Trends
– A Brief History
• SPIN
– VeriSoft
– Current Trends
• Case Studies

Semi-Automatic Translation
2002
• Table-driven translation and abstraction
– Feaver system by Gerard Holzmann
– User specifies code fragments in C and how to translate
them to Promela (SPIN)
– Translation is then automatic
– Found 75 errors in Lucent’s PathStar system
– http://cm.bell-labs.com/cm/cs/who/gerard/
• Advantages
– Can be reused when program changes
– Works well for programs with long development and
only local changes

Fully Automatic Translation
2002
• Advantage
– No human intervention required
• Disadvantage
– Limited by capabilities of target system
• Examples
– Java PathFinder 1- http://ase.arc.nasa.gov/havelund/jpf.html
• Translates from Java to Promela (Spin)
– JCAT - http://www.dai-arc.polito.it/dai-arc/auto/tools/tool6.shtml
• Translates from Java to Promela (or dSpin)
– Bandera - http://www.cis.ksu.edu/santos/bandera/
• Translates from Java bytecode to Promela, SMV or dSpin

Overview
2002
– Major Trends
– A Brief History
– Current Trends
• Abstraction
• SLAM
• JPF
• Summary
• Examples of other software analyses
• Case Studies
2002 Current Trends
Program
Abstract Program Correct
buffer[head] = o;
head = (head+1)%size; T1 > T2
T3 > T4
Custom
}
T5 > T6 Model Checker
Object take() {
…
Abstraction …
tail=(tail+1)%size;
return buffer[tail]; Error-trace
}
Abstraction refinement
Abstraction
• Custom-made model checkers for programming languages with

automatic abstraction at the source code level
• Automatic abstraction & translation based transformation to new
“abstract” formalism for model checker
• Abstraction refinement mostly automated
Custom-made Model Checkers
2002
• Translation based
– dSpin
• Spin extended with dynamic constructs
• Essentially a C model checker
• Source-2-source abstractions can be supported
• http://www.dai-arc.polito.it/dai-arc/auto/tools/tool7.shtml
– SPIN Version 4 (not released yet)
• PROMELA language augmented with C code
• Table-driven abstractions
– Bandera
• Translated Bandera Intermediate Language (BIR) to a number of
back-end model checkers, but, a new BIR custom-made model
checker is under development
• Supports source-2-source abstractions as well as property-specific
slicing
• http://www.cis.ksu.edu/santos/bandera/

Custom-made Model Checkers
2002
• Abstraction based
– SLAM
• C programs are abstracted via predicate abstraction to boolean
programs for model checking
• http://research.microsoft.com/slam/
– BLAST
• Similar basic idea to SLAM, but using lazy abstraction, i.e.
during abstraction refinement don’t abstract the whole program
only certain parts
• http://www-cad.eecs.berkeley.edu/~tah/blast/
– 3-Valued Model Checker (3VMC) extension of TVLA
for Java programs
• http://www.cs.tau.ac.il/~yahave/3vmc.htm
• http://www.math.tau.ac.il/~rumster/TVLA/

Overview
2002
– Major Trends
– A Brief History
– Current Trends
• Abstraction
• SLAM
• JPF
• Summary
• Case Studies
Abstraction
2002
• Data type based abstractions
– Abstract Interpretation (Patrick Cousot)
– E.g. replace integer variable with odd-even range or
Signs abstraction {negative,zero,positive}
– Replace all operations on the concrete variable with
corresponding abstract operations
• add(pos,pos) = pos
• subtract(pos,pos) = negative | zero | pos
• eq(pos,pos) = true | false
• Predicate Abstraction (Graf, Saïdi see also Uribe)
– Create abstract state-space w.r.t. set of predicates
defined in concrete system
Data Type Abstraction
2002
Collapses data domains via abstract interpretation:

Code Data domains
int
int x = 0;
if (x == 0)
x = x + 1;
(n<0) : NEG
(n==0): ZERO
(n>0) : POS
Signs
Signs x = ZERO;
if (Signs.eq(x,ZERO)) NEG ZERO POS
x = Signs.add(x,POS);
Predicate Abstraction
2002
T F
Abstract EQ = T EQ := F EQ = F
int  int bool EQ (x = y) EQ (x = y)
Concrete xy x y

y++
x=0 x=0
y=0 y=1
• Mapping of a concrete system to an abstract system, whose states

correspond to truth values of a set of predicate
• Create abstract state-graph during model checking, or,
• Create an abstract transition system before model checking
Example
2002 Predicate Abstraction
Predicate: B  (x = y)
Concrete Statement Abstract Statement
y := y + 1
Step 1: Calculate pre-images
{x = y + 1} y := y + 1 {x = y}
{x  y + 1} y := y + 1 {x  y}
Step 2: Rewrite in terms of predicates
Step 2a: Use Decision Procedures
{x = y + 1} y := y + 1 {B}
x=yx=y+1 x=yxy+1 {B} y := y + 1 {~B}
xyx=y+1 xyxy+1
Step 3: Abstract Code
IF B THEN B := false
ELSE B := true | false
Example of Infeasible Counter-example
2002
{NEG,ZERO,POS}
[1] if (-2 + 3 > 0) [1] if(Signs.gt(Signs.add(NEG,POS),ZERO))
then then
[2] assert(true); Signs:
[2] assert(true);
else else
n < 0 -> neg
[3] assert(false); [3] assert(false);
0 -> zero
n > 0 -> pos
In
fe
as
bli
[1]: [1]:
e
co
un
te
r-
ex
am
X
p
[2]:
le
[2]: [3]:

Overview
2002
– Major Trends
– A Brief History
– Current Trends
• Abstraction
• SLAM
– Abstraction Refinement
• JPF
• Summary
• Case Studies

SLAM
2002 Simplified View
Predicate
C Program Abstraction
annotated with C2BP
API usage rules
buffer[head] = o;
head = (head+1)%size; Custom
}
Predicates Model Checker Correct
Object take() {
… BEBOP
tail=(tail+1)%size;
}
Symbolic Error-trace
Execution
NEWTON
False
Error-trace is Feasible
SLAM
2002
• Check API usage rules for sequential C programs
– Mostly applied to device driver code
• C2BP
– Inputs: C program and predicates
– Output: boolean program over the predicates
• BEBOP
– Symbolic interprocedural data flow analysis
– Concrete CFG and BDD encoding of states
• NEWTON
– Symbolic execution of C programs
– Using Simplify theorem prover for checking feasibility
of conditionals
Abstraction Refinement Example
Adapted from Ball & Rajamani POPL02
2002
Property: if a lock is held it must be released before reacquiring

do {
//get the write lock
KeAcquireSpinLock(&devExt->writeListLock);
nPacketsOld = nPackets;
request = devExt->WLHeadVa;
if (request){
KeReleaseSpinLock(&devExt->writeListLock);
...
nPackets++;
}
} while (nPackets != nPacketsOld);
KeReleaseSpinLock(&devExt->writeListLock);

Initial Abstraction and
2002 Model Checking
Boolean Program
[1] do
//get the write lock
[2] AcquireLock();
[3] if (*) then
[4] ReleaseLock();
fi
[5] while (*);
[6] ReleaseLock();
Error-trace : 1,2,3,5,1,2

Symbolic Execution
2002
[1] do {
[2] KeAcquireSpinLock(&devExt->writeListLock);
nPacketsOld = nPackets;
request = devExt->WLHeadVa;
[3] if (request){
[4] KeReleaseSpinLock(&devExt->writeListLock);
...
nPackets++;
}
[5] } while (nPackets != nPacketsOld);
[6] KeReleaseSpinLock(&devExt->writeListLock);
Symbolic execution of 1,2,3,5,1,2 shows that when 5 is executed

nPackets == nPacketsOld hence the path is infeasible.
The predicate nPackets == nPacketsOld is then added and
used during predicate abstraction
Next Abstraction and
2002 Model Checking
New Predicate b : (nPacketsOld == nPackets)
[1] do
[2] AcquireLock();
[3] b = true; // nPacketsOld = nPackets
[4] if (*) then
[5] ReleaseLock();
[6] b = b ? False : *; // nPackets++
fi
[7] while (!b); //(nPacketsOld != nPackets)
[8] ReleaseLock();
Now property holds

Overview
2002
– Major Trends
– A Brief History
– Current Trends
• SLAM
• JPF
– Abstractions
– Partial-order and symmetry reductions
– Heuristics
– New Stuff
• Summary
• Case Studies

Java PathFinder (2)
2002 Direct Approach
• Based on custom-made Java Virtual Machine
– Handle all of Java, since it works with bytecodes
– Written in Java
• Efficient encoding of states
• Modular design for easy extensions
• Supports LTL checking with properties expressed in
Bandera’s BSL notation
• Incorporates a number of search strategies
– DFS, BFS, A*, Best-first, etc.
• Supports source-2-source abstractions
• http://ase.arc.nasa.gov/jpf
Java PathFinder (JPF)
2002
Java Code Bytecode
buffer[head] = o; 0: iconst_0
head = (head+1)%size; 1: istore_2
} 2: goto #39
Object take() { JAVAC 5:

8:
getstatic
aload_0 JVM
… 9: iload_2
tail=(tail+1)%size; 10: aaload
}
Model
Checker
Special
JVM

Bandera & JPF
Architecture
2002
Property Tool Abstraction Translators

Analyses Engine BIRC BIR
SPIN
dSPIN
Java
Jimple (BC)
Parser
SMV
Slicer Simulator
Error Trace Display Decompile ; javac
JPF
Key Points
2002
• Models can be infinite state
– Unbounded objects, threads,…
– Depth-first state generation (explicit-state)
– Verification requires abstraction
• Handle full Java language
– but only for closed systems
– Cannot handle native code
• no Input/output through GUIs, files, Networks, …
• Must be modeled by java code instead
• Allows Nondeterministic Environments
– JPF traps special nondeterministic methods
• Checks for User-defined assertions, deadlock and LTL properties

Java Extensions
-Verify Class-
2002
• Used for annotations

• Atomicity
– beginAtomic(), endAtomic()
• Nondeterminism
– int random(int);
boolean randomBool();
Object randomObject(String cname);
• Properties
– AssertTrue(boolean cond)
• Miscellaneous
– Dump states, print, Daikon logs, …

JPF Technologies
2002
• Predicate abstraction and type based
abstractions
• Partial order and symmetry reductions
• Slicing
• Heuristic Search
• New Stuff
– Symbolic Execution
– Error Explanations
Type-based Abstractions
Abstract Interpretation
2002
abstraction Signs abstracts int public class Signs {
public static final int NEG = 0;
TOKENS = { neg, zero, pos }; public static final int ZERO = 1;
public static final int POS = 2;
abstraction mapping: public static int abs(int n) {

if (n < 0) return NEG;
n < 0 -> {neg}; if (n == 0) return ZERO;
n == 0 -> {zero}; if (n > 0) return POS;
n > 0 -> {pos}; }
public static int add(int a, int b){

int r;
Verify.beginAtomic();
zero pos neg if (a==NEG && b==NEG) r=NEG;
+abs
if (a==NEG && b==ZERO) r=NEG;
zero zero pos neg if (a==ZERO && b==NEG) r=NEG;
if (a==ZERO && b==ZERO) r=ZERO;
pos pos pos {zero,pos,neg}
if (a==ZERO && b==POS) r=POS;
neg neg {zero,pos,neg} neg if (a==POS && b==ZERO) r=POS;
if (a==POS && b==POS) r=POS;
else r=Verify.choose(2);
24 September 2002 © Willem Visser 2002
Verify.endAtomic();
69
return r; }}
JPF
Predicate Abstraction
2002
• Annotations used to indicate abstractions

– Abstract.remove(x);
Abstract.remove(y);
Abstract.addBoolean(“EQ”, x==y);
• Tool generates abstract Java program
– Using Stanford Validity Checker (SVC)
– JVM is extended with nondeterminism to handle over approximation
• Abstractions can be local to a class or global across multiple
classes
– Abstract.addBoolean(“EQ”, A.x==B.y);
– Dynamic predicate abstraction - works across instances

Partial Order Reduction in JPF
2002
• Persistent set approach
• Find transitions that are globally independent
– Always independent of all transitions in other
threads
• Requires static analysis before model checking
to determine global independence
– Advanced alias analysis

Partial-order Reduction
2002
class S1 {int x;} class S2 {int y;}
public class Example {

public static void main (String[] args) {
FirstTask t1 = new FirstTask();
SecondTask t2 = new SecondTask();
t1.start(); t2.start();
}}
class FirstTask extends Thread { class SecondTask extends Thread {

public void run() { public void run() {
int x; S1 s1; int x; S2 s2;
x = 1; x = 2;
s1 = new S1(); s2 = new S2();
x = 3; x = 3;
}} }}
• 43 states with no reduction
• 18 states with partial-order reduction
• all statements are globally independent (safe)
Partial-order Reduction
2002
class S1 {int x;} class S2 {int y;}
public static int x = 10;
}}

int x; S1 s1; int x; S2 s2;
Example.x = 1; Not Safe Example.x = 2;
s1 = new S1(); s2 = new S2();
x = 3; x = 3;
}} }}
• 43 states with no reduction
• 27 states with partial-order reduction
• 2 statements are not globally independent
Symmetry Reduction
2002
class S {int x;}
}}
S s1; S s2;
1 s1 = new S1(); 2 s2 = new S2();
}} }}
1 followed by 2
t1 t2 s1 s2 • Map allocation to Heap
0 1 2 3 position on First Path
2 followed by 1 • Reuse same Positions
t1 t2 s2 s1 on later paths
0 1 2 3
Garbage Collection
2002
class gc {
while(true) {
System.out.println(“0”);
}}}
• Infinite State program

• Garbage makes the state grow…
• JPF supports
• Mark-and-Sweep
• Reference counting

Property-directed
Slicing
2002
indirectly
relevant
Slice
mentioned
in property Source program Resulting
slice
• slicing criterion generated automatically from
observables mentioned in the property
• backwards slicing automatically finds all components that
might influence the observables.
Property-directed Slicing
2002
/**
* @observable EXP Full: (head == tail) Slicing Criterion
*/ All statements
that assign to
class BoundedBuffer { head, tail.
Object [] buffer_;
int bound; removed by
int head, tail; slicing
public synchronized void add(Object o) {

while ( tail == head )
try { wait(); } catch ( InterruptedException ex) {}
buffer_[head] = o;
Included in
head = (head+1) % bound;
slicing notifyAll();
criterion } indirectly
...
relevant
}
Slicing in JPF
2002
• JPF uses Bandera’s slicer
• Bandera slices w.r.t.
– Deadlock - i.e. communication statements
– Variables occurring in temporal properties
– Variables participating in race-violations
• Used with JPF’s runtime analysis
• More examples of slicing for model checking
– Slicing for Promela (Millet and Teitelbaum)
• http://netlib.bell-labs.com/netlib/spin/ws98/program.html
– Slicing for Hardware Description Languages (Shankar et al.)
• http://www.cs.wisc.edu/~reps/

Heuristic Search
2002
• Best-First, Beam and A* Search
• Heuristics based on property
– deadlock
• Maximize number of blocked threads
– Assertions
• Minimize distance to assertion
• Heuristics on structure of Program
– Interleaving heuristic
• Maximize different thread scheduling
– Branch Exploration
• Maximize the coverage of new branches
– Choose-free heuristic
• Minimize non-deterministic choice
• User-defined heuristics
– Full access to JVM’s state via API
• Combine heuristics

Choose-free state space
search
2002
• Theorem [Saidi:SAS’00]
Every path in the abstracted program where all assignments
are deterministic is a path in the concrete program.
• Bias the model checker
– to look only at paths that do not
include instructions that introduce
non-determinism
• JPF model checker modified
– to detect non-deterministic choice and backtrack from those
points

Choose-free Heuristic
2002
• Infeasible error elimination during
abstraction
• Heuristic function returns best value
Sta
for states with least number of non-
te
deterministic choices enabled
s pa
ce s
• If no “deterministic” error exists it
ear
also searches rest of the state space
c he
d fi
rst
X X

Test Coverage for Model
Checking
2002
• Reality check:
– Model Checkers run out of memory OFTEN!
• If no error was found, how confident can we be that
none exists?
– Require coverage measure
• JPF extended with Branch coverage calculations
• Can the coverage measure guide the model checker
– Yes, as a heuristic
– Better heuristic values are given for least explored branches

Concurrency Bugs
2002
• Often due to unanticipated interleaving
• Increased thread interleaving might expose
them
• Interleaving heuristic
– Executions in which context is switched more
often are given better heuristic values

Scaling Program Model Checking
Error-Detection
2002
1200.00
1000.00 Autopilot
JPF
1000.00
DEOS
800.00 Java-translation
JPF
Mars Rover
600.00 Remote Agent 550.00 JPF
Hand-translation
400.00 SPIN 330.00 DEOS
Systematic
200.00 Hand-translation
50.00 SPIN
30.00
0.00
1997 1998 2000 2001 2002
24 September 2002
LOC analyzed per Person day
© Willem Visser 2002 84
JPF
2002 Most Recent Extensions
• Symbolic execution based model checking
– Sarfraz Khurshid (MIT), Corina Pasareanu and
Doron Peled (Warwick)
– Traverses all paths and collects and solves constraints on-the-fly
during model checking
– Using the Omega libraries for constraint solving
– Main application is test-case generation
• Error Explanations
– Alex Groce (CMU)
– Find an error, then try and find “similar” paths that lead to errors
(negatives) and that don’t lead to an error (positives).
– Analyze the positives and negatives
• Find smallest change to make a positive a negative
• Find invariants on positives and negatives (using Diakon)
• Find statements that are on all and only positives (negatives)

Overview
2002
– Major Trends
– A Brief History
– Current Trends
• SLAM
• JPF
• Summary
• Case Studies

2002 Executive summary
• Model checking by itself cannot deal with the
complexity of software
• Techniques from static analysis are required
– Abstract interpretation, slicing, alias&shape analysis,
symbolic execution
• Even then, we need to borrow some more!
– Heuristic search, constraint solving, etc.
• Abandon soundness
– Aggressive heuristics
– Runtime analysis and runtime monitoring
More Software Analysis Techniques
2002 A small sample
• Program Verification
– For example, ESC/Java from Compaq
• http://research.compaq.com/SRC/esc/
• Static analysis for runtime errors
– For example, PolySpace for C, Ada and Java
• http://www.polyspace.com/
• Requirements and Design Analysis
– Analysis for SCR, RSML, Statecharts, etc.
• Runtime analysis
– See Runtime Verification Workshops
• http://ase.arc.nasa.gov/rv2002/
• Analysis Toolsets
– IF (Verimag), SAL (SRI), etc.
Overview
2002
• Case Studies
– Remote Agent
– DEOS
– Mars Rover

Case Studies
2002
DEOS Remote Agent
Mars Rover

Case Study:
DS-1 Remote Agent
2002
Spacecraft
Commands
Tasks Sensors
Achieve
Property
Subscribe
Property Data
Locks base
Lock
Event
Change
Interrupt
Event
Properties Monitor
• Several person-months to create verification model.

• One person-week to run verification studies.
Case Study:
DS-1 Remote Agent
2002
Unexpected timing
DB change? of change event
Monitor Logic
yes no
check wait
• Five difficult to find concurrency errors detected
• “[Model Checking] has had a substantial impact, helping the RA team improve the
quality of the Executive well beyond what would otherwise have been produced.” -
RA team
• During flight RA deadlocked (in code we didn’t analyze)
– Found this deadlock with JPF
DEOS Operating System
2002
• Integrated Modular Avionics (IMA)
– DEOS Guarantee Space and Time partitioning
• FAA Certification Process

– Requires Structural Testing Coverage (MC/DC)
– Inadequate for finding Time Partitioning Errors
• Timing Error not found by Testing occurred
• Behavioral Analysis of Time Partitioning

– NASA Ames and Honeywell HTC collaboration
– Model Check slice of DEOS containing timing error

DEOS Analysis
2002
• Translated C++ 1-to-1 to PROMELA/SPIN (1500 lines of C++ code)

– Found the time-partitioning error without any prior knowledge, what the error
was, where it was or what made it show up.
– Required very limited abstraction
• DEOS Team Reaction

– Surprised that error was found by directly checking code
– They expected NASA team to ask for smaller “slice”
– They now have their own model checking group building on our work
• Then translated DEOS to Java and applied JPF

– Backwards dependency analysis from the time partitioning assertion being
checked revealed candidate variables to abstract
– Applied “range” abstraction {0,1,many} to a specific integer variable
– Too much of an over-approximation that led to many spurious errors
– However with the choose-free heuristic the non-spurious error was found

Error Trace
2002
Idle never ran

User2: 16/21
idle
8 8
User2: 21/60 preempt
User1: 21/60 15
delete
5 12 12
Main: 5/20 timer timer timer
12/20
0 20 40 60

Analysis of the K9 Mars Rover
2002 “The Experiment”
• Rover is 8000 lines of code with 6 threads
– heavy use of synchronization between the threads
– Complex queue manipulation
• Purpose
– Benchmark current state of the art in model checking, static analysis for
runtime error detection and runtime analysis
– Use traditional testing as baseline
– Original code was in C++ that was translated to Java
• About half the code was translated to C for the static analysis that used PolySpace
• Method
– Controlled experiment: 4 groups of 2 people, each group uses one technology
on the Mars rover code to find seeded bugs
– 3 versions created and each group gets 2 days/version
– Some bugs are removed/introduced between versions
– Any new bugs discovered are not fixed, only known ones

Analysis of the K9 Mars Rover
2002 How did Model Checking do?
• Methodology for model checking
– Asked never to “run” the code, only model check it
• Keep the results clean from any testing influence
– Code is heavily dependent on time
• Given a gross over-approximation of time, where all time-related decisions became
nondeterministic
• Found all, but one, of the known concurrency errors and some new ones
– Better than any of the other teams
– Only team that could always produce not just the error but how to get to it!
– Also found all the non-concurrency errors
• Interesting observations
– Abandoned the time abstraction within the first hour for one that is closer to
real-time, but might miss errors
• It was too hard for them to determine if errors were spurious not knowing the code
well enough
– Found a number of bugs in the first version, had a slow 2nd version, and then
found all the remaining bugs in the 1st hour of the 3rd version
• Took them some time to get their framework setup, but once done, they were flying
Overview
2002
• Case Studies

The Future of
2002 Software Model Checking
• Abstraction based approaches
– Combine object abstractions (e.g. shape analysis) with predicate abstraction
– Automation is crucial
• Symbolic Execution
– Solving structural (object) and numerical constraints
– Acceleration techniques (e.g. widening)
• Model checking as a companion to testing
– Test-case generation by model checking
– Runtime monitoring and model checking
• Modular model checking for software
– Exploiting the interface between components
– Interface automata (de Alfaro & Henzinger )
• Environment generation
– How to derive a “test-harness” for a system to be model checked
• Result representation
– Much overlooked, but without this we are nowhere!
– “Analysis is necessary, but not sufficient” – Jon Pincus

Software Model Checking: Willem Visser

Enviado por

Dados do documento

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Software Model Checking: Willem Visser

Enviado por

Direitos autorais:

Formatos disponíveis

2002

Software Model Checking

Research Institute for Advanced Computer Science

24 September 2002 © Willem Visser 2002 2

Each state represents all

24 September 2002 © Willem Visser 2002 4

• Linear Time • Branching Time

24 September 2002 © Willem Visser 2002 5

24 September 2002 © Willem Visser 2002 6

C1N2S1 T1T2S0 N1C2S1

C1N2S1 T1T2S0 N1C2S1

24 September 2002 © Willem Visser 2002 9

C1N2S1 T1T2S0 N1C2S1

24 September 2002 © Willem Visser 2002 10

C1N2S1 T1T2S0 N1C2S1

24 September 2002 © Willem Visser 2002 11

C1N2S1 T1T2S0 N1C2S1

24 September 2002 © Willem Visser 2002 12

C1N2S1 T1T2S0 N1C2S1

24 September 2002 © Willem Visser 2002 13

• Normally, some states of the concurrent system are

24 September 2002 © Willem Visser 2002 14

24 September 2002 © Willem Visser 2002 15

24 September 2002 © Willem Visser 2002 16

24 September 2002 © Willem Visser 2002 19

Most model checkers cannot deal with the features of

• Bringing programs to model checking

24 September 2002 © Willem Visser 2002 20

Infinite state Finite state

24 September 2002 © Willem Visser 2002 22

24 September 2002 © Willem Visser 2002 23

24 September 2002 © Willem Visser 2002 25

24 September 2002 © Willem Visser 2002 26

24 September 2002 © Willem Visser 2002 27

24 September 2002 © Willem Visser 2002 29

24 September 2002 © Willem Visser 2002 30

• Hand translation of program to model checker’s input notation

24 September 2002 © Willem Visser 2002 31

24 September 2002 © Willem Visser 2002 33

24 September 2002 © Willem Visser 2002 34

No Reductions Transitions Reduced States Reduced

24 September 2002 © Willem Visser 2002 35

24 September 2002 © Willem Visser 2002 36

limit the number of times a state is

24 September 2002 © Willem Visser 2002 38

• DFS with state storage – executes 12 transitions

24 September 2002 © Willem Visser 2002 39

• DFS with state storage – executes 12 transitions

24 September 2002 © Willem Visser 2002 40

• DFS with state storage – executes 12 transitions

24 September 2002 © Willem Visser 2002 42

24 September 2002 © Willem Visser 2002 43

24 September 2002 © Willem Visser 2002 44

• Custom-made model checkers for programming languages with

24 September 2002 © Willem Visser 2002 47

24 September 2002 © Willem Visser 2002 48

Collapses data domains via abstract interpretation:

int  int bool EQ (x = y) EQ (x = y)

Concrete xy x y

• Mapping of a concrete system to an abstract system, whose states

24 September 2002 © Willem Visser 2002 54

24 September 2002 © Willem Visser 2002 55

Property: if a lock is held it must be released before reacquiring