Accountin~ Organizations and Society, Vol. 17, No. 2, p p ! 69-183, 1992.

Printed in Great Britain

0361-3682/92 $5.00 +.O0

Pergamon Press pie

AUDITOR BELIEF REVISIONS IN A PERFORMANCE AUDITING SETTING: AN

APPLICATION OF THE BELIEF-ADJUSTMENT MODEL*

B U C K K. W. PEI

Arizona State University

SARAH A. REED

Texas A&M University

and
B R U C E S. K O C H

University o f North Texas

Abstract

Based upon Hogarth and Einhorn's "belief-adjustment model", the present research tested the model's
recency effects prediction in a "performance auditing" context using state auditors. The results showed that
the auditors' "belief revisions" were subject to the recency effects predicted by the model. The results also
validated an important assumption of the mendel that serial position effects were absent in a short evidence
series. That is, serial position had no effect on the subject's subjective evaluation of evidence. Finally, analyses
of the subjects' intermediate belief changes suggested that they displayed a "confirmatory attitude" towards
evidence rather than a "disconfirmatory attitude" found in the prior studies conducted in financial auditing
contexts.

T h e p r o c e s s b y w h i c h n e w i n f o r m a t i o n is
integrated with current beliefs has long been an
important area of research in decision making.
O n e i m p o r t a n t f o c u s o f t h i s r e s e a r c h is t h e
study of order effects on belief revision.
However, the research reveals that the order of
information presentation
sometimes has no
effect, m a y i n d u c e p r i m a c y effects, o r m a y
c a u s e r e c e n c y e f f e c t s o n b e l i e f r e v i s i o n (e.g.
N i s b e t t & Ross, 1 9 8 0 ) . R e c e n t l y , H o g a r t h &
Einhorn (1990)
proposed
and validated a
descriptive model, called the "belief-adjustment
model", to account for the mixed results found
in this literature. The model posits that decision
makers tend to use a simple anchoring-andadjustment strategy for belief revision, and that

this tendency and their adaptive behaviors to

the characteristics of the decision tasks yield "a
complex pattern of predictions concerning
order effects" (Hogarth & Einhorn, 1990, p.3).
T h e p u r p o s e o f t h e p r e s e n t r e s e a r c h is t o t e s t
the belief-adjustment model predictions in a
performance auditing context. This research
e f f o r t is m o t i v a t e d By t h e f o l l o w i n g t h r e e
o b s e r v a t i o n s . First, m o s t r e c e n t a p p l i c a t i o n s o f
the belief-adjustment model have been conducted
i n f i n a n c i a l a u d i t i n g c o n t e x t s (e.g.
A s h t o n & A s h t o n , 1 9 8 8 ; T u b b s et al., 1 9 9 0 ; f o r
a literature review see Asare & Messier,
1990). However, the decision environment of a
p e r f o r m a n c e a u d i t is s i g n i f i c a n t l y d i f f e r e n t f r o m
that of a financial audit. The objective of a

* This research is partially funded by a summer research grant to the first author by the FGIA program of Arizona State
University. The authors gratefully acknowledge comments by Robin M. Hogarth, Robert Ashton and three anonymous
referees on the earlier versions of this paper.
169

BUCK K. W. PEI et al.

170

performance
audit is to evaluate
a social
programs efficiency and effectiveness in utilizing
public resources
(e.g. Herbert,
1979). In contrast, the objective of a financial audit is to attest
to the company managements
assertions in the
financial statements.
While the generic process
of conducting
both financial and performance
audits may be identical,
the decision
consequences
perceived
by the auditors may be
quite different. Auditors performing
a financial
audit are subject
to potential
legal penalties
when they fail to detect misrepresentations
in
the financial statements. On the other hand, the
possibility
of legal penalties
is absent
for
the auditors performing
a performance
audit.
Evidence
in the decision-making
literature
suggests that different perceptions
consequences
(including
their
indeed, frequently
1987;
1982).

Kahneman

alter judgments
&

Tversky,

of decision
magnitude),
(e.g. Hogarth
1979;

Payne,

evidence may be of low diagnosticity

since it is
dBicult
to make unequivocal
causal connections between
the evidence
and a social
programs efficiency or effectiveness.
Third, both Ashton & Ashton ( 1988)
and
Tubbs et al. ( 1990) found that auditors seemed
to exhibit a disconfirmatory
attitude towards
evidence.
The studies showed
that auditors
revised their beliefs to a greater extent when
they received negative evidence as opposed to
positive evidence,
suggesting
that they were
more sensitive to negative evidence than positive
evidence in belief revision. This is an interesting
finding because research in behavioral decision
theory
has found the opposite:
a stronger
tendency for people to seek and use evidence
that confirms their belief (e.g. Klayman & Ha,
1987;

Fischhoff

& Reyth-Marom,

1983;

Einhorn

& Hogarth, 1987).3 In the light of the previous

observations, we consider that a disconfirmdtory
attitude towards evidence found in Ashton &

Second, in many substantive domains, information diagnosticity

is germane to belief revision
(cf. Einhorn & Hogarth 1985a, 1986). Information is high (low)
in diagnosticity
if it is

Ashton ( 1988) and Tubbs et al. ( 1990) may not

be generalizable
to performance
auditing contexts. Specifically,
Ashton & Ashton (1988)
state: the nature of auditing is such that it is

more easy (difficult)

to allow cause-to-effect
inferences.
In most prior
belief-adjustment
model studies (e.g. Ashton & Ashton, 1988;

appropriate
for negative evidence to be given
more weight than positive evidence, since the
primary risk that auditors face is that errors

Tubbs et al., 1990) the subjects were presented

with tasks of very specific and narrow scope*
(e.g. internal
conrol
evaluations
for specific

in financial statements
may
(pp. 639-640).
This suggests

transaction

cycles,

collectability

of accounts

receivable).
Thus, the evidence
presented
in
those studies may be high in diagnosticity.
Comparatively,
performance
auditing of a social
program
necessarily
involves
examining
a
broader scope of information with the available
evidence
subject
to uncertainty
in making
proper causal inferences.
That is, the available

go undetected
that the discon-

lirmatory attitude towards evidence exhibited

by the subjects may be due to their concern for
potential legal litigation, which is absent in the
performance auditing environment. Additionally,
low evidence diagnosticity
in this audit context
seems to further discourage
a disconfirmatory
attitude towards evidence (cf. Klayman & Ha,
1987).
Instead, prior evidence
suggests that
when information

is low in diagnosticity,

The preceding argument does not suggest that the decision environment
of state auditors is penaltyfree.
was to highlight legal penalties as a salient factor that clearly distinguish the decision environments
between
of audit. Further, these legal penalties frequently are immense.
See

Asare ( 1990)

people

Our intention
the two types

for an exception.

This phenomenon
is frequently
labeled as confirmatory
bias. Note, most of thse confirmatory
bias studies were
interested in examining peoples hypothesis testing and/or information search behaviors. Thus, the results from Ashton &
Ashton (1988)
and Tubbs et al. (1990)
should not be interpreted
as contradictory,
since their findings relate to the
weighing of evidence rather than the searching of evidence.

AUDITOR BELIEFREVISIONS
are m o r e affected b y p o s i t i v e e v i d e n c e than
n e g a t i v e e v i d e n c e (e.g. H o c h & Ha, 1986;
D e i g h t o n , 1984; Reed, 1986). In brief, o t h e r
than testing the m o d e l ' s o r d e r effects prediction,
this s t u d y also e x p l o r e s w h e t h e r p e r f o r m a n c e
a u d i t o r s d i s p l a y a c o n f i r m a t o r y ( r a t h e r than
a d i s c o n f i r m a t o r y ) a t t i t u d e t o w a r d s evid e n c e b e c a u s e o f t h e d i f f e r e n c e s in t h e task
environment.
As a further a t t e m p t to validate the beliefadjustment model, the p r e s e n t study also tests a
crucial assumption o f the belief-adjustment model:
t h e a b s e n c e o f serial p o s i t i o n effects ( d e s c r i b e d
later). To the best o f o u r knowledge, this
assumption has n o t b e e n tested b y Hogarth &
Einhorn ( 1 9 9 0 ) , n o r b y the p r i o r auditing studies.
T h e r e m a i n d e r o f this p a p e r p r o c e e d s
as follows. First, w e s u m m a r i z e t h e beliefa d j u s t m e n t m o d e l p r o p o s e d b y H o g a r t h and
Einhorn and describe the model's predictions
t e s t e d in t h e p r e s e n t study. Second, w e highlight t h e d i s t i n c t n a t u r e o f t h e p e r f o r m a n c e
a u d i t i n g tasks u s e d in t h e study. Third, w e
d e s c r i b e t h e r e s e a r c h m e t h o d . Fourth, w e
r e p o r t t h e results. Finally, w e e l a b o r a t e o n t h e
r e s e a r c h findings and suggest s o m e a v e n u e s for
future r e s e a r c h .
BELIEF-ADJUSTMENT MODEL
Hogarth & Einhorn's ( 1 9 9 0 ) belief-adjustment
m o d e l a s s u m e s that p e o p l e u p d a t e beliefs using
a s i m p l e a n c h o r i n g and a d j u s t m e n t strategy.
Hogarth and Einhorn argue that decision makers
are a d a p t i v e to t h e task e n v i r o n m e n t ; t h e r e f o r e
the p r e d i c t i o n o f a l t e r n a t i v e o r d e r effects o n
belief revision can be better understood by
explicitly considering the characteristics of the
d e c i s i o n tasks ( a l s o see Payne, 1982; Payne e t
al., in press). To illustrate this adaptive behavior,
they identify t h r e e key task variables: complexity,
l e n g t h o f e v i d e n c e series, and r e s p o n s e m o d e .
Based u p o n the belief-adjustment model, Hogarth
and Einhorn have illuminated various task conditions u n d e r w h i c h alternative o r d e r effects (i.e.
p r i m a c y r e c e n c y no effect) m a y prevail. Following
t h e i r d i s t i n c t i o n , t h e m o d e l ' s p r e d i c t i o n s are
briefly s u m m a r i z e d using t h r e e task categories:

171

(A) For tasks that involve simple evidence with a short

evidence series, response mode makes a difference. Endof-Sequence (EoS) induces primacy, whereas Step-byStep (SbS) induces recency.
(B) For tasks that involve complex evidence with a
short evidence series, both EoS and SbS response modes
induce recency (for a m i x e d series of positive and
negative evidence) or induce no order effect (for a
consistent series of all positive or negative evidence).
(C) For tasks that involve a long evidence series, both
EoS and SbS response modes induce primacy, regardless
of the complexity of evidence (simple or complex) and
the consistency o f the evidence series (mixed or
consistent ).
Based o n a task analysis o f o v e r 6 0 e x p e r i m e n t a l
studies, H o g a r t h a n d E i n h o r n o b s e r v e d that
m o s t o f t h e s e s t u d i e s u s e d d e c i s i o n tasks in
e i t h e r c a t e g o r y ( A ) o r ( C ) , a n d that t h e findings
of these studies were consistent with the above
p r e d i c t i o n s . V e r y few s t u d i e s h a v e e x a m i n e d
tasks in c a t e g o r y (B). Using tasks in c a t e g o r y
(B), H o g a r t h a n d E i n h o r n c o n d u c t e d a series o f
e x p e r i m e n t s to test t h e p r e d i c t e d o r d e r effects.
A similar a p p r o a c h also w a s u s e d in t h e p r i o r
a u d i t i n g s t u d i e s (e.g. A s h t o n & Ashton, 1988;
T u b b s e t al., 1990). C o n s i s t e n t w i t h t h e
m o d e l ' s p r e d i c t i o n s , t h e s e s t u d i e s f o u n d no
o r d e r effect for c o n s i s t e n t e v i d e n c e a n d a
r e c e n c y effect for m i x e d e v i d e n c e , i r r e s p e c t i v e
o f a l t e r n a t i v e r e s p o n s e m o d e s . W e are also
i n t e r e s t e d in t e s t i n g t h e m o d e l ' s r e c e n c y
effect p r e d i c t i o n s for tasks u n d e r c a t e g o r y (B).
Specifically, in this s t u d y w e t e s t t h e m o d e l ' s
r e c e n c y effect p r e d i c t i o n for m i x e d e v i d e n c e
u n d e r an SbS r e s p o n s e m o d e .
To e x p l a i n w h y r e c e n c y effects o c c u r u n d e r
this c o n d i t i o n , t h e b e l i e f - a d j u s t m e n t m o d e l
specifies belief revisions, u p o n receiving negative
and p o s i t i v e e v i d e n c e , r e s p e c t i v e l y , as follows:
S~ =S~_l + 0~._, [S(xle) -- R ] [ - - 1 <~ s ( x ~ ) ~< O;
for n e g a t i v e e v i d e n c e ] ,
( 1a )
S~ = S~_, + ]3( 1 - Sa~-]) [s(xje) - R} [ 1 ~< s ( k ) > 0 ;
for p o s i t i v e e v i d e n c e ],
( 1b )
where
Sk
Sj~--I

= r e v i s e d b e l i e f after e v a l u a t i n g k
p i e c e s o f e v i d e n c e , 0 <~S~<~ 1,
= a n c h o r o r p r i o r o p i n i o n . T h e initial
b e l i e f is d e n o t e d So,

BUCK K. W. PE! et al.

172

s(x~)

-- s u b j e c t i v e e v a l u a t i o n o f t h e k t h p i e c e
of evidence,
R
= the r e f e r e n c e p o i n t against w h i c h t h e
i m p a c t o f t h e k t h p i e c e o f e v i d e n c e is
evaluated. (Similar to t h e p r i o r studies,
this s t u d y a s s u m e s R = 0 b e c a u s e t h e
e x p e r i m e n t a l tasks u s e d i n v o l v e evaluation r a t h e r than e s t i m a t i o n ) .
ct and ~ are t h e sensitivity t o w a r d n e g a t i v e a n d
p o s i t i v e e v i d e n c e , r e s p e c t i v e l y ( 0 ~< ~,
~<1).
To illustrate w h y r e c e n c y effects occur, c o n s i d e r
belief revisions w i t h t w o p i e c e s o f e v i d e n c e
u n d e r t w o a l t e r n a t i v e o r d e r s ( + , - ) vs ( - , + ) .
G i v e n e q u a t i o n ( l b ) , p o s i t i v e e v i d e n c e in t h e
(+,-)
o r d e r has a s m a l l e r u p w a r d r e v i s i o n
effect than t h e p o s i t i v e e v i d e n c e in t h e ( - , + )
order. This is b e c a u s e t h e a n c h o r o f t h e f o r m e r
( 1 - S o ) is s m a l l e r than t h e a n c h o r o f t h e
l a t t e r (1-$1 ), ceteris paribus. Likewise, given
e q u a t i o n ( l a ) , n e g a t i v e e v i d e n c e has a s m a l l e r
d o w n w a r d r e v i s i o n effect in t h e ( - , + ) o r d e r
than t h e e v i d e n c e in t h e ( + , - ) o r d e r b e c a u s e
t h e a n c h o r o f t h e former, So, is s m a l l e r than t h e
a n c h o r o f t h e latter, St, ceteris paribus. T h e s e
c o n t r a s t effects p r o v i d e t h e m o d e l p r e d i c t i o n s
that t h e final p o s i t i o n o f b e l i e f r e v i s i o n r e a c h e d
u n d e r t h e ( + , - ) o r d e r will b e l o w e r than its
counterpart reached under the (-,+)
order
d u e to t h e d i f f e r e n c e s ( b e t w e e n t h e t w o
o r d e r s ) in t h e a n c h o r s at k = 1. This p r e d i c t i o n
is t e r m e d t h e recency effect.
H o g a r t h a n d E i n h o r n also d e m o n s t r a t e d t h e
a b o v e r e c e n c y analytically. T h e y d e f i n e d the
r e c e n c y effect as ( s e e H o g a r t h & E i n h o r n 1990,
a p p e n d i x C for d e r i v a t i o n ) :
D = s(-,+)-s(+,-),
where D = -

~s(x_)s(x+) ( a n d

D>

0).

(2)

Thus for t w o p i e c e s o f m i x e d e v i d e n c e , t h e size

o f the r e c e n c y effect is t h e p r o d u c t o f b o t h
sensitivity to e v i d e n c e [ct and [~] a n d t h e
subjective strengths of the two pieces of
e v i d e n c e [ s ( x _ ) and s(x+)].
O n e i m p o r t a n t a s s u m p t i o n o f the m o d e l ' s
r e c e n c y effects p r e d i c t i o n is that serial p o s i t i o n

has n o effect o n s u b j e c t i v e e v a l u a t i o n o f
e v i d e n c e . That is, t h e s t r e n g t h o f s ( x k ) w i l l n o t
d e c l i n e d u e to d e c r e a s i n g sensitivity t o w a r d
e v i d e n c e . This a s s u m p t i o n is also i m p l i e d in
t h e task t a x o n o m y m e n t i o n e d earlier. A k e y
d i f f e r e n c e b e t w e e n c a t e g o r i e s ( B ) a n d ( C ) rests
o n the l e n g t h o f e v i d e n c e ; a s h o r t ( l o n g )
e v i d e n c e series i n d u c e s r e c e n c y ( p r i m a c y ) .
Hogarth and Einhorn argue that for a short ( l o n g )
e v i d e n c e series, sensitivity t o w a r d e v i d e n c e
s h o u l d b e c o n s t a n t ( m a y d e c l i n e ) . Thus, for
s(xk), t h e e v a l u a t i o n is insensitive to t h e serial
p o s i t i o n in a s h o r t e v i d e n c e series, w h e r e a s t h e
e v a l u a t i o n m a y b e s u b j e c t to fatigue ( t h u s
i n d u c i n g p r i m a c y ) in a l o n g e v i d e n c e series. In
t h e p r e s e n t study, w e e x p l i c i t l y t e s t e d t h e
p r e c e d i n g crucial a s s u m p t i o n w h i l e t e s t i n g t h e
r e c e n c y effects p r e d i c t i o n o f t h e m o d e l .

PERFORMANCE AUDITING ENVIRONMENT

A p e r f o r m a n c e a u d i t o f a social p r o g r a m
a t t e m p t s to establish a link b e t w e e n t h e
p r o g r a m i n t e r v e n t i o n s a n d t h e r e s u l t i n g outc o m e s . In t h e c o u r s e o f a p e r f o r m a n c e audit,
t h e a u d i t o r g a t h e r s e v i d e n c e o n the effects o f a
.social p r o g r a m ' s interventions using t w o related
measures: outputs and impacts. O u t p u t measures
f o c u s o n a social p r o g r a m ' s efficiency b y
gauging its r e s o u r c e utilization. I m p a c t measures,
b y contrast, focus o n a social p r o g r a m ' s effectiveness in m e e t i n g its p r e - e s t a b l i s h e d l o n g - r a n g e
objectives (Reed, 1986). The e v i d e n c e o b t a i n e d
in a p e r f o r m a n c e a u d i t m a y b e l o w e r in
d i a g n o s t i c i t y than that o b t a i n e d in t h e financial
a u d i t c o u n t e r p a r t . In p a r t i c u l a r , t h e o u t p u t
m e a s u r e e v i d e n c e d e s c r i b e s the actual g o o d s o r
s e r v i c e s p r o d u c e d b y a social p r o g r a m . T h e
i m p a c t m e a s u r e e v i d e n c e d e p i c t s t h e d e g r e e to
w h i c h t h e p r o g r a m ' s s t a t e d d e s i r a b l e social
o u t c o m e s have b e e n r e a l i z e d a n d t h e basic
s o c i e t a l p r o b l e m s that t h e p r o g r a m is a t t a c k i n g
h a v e b e e n rectified ( R e e d , 1986). Both t y p e s o f
e v i d e n c e are n o t easily available o n a shortt e r m basis. Further, e v e n w h e n s u c h e v i d e n c e is
available, it m a y b e difficult for a d e c i s i o n m a k e r

AUDITOR BELIEFREVISIONS
to d r a w u n e q u i v o c a l causal inferences. In part,
this is b e c a u s e a social p r o g r a m ' s efficiency a n d
effectiveness are f r e q u e n t l y affected b y factors
u n r e l a t e d to t h e p r o g r a m ' s i n t e r v e n t i o n p r o c e s s
( G a m b i n o & R e a r d o n , 1981; Nachmias, 1979;
Rossi et al., 1979). 4
The preceding observations may be further
e m p h a s i z e d from a causal r e a s o n i n g p e r s p e c tive. A c c o r d i n g to E i n h o r n & H o g a r t h ( 1 9 8 6 ) ,
a d e c i s i o n m a k e r ' s causal j u d g m e n t will b e
s u b j e c t to h i g h e r u n c e r t a i n t y w h e n t h e causeand-effect r e l a t i o n s h i p is l o w in c o n t i g u i t y in
t i m e and space, a n d w h e n t h e n u m b e r o f
alternative causal explanations increases. Jointly,
t h e s e o b s e r v a t i o n s suggest that in c o n d u c t i n g
a p e r f o r m a n c e audit, t h e a u d i t o r s m a y b e
c o n f r o n t e d w i t h h i g h u n c e r t a i n t y in j u d g i n g a
social p r o g r a m ' s efficiency a n d effectiveness,
s i n c e t h e d i a g n o s t i c i t y o f t h e available e v i d e n c e
is fairly low.
W i t h r e s p e c t to b e l i e f revision, t h e p r e c e d i n g
o b s e r v a t i o n s p r e s e n t an i n t e r e s t i n g question.
Will t h e d e c i s i o n m a k e r ' s sensitivity t o w a r d s
e v i d e n c e b e affected b y e v i d e n c e diagnosticity?
W h i l e t h e p r e c e d i n g q u e s t i o n has n o t b e e n
e x p l i c i t l y a d d r e s s e d in t h e p r i o r beliefa d j u s t m e n t m o d e l studies, available e v i d e n c e
suggests that w h e n e v i d e n c e d i a g n o s t i c i t y is
low, p e o p l e t e n d to b e m o r e sensitive to
p o s i t i v e e v i d e n c e t h a n to n e g a t i v e e v i d e n c e (cf.
K l a y m a n & Ha, 1987). F o r e x a m p l e , H o c h & Ha
( 1 9 8 6 ) h y p o t h e s i z e d a n d f o u n d that in j u d g i n g
t h e quality o f a p r o d u c t , c o n s u m e r s ' j u d g m e n t s
w e r e affected b y a d v e r t i s i n g w h e n t h e e v i d e n c e
o f p r o d u c t quality w a s l o w in diagnosticity,
whereas their judgments were unaffected by
advertising when the evidence of product
quality was high in diagnosticity. Such findings,
as t h e y argued, i n d i c a t e that p e o p l e are
m o r e easily p r o n e to e n g a g i n g in c o n f i r m a t o r y
b e h a v i o r s w h e n t h e available e v i d e n c e is
a m b i g u o u s , s i n c e t h e y t e n d to i n t e r p r e t ambig-

173

u o u s e v i d e n c e as b e i n g m o r e b e l i e f - c o n s i s t e n t ,
and t e n d to give such e v i d e n c e h i g h e r weightings
than w a r r a n t e d . Similar findings w e r e also
d o c u m e n t e d in D e i g h t o n ( 1 9 8 4 ) . In a d d i t i o n ,
in a s t u d y that i n v o l v e d j u d g i n g t h e effectiveness o f a c r i m e p r e v e n t i o n p r o g r a m , R e e d
( 1 9 8 6 ) f o u n d that s u b j e c t s r e a c t e d differently
to t h e s a m e e v i d e n c e d e p e n d i n g u p o n w h e t h e r
t h e y w e r e t o l d t h e p r o r a m h a d b e e n successful
o r u n s u c c e s s f u l in t h e past. Specifically, using
h i e r a r c h i c a l r e g r e s s i o n analysis, s h e f o u n d that
the amount of judgment variation explained by
t h e s a m e e v i d e n c e was h i g h e r for t h e successful
s u b j e c t g r o u p ( i n c r e m e n t a l R square, 3 3 % )
than for t h e failure s u b j e c t g r o u p ( i n c r e m e n t a l
R square, 15% ). In part, this e v i d e n c e suggests
that s u b j e c t s m a y h a v e r e a c t e d in a m o r e
c o n g r u o u s ( p e r h a p s s e n s i t i v e ) m a n n e r to t h e
evidence when they were prompted positively
r a t h e r t h a n negatively.
To e x p l o r e t h e effects o f e v i d e n c e diagnosticity o n b e l i e f revision, w e c o n d u c t e d an
e x p e r i m e n t ( d e s c r i b e d in t h e n e x t s e c t i o n )
involving t w o p e r f o r m a n c e a u d i t i n g tasks
characterized by high uncertainty. Our a priori
e x p e c t a t i o n w a s that t h e p e r f o r m a n c e a u d i t o r s
would display a confirmatory attitude towards
e v i d e n c e , as o p p o s e d to t h e d i s c o n f i r m a t o r y
a t t i t u d e f o u n d in t h e p r i o r a u d i t i n g studies.

RESEARCH M E T H O D

Design
A 2 2 2 e x p e r i m e n t a l d e s i g n w a s u s e d to
test t h e b e l i e f - a d j u s t m e n t m o d e l p r e d i c t i o n s .
All s u b j e c t s e x a m i n e d t w o c a s e s c e n a r i o s - - an
efficiency a u d i t a n d an effectiveness audit.
W i t h i n e a c h audit, s u b j e c t s w e r e r a n d o m l y
assigned to o n e o f t h e four b e t w e e n - s u b j e c t s
conditions. Two orders of presentation were
used: t w o p i e c e s o f p o s i t i v e e v i d e n c e f o l l o w e d

For example, the social program examined in the present study involves providing services to move the recipients of
public assistance into a self-support status, thereby reducing state welfare roles. If the output measures indicate that the
program is relatively "over-staffed" for a given fiscal year, such evidence may be easily refuted as resulting from "temlx~rary
idle capacity". Likewise, when the program shows a lack of success (e.g. a low employment rate of its recipients), such
evidence could be attributed to the local state's economy.

174

BUCK K. W. PEI et al.

TABLE 1. Cell means for sequential belief adjustments

So

S~

S2

$3

$4

$4-So

51.88
51.44

45.01
65.78

36.38
67.44

54.14
54.22

61.62
55.78

9.75
4.33

(--++)
(++--)

41.67
47.22

27.22
53.01

27.22
56.01

44.44
52.56

52.22
48.33

10.56
1.11

(--++)
(++--)

46.47
49.33

35.39
59.39

31.53
61.72

49.01
53.39

56.65
52.06

10.18
2.72

52.51
54.22

45.01
67.01

40.63
73.33

58.88
68.74

69.01
64.78

16.51
10.56

(--++)
(++--)

55.01
57.22

42.78
64.44

42.89
70.22

56.11
60.78

69.44
55.56

14.44
-1.67

(--++)
(++--)

53.83
55.72

43.82
65.72

41.82
71.78

57.41
64.72

69.24
60.17

15.41
4.44

Efficiency audit
Serial position: RETS
Order

1.
2.

(--++)
(++--)

Serial position: STER

Order

3.
4.
Overall

Effectiveness audit
Serial position: PGWR
Order

i.
2.

(--++)
(++--)

Serial position: RWGP

Order

3
4.
Overall

by two pieces of negative evidence ( + + - - ) ,

and two pieces of negative evidence followed
by two pieces of positive evidence ( - - + + ) .
The serial position of the evidence was also
manipulated. In the efficiency audit scenario,
evidence was presented to half of the subjects
in one sequence: supportive social services,
training services, employment services and
registration services (STER), with the remaining
half subjects examining the same set of evidence
in the reverse sequence (RETS). Likewise, in
the effectiveness audit scenario, half of the
subjects viewed the evidence in the following
sequence: job retention, entry wages, grand
reduction and job placement (RWGP); while
the remaining half subjects received the reverse
sequence (PGWR). Table 1 presents the above
manipulations.
It is worthwhile noting that the above serial
position manipulations were designed to assess
an important assumption of the belief-adjustment

model, i.e. the serial position effect is absent

with a short series of mixed evidence. This
assumption implies that the decision maker's
sensitivity towards evidence is constant such
that s(xk) is the same for all pieces of evidence in the series. The authors are not aware
of any prior study that has explicitly tested
this crucial assumption. With the serial position manipulations used in the present study,
the preceding assumption can be explicitly
tested.
Additionally, the present study also elicited
the subjects' subjective weightings of evidence
[i.e. s(xk)]. Specifically, upon receiving each
piece of evidence, the subjects were requested
to rate the importance of the evidence prior to
revising their beliefs. This allowed us to
examine the equality of the subjects' weightings of evidence, and whether or not the
weightings were "affected by the serial position
of evidence.

AUDITOR BELIEFREVISIONS

Subjects
T h i r t y - n i n e state p e r f o r m a n c e auditors, w h o
w e r e c u r r e n t l y taking a p e r f o r m a n c e a u d i t i n g
c o n t i n u i n g e d u c a t i o n c o u r s e at t h e state headquarters, p a r t i c i p a t e d in the study. Four subjects
failed to c o m p l e t e t h e e x p e r i m e n t , thus leaving
35 u s a b l e r e s p o n s e s . T h e s u b j e c t s w e r e e x p e r i e n c e d a u d i t o r s w h o h a d w o r k e d an average o f
five and a half y e a r s for t h e State A u d i t o r ' s
Office. T h e e x p e r i m e n t a l session t o o k a p p r o x i m a t e l y 60 minutes.

Experimental task and procedures

T h e s u b j e c t s w e r e t o l d that t h e y w e r e
r e s p o n s i b l e for d e t e r m i n i n g w h e t h e r o n e part i c u l a r state p r o g r a m - - t h e C o m m u n i t y Activat i o n for Self-Help ( C A S H ) - - w a s m a n a g i n g and
utilizing its r e s o u r c e s efficiently, and w h e t h e r
t h e d e s i r e d results a n d benefits w e r e effectively
b e i n g a c h i e v e d . 5 T h e y w e r e t o l d that t h e
efficiency o f t h e CASH p r o g r a m w o u l d b e
m e a s u r e d b y four t y p e s o f o u t p u t services:
c l i e n t registration, j o b training, e m p l o y m e n t
c o u n s e l i n g a n d s u p p o r t i v e social services;
w h e r e a s t h e p r o g r a m ' s effectiveness w o u l d b e
g a u g e d b y f o u r t y p e s o f i m p a c t m e a s u r e s o n its
clients: suitable j o b p l a c e m e n t , a d e q u a t e e n t r y
wages, a c c e p t a b l e j o b r e t e n t i o n and r e d u c t i o n
o f p u b l i c grants. T h e s u b j e c t s w e r e f u r t h e r
i n f o r m e d that e a c h o f t h e four o u t p u t m e a s u r e s
and e a c h o f t h e four i m p a c t m e a s u r e s w e r e o f
equal i m p o r t a n c e to the a c c o m p l i s h m e n t of
CASH's o v e r a l l p r o g r a m goal o f m o v i n g e m p l o y able a d u l t s into self s u p p o r t .
Subjects w e r e a s k e d to c o m p l e t e t w o tasks.
T h e y w e r e asked to c o n d u c t an efficiency
audit for o n e local CASH office (Dallas office)
and to p e r f o r m an effectiveness audit at a
s e c o n d u n r e l a t e d l o c a t i o n ( H o u s t o n office). ~"
To e n h a n c e t h e e x p e r i m e n t a l task's realism,
s u b j e c t s w e r e p r o v i d e d w i t h the n e c e s s a r y
b a c k g r o u n d i n f o r m a t i o n a b o u t b o t h offices.

175

Specifically, s u b j e c t s w e r e g i v e n a b r i e f d e s c r i p tion o f the CASH program, and details pertaining

to each t y p e of o u t p u t / i m p a c t measure. Additionally, t h e y w e r e also p r o v i d e d w i t h a t w o y e a r
( 1 9 8 6 - 1 9 8 7 ) statistical s u m m a r y o f t h e CASH
p r o g r a m ' s t a r g e t p o p u l a t i o n a b s t r a c t e d from an
actual federal e v a l u a t i o n o f t h e W o r k I n c e n t i v e
( W I N ) P r o g r a m (U. S. D e p a r t m e n t o f Labor,
1977, 1980). Finally, for e a c h local CASH office,
subjects were furnished with a brief summary
o f p r o g r a m e x p e n d i t u r e s and t h e expected
p e r f o r m a n c e targets a p p r o v e d for t h e y e a r
b e i n g a u d i t e d ( 1 9 8 7 ) . In o r d e r to h e l p s u b j e c t s
d e v e l o p a n e u t r a l initial belief, all p e r f o r m a n c e
targets p r o v i d e d w e r e b a s e d u p o n t h e national
m e a n c a l c u l a t e d for t h e v a r i o u s state W I N
p e r f o r m a n c e indicators.
After s u b j e c t s r e v i e w e d t h e b a c k g r o u n d data,
t h e y w e r e asked to p r o v i d e an initial a s s e s s m e n t
o f t h e p e r f o r m a n c e (i.e. efficiency o r effectiven e s s ) o f that p a r t i c u l a r office. Subjects w e r e
t h e n s e q u e n t i a l l y p r e s e n t e d w i t h four p i e c e s
o f m i x e d e v i d e n c e p e r t a i n i n g to t h e actual
p e r f o r m a n c e o f t h e local office. T h e d i r e c t i o n o f
e v i d e n c e was e i t h e r p o s i t i v e o r negative, and
was m a n i p u l a t e d b y using t h e values o f t h e
office's e x p e c t e d p e r f o r m a n c e (i.e. national
m e a n ) p l u s o r m i n u s t w o s t a n d a r d deviations.
F o r e a c h p i e c e o f e v i d e n c e , s u b j e c t s w e r e asked
to e x a m i n e t h e e v i d e n c e and t h e n p e r f o r m
t h r e e tasks. First, t h e y r a t e d t h e e v i d e n c e that
t h e y h a d just e x a m i n e d as e i t h e r a p o s i t i v e o r
negative i n d i c a t i o n o f a g e n c y p e r f o r m a n c e .
Subjects t h e n assigned a s u b j e c t i v e w e i g h t to
t h e e x a m i n e d evidence" o n a scale ranging from
0 ( v e r y w e a k e v i d e n c e ) to 100 ( v e r y s t r o n g
e v i d e n c e ) . Finally, s u b j e c t s c o n s i d e r e d all the
e v i d e n c e that t h e y h a d r e c e i v e d thus far o n that
p a r t i c u l a r local office a n d i n d i c a t e d t h e i r b e l i e f
on a scale r a n g i n g f r o m 0 ( e x t r e m e l y u n l i k e l y )
to 100 ( e x t r e m e l y l i k e l y ) as to w h e t h e r the
local office p e r f o r m a n c e was efficient o r effective.

The CASH program was established to move recipients of public assistance into productive unsubsidized jobs, thereby
increasing their self-sufficiency and reducing the state weffare roles.
6 Half of the subjects were randomly assigned to perfnrm the efficiency audit, followed by the effectiveness audit, whereas
the remaining half of the subjects were assigned to perform the tasks in the reverse order. The order of cases performed had
no effect on the subjects' performance.

176

BUCK K. W. PEI e t al.

ANALYSIS AND RESULTS

M a n i p u l a t i o n check
T h e following manipulation c h e c k s w e r e
c o n d u c t e d . First, w e e x a m i n e d the e n c o d i n g o f
e v i d e n c e with r e s p e c t to its direction. It was
f o u n d that, o f 280 responses ( 3 5 subjects 2
senarios x 4 cues), only o n e r e s p o n s e involved
c o d i n g errors. Also, by e x a m i n i n g the subjects'
responses o n belief revision u p o n receiving
each piece o f evidence, it was f o u n d that only
ten ( o u t o f 2 8 0 ) responses involved c o d i n g
errors (i.e. an u p w a r d / d o w n w a r d revision u p o n
receiving negative/positive evidence, respectively). Also, the errors w e r e not disproportionally distributed across different t r e a t m e n t
c o n d i t i o n s for the eight pieces o f evidence.
Based u p o n the p r e c e d i n g results, w e view the
manipulation of positive and negative e v i d e n c e
as successful and the c o d i n g errors as random.
Second, using the subjects' subjective weightings o f e v i d e n c e as the d e p e n d e n t variable, w e
e x a m i n e d the equality of the subjects' weightings of evidence. T w o separate MANOVA
p r o c e d u r e s w e r e p e r f o r m e d to e x a m i n e the
equality o f e v i d e n c e weightings a m o n g the
four types o f e v i d e n c e used in each task. The
results s h o w e d that the null hypothesis of equal
e v i d e n c e weightings c o u l d not be r e j e c t e d (for
the effectiveness audit, F = .747, p = .533; for
the efficiency audit, F = .658, p = .584). That is,
for b o t h tasks, subjects, on average, v i e w e d the
four types of e v i d e n c e as equally important.
Also p e r f o r m e d w e r e eight (2 tasks x 4 types
of evidence used in each task)separate individual
ANOVA procedures. The d e p e n d e n t variable
was, again, the subjects' subjective weightings
o f evidence. The i n d e p e n d e n t variables w e r e
serial position (earlier in the s e q u e n c e vs
later in the s e q u e n c e ) and e v i d e n c e direction
(positive vs negative). Note, the focus o f the
analysis was to d e t e r m i n e w h e t h e r the subjec-

tive weightings o f each type o f evidence

differed across the four t r e a t m e n t conditions.
The results s h o w e d that, for each type o f
e v i d e n c e in b o t h tasks, the serial position effect
was n o t significant (for efficiency audit .416 ~<
p s <<- .868; for effectiveness audit .421 ~< p s <~
.868). Further, in all cases, the interaction
b e t w e e n serial position and e v i d e n c e direction
was n o t significant. This suggested that the
serial position manipulations had no effect on
the subjects' weightings o f evidence. That is,
the subjective weightings of e v i d e n c e w e r e
unaffected w h e t h e r a p i e c e o f e v i d e n c e (e.g.
registration) was p r e s e n t e d earlier in the
s e q u e n c e (e.g. in a RETS s e q u e n c e ) or later in
the s e q u e n c e (e.g. in a STER s e q u e n c e ) .
With r e s p e c t to the e v i d e n c e direction effect,
the p r e c e d i n g ANOVAs results are s u m m a r i z e d
in Table 2. In this table, for each type of
evidence, the average cell means of subjective
c u e weightings for positive ( a n d negative)
e v i d e n c e are presented. Also p r e s e n t e d are the
c o r r e s p o n d i n g F statistic and P value to indicate
the equality o f means b e t w e e n the positive and
negative means. As s h o w n in this table, for each
o f the four types o f e v i d e n c e in the efficiency
audit (i.e. registration, e m p l o y m e n t , service and
training), the effect was not significant (. 108
p s <~ .734). That is, w h e t h e r a p i e c e o f e v i d e n c e
(e.g. registration) was manipulated as positive
o r negative did not affect its subjective weightings. In contrast, in the effectiveness audit, the
e v i d e n c e direction effect was significant for
three types of e v i d e n c e ( j o b placement, grand
reduction, job r e t e n t i o n ) (.000 ~< p s <~ .036).
An examination o f cell means revealed that the
subjective weightings of e v i d e n c e w e r e higher
w h e n the e v i d e n c e was manipulated as positive.
For the purpose of testing the model's predicted
r e c e n c y effect, an unequal subjective weighting
b e t w e e n positive and negative e v i d e n c e does
not suggest a manipulation failure. 7

7 The belief-adjustment model defines recency effects as D = --ti~$(x )s(x+). Therefl)re, re c e nc y effects should always
prevail, regardless of a lack of c o m p l e t e symmetry b e t w e e n s ( x ) and s(x+). Tubbs et al.s' ( 1 9 9 0 ) findings also
( e x p e r i m e n t s 3 and 4 ) indicate a lack of equality in subjective weightings b e t w e e n l~)sitive and negative evidence, which
does not affect the p r e d i c t e d recency effects. A later analysis (s e e footnote 9) also seems to suggest that this inequality of
subjective weig htings does not confi)und the recency effects.

177

AUDITOR BELIEF REVISIONS

TABLE 2. Cell means and partial ANOVA table for subjective cue weightings
Panel A: Efficiency audit

Dependent variables
Registration (R)
Employment (E)
Service (S)
Training ( T )

Direction of evidence*t
Average cell means
Main effect
Positive
Negative
F
67.83
(22.31)
65.39
(20.57)
70.59
(23.32)
64.41

56.47
(17.12)
52.94
(25.73)
61.28
(14.46)
61.83

(21.54)

(17.41)

2.741

.108

2.462

. i 27

1.921

.176

0.118

.734

Panel B: Effectiveness audit

Dependendent variables
Placement (P)
Grant ( G )
Wage ( W )
Retention (R)

Direction of evidence*$
Cell means
Main effect
Positive
Negative
F
76.22
(17.74)
79.00
(17.96)
69.71
(19.99)
72.06
(16.63)

49.71
(14.25)
40.88
(14.15)
61.00
(21.87)
56.33
(20.67)

20.919

.000

41.783

.000

1.676

.205

4.803

.036

* The cell means for serial position from the ANOVA analyses were not presented because, in each of the four ANOVAs, this
manipulation was not significant to the dependent variable ( t .416 ~< ps <~ .868; ~: .421 <~ ps ~ .982).
Standard deviations are in parentheses.

Finally, i n t e r s u b j e c t reliabilities also w e r e

c o m p u t e d using W i n e r ' s r I statistic ( W i n e r ,
1971, p. 289). Subjects e x h i b i t e d fairly l o w
reliability for b o t h tasks ( r l -- .29 a n d .23,
r e s p e c t i v e l y , for t h e efficiency a u d i t and t h e
effectiveness audit). Also c o m p u t e d w e r e t h e
e v i d e n c e - t o - e v i d e n c e v a r i a n c e in e a c h s u b j e c t ' s
ratings for e a c h a u d i t task. A p a i r e d t-test
i n d i c a t e d that t h e v a r i a n c e in ratings o f t h e
e v i d e n c e u s e d in t h e efficiency a u d i t ( m e a n =
2 3 8 ) w a s n o t different from t h e v a r i a n c e in
ratings o f t h e e v i d e n c e u s e d in t h e effectiveness
a u d i t ( m e a n = 280, t = .64, d.f. = 34). T h e s e
results s e e m to c o n f i r m o u r e x p e c t a t i o n s that
t h e p e r f o r m a n c e a u d i t i n g tasks u s e d in the
p r e s e n t study m a y b e subject to high uncertainty
d u e to t h e l o w d i a g n o s t i c i t y o f t h e available
e v i d e n c e in t h e d e c i s i o n tasks.

Recency effects
R e c e n c y effects p r e d i c t e d b y t h e m o d e l w e r e
t e s t e d using a 2 x 2 x 2 ANOVA w i t h o r d e r

o f e v i d e n c e a n d serial p o s i t i o n as b e t w e e n s u b j e c t s factors, a n d tasks (efficiency and

effectiveness a u d i t s ) as a w i t h i n - s u b j e c t factor.
T h e d e p e n d e n t variable was t h e size o f t h e
b e l i e f u p d a t i n g from t h e initial a n c h o r ($4 - S o ) .
Results i n d i c a t e d that o n l y t h e main effect for
p r e s e n t a t i o n o r d e r was significant ( F = 6.69,
p = .015). T h e r e w e r e no significant main effects
for task ( F -- 1 . 2 5 , p = : 6 4 5 ) a n d serial p o s i t i o n
( F = 1.37, p = .557). F u r t h e r individual
ANOVA analyses for e a c h task p r o d u c e d s o m e w h a t similar results. For t h e efficiency audit, the
m a i n effect for o r d e r w a s n o t significant,
a l t h o u g h it w a s in t h e p r e d i c t e d d i r e c t i o n ( F =
2.12, p = .150). T h e effectiveness audit had a
significant m a i n effect for o r d e r ( F = 6.67, p =
.015). Overall, t h e r e c e n c y effect w a s m o r e
p r o n o u n c e d in t h e effectiveness audit. In all t h e
a b o v e analyses, serial p o s i t i o n effects w e r e n o t
significant ( f o r efficiency a u d i t F = .061, p =
.807; for effectiveness a u d i t F = 2.83, p ----.120),
n o r w e r e t h e i n t e r a c t i o n t e r m s ( f o r efficiency

BUCK K. W. PEI et al.

178

TABLE 3. Intermediate belief changes due to positive vs negative cues

Panel A: Cell m e a n s (S.D.s)
Efficiency audit

Serial position: RETS

Order
1.
(----++)
2.

(++--)

Serial position: STER

Order
3.
(----++)

4.

(++--)

Effectiveness audit
Serial position: PGWR
Order
l.
(--++)

2.

(++--)

Serial position: RWGP

Order"
3.
(--++)

4.

(++--)

ABS(S2-So)

ABS(S4-S2)

15.50
(13.461
16.0{I
(9.43)

25.25
(18.32)
11.52
(12.85)

14.44
(19.91)
8.78
(14.39)

25.00
(24.11)
7.67
(11.57)

11.88
(10.02)
19.11
11.25)

28.38
(16.11)
8.56
(11.21)

12.11
11.57)
13.00
10.35)

26.56
(18.48)
14.67
(13.45)

Panel B: Partial s u m m a r y of repeated measure ANOVA results

Efficiency audit
F
p

Effectiveness audit
F
p

Between-subjects effects
Order ( O )
Serial Position (SP)
O x SP

2.68
.32
.20

.112
.574
.656

1.59
.01
.01

.217
.933
.933

Within-subjects effects
[$2-So) vs [S~-$2] (DP)
O x DP
SP x DP
O SP DP

2.08
6.23
.15
.2{I

.160
.018
.699
.656

6.48
21.12
1.38
2.71

.016
.I1011
.250
.ll6

audit F = . 152, p = .699; for e f f e c t i v e n e s s audit

F = 1.37, p = .250).
Table 1 p r e s e n t s the resulting cell m e a n s
from the p r e c e d i n g analyses. T h e pattern o f the
cell m e a n s for b o t h tasks s h o w s that w i t h i n the
s a m e order, the size o f [$4-So] is virtually
identical (e.g. c o n d i t i o n s 1 vs 3 or 2 vs 4),
regardless o f alternative serial positions. T h e
pattern also clearly reflects the r e c e n c y effects
p r e d i c t e d by the belief-adjustment m o d e l .
The cell m e a n s in Table 1 further indicate

that the final p o s i t i o n s r e a c h e d for both tasks in

nearly all c o n d i t i o n s ( e x c e p t for c o n d i t i o n 4 o f
the e f f e c t i v e n e s s a u d i t ) are h i g h e r than the
initial anchors. This suggests that, overall, the
subjects' b e l i e f revisions are m o r e affected by
p o s i t i v e e v i d e n c e than n e g a t i v e e v i d e n c e . This
is an interesting finding in light o f the o p p o s i t e
results obtained in t w o recent studies c o n d u c t e d
in financial audit contexts. A s h t o n & Ashton
( 1 9 8 8 ) f o u n d that external auditors p r e d o m i n antly displayed a negative b e l i e f c h a n g e w h e n

AUDITORBELIEFREVISIONS
confronted with m i x e d e v i d e n c e in two different
tasks ( e x p e r i m e n t s 2A a n d 2B). Similar results
w e r e also f o u n d b y T u b b et al. ( 1 9 9 0 ) u s i n g
t w o o t h e r audit tasks.
Panel A of Table 3 displays the cell m e a n s of
the relative b e l i e f c h a n g e s at t w o - c u e intervals
(i.e. [$2---So] a n d [$4--$2]) across the four treatm e n t c o n d i t i o n s in each task. T h e p a t t e r n of
cell m e a n s reveals that, in b o t h tasks a n d u n d e r
the ( - - + + )
c o n d i t i o n s , the m a g n i t u d e of
b e l i e f revisions from [$4--$2] is n o t a b l y higher
than that of belief revisions from [$2-So] order.
In contrast, n o s u c h a p a t t e r n prevails u n d e r the
(++--)
c o n d i t i o n s for either task.
Recall that the belief-adjustment model predicts
a contrast effect o n belief revision. That is, w h e n
the p r e c e d i n g a n c h o r is l o w ( h i g h ) , positive
e v i d e n c e will result in a greater ( s m a l l e r )
u p w a r d belief revision; w h e r e a s negative
e v i d e n c e will result in a smaller ( g r e a t e r )
d o w n w a r d b e l i e f revision. This implies that,
u n d e r the ( - - + + ) c o n d i t i o n , the relative size
of [S4-Sz] s h o u l d b e larger than the relative size
of [$2-So] b e c a u s e $2 ( t h e a n c h o r of [$4-$2])
is l o w e r t h a n So ( t h e a n c h o r of [$2-So]), ceteris
paribus. Likewise, u n d e r the ( + + - - )
conditions, the relative size of [$4-$2], s h o u l d b e
greater t h a n the relative size of [$2-So] because the a n c h o r of [$4-$2], $2, is higher
than the a n c h o r of [$2-So], So, ceteris paribus.
T h e p a t t e r n of cell m e a n s in p a n e l A, Table
3, n o n e t h e l e s s , reveals that the p r e c e d i n g
c o n t r a s t effect p r e d i c t i o n s prevail o n l y u n d e r
the ( - - + + )
c o n d i t i o n , b u t n o t u n d e r the
(++--)
c o n d i t i o n . To e x a m i n e this observation, the subjects' i n t e r m e d i a t e belief c h a n g e s
w e r e analysed.

179

Intermediate belief changes

The subjects' intermediate belief changes
w e r e analysed b y t w o separate 2 2 r e p e a t e d
m e a s u r e ANOVA p r o c e d u r e s , o n e for each audit
task. Specifically, the d e p e n d e n t variables w e r e
the relative size of b e l i e f revisions r e s u l t i n g
from t w o c o n s i s t e n t p i e c e s of e v i d e n c e , i.e. [SzSo] a n d [$4-$2]. The t w o b e t w e e n - s u b j e c t s
i n d e p e n d e n t variables w e r e o r d e r ( [ - - + + ] vs
[++--])
a n d serial p o s i t i o n ([RETS vs STER]
for the efficiency audit; [PGWR] vs [RWGP] for
the effectiveness audit).
T h e results in p a n e l B, Table 3, s h o w e d that,
for the efficiency audit, t h e r e was a significant
i n t e r a c t i o n effect b e t w e e n the d e p e n d e n t variables (i.e. [$2-So] a n d [$4-$2]) a n d o r d e r ( F =
6.23, p = .018). Specifically, regardless of
alternative serial positions, the p a t t e r n of the
cell m e a n s s h o w e d that b e t w e e n [$2--So] and
[$4-$2] the m a g n i t u d e of belief revisions was
the same u n d e r the ( + + - - )
condition, but
was different u n d e r the ( - - + + )
condition.
O r d e r effect a l o n e was n o t significant ( F = 2.68,
p = .112). I n addition, n e i t h e r the serial
p o s i t i o n effects ( F = .32, p = . 5 7 4 ) n o r the
i n t e r a c t i o n t e r m b e t w e e n o r d e r a n d serial
p o s i t i o n ( F = .20, p = . 6 5 6 ) w a s significant. For
the effectiveness audit, similar results w e r e
obtained throughout. 8
T h e a b o v e results confirm the o b s e r v a t i o n
that in b o t h audit tasks, the c o n t r a s t effects
prevail only u n d e r the ( - - + + ) c o n d i t i o n , b u t
n o t u n d e r the ( + + - - )
c o n d i t i o n . That is,
w h i l e positive evidence induces larger belief
revisions w h e n the a n c h o r is low, negative
e v i d e n c e fails to i n d u c e larger belief revisions
w h e n the a n c h o r is high. Note that this is true in

8 Additionally, we ran a set of paired t-tests. For the efficiency/effectiveness audit, the results showed that, under the
( - - + + ) condition, the combined average cell mean of [$4 - $2] for the two serial positions was greater than that of the
[$2-So](for efficiency audit t = 3.09,p = .007; for effectiveness audit t = 4.66,p ~<.000). In contrast, under the ( + + - - )
condition and in both tasks, the results revealed no such differences (for efficiency audit t = .73,p = .477; for effectiveness
audit t = 1.47, p = .160).

BUCK K. W. PEI et aL

180

b o t h a l t e r n a t i v e serial p o s i t i o n s , s u g g e s t i n g
that the results cannot be caused by the
d i f f e r e n c e s in e v i d e n c e d u e to serial p o s i t i o n
manipulations. 9
O n e p l a u s i b l e e x p l a n a t i o n is a l a c k o f s y m metrical sensitivity towards evidence. This can
be illustrated by examining the analytical
implications of the belief-adjustment model. To
s e e h o w , t h e r e l a t i v e size o f b e l i e f r e v i s i o n s f o r
[S,I--S2] a n d [S2---So] u n d e r t h e ( + + - - )
condit i o n c a n b e w r i t t e n , r e s p e c t i v e l y , as f o l l o w s
( s e e A p p e n d i x for d e r i v a t i o n ) :
[$4-6"2] = S2 ( 2 ~ - ( / 2 )
a n d [$2-So] = ( 1 - S o ) ( 2 1 3 -

[32).

G i v e n t h e c o n t r a s t effect, o n e is e x p e c t e d t o
o b s e r v e [$4-6"2] 2> [$2-So]. T h i s n e c e s s i t a t e s
$2 2> [ 1 - St, ] a n d / o r ( 2 { z - ~t2) 2> ( 2 [3- [32 ). N o n e t h e l e s s , o u r findings s h o w that, for b o t h a u d i t
tasks, t h e c o n t r a s t e f f e c t is absent u n d e r this
c o n d i t i o n (i.e. [84--82] = [82--80]). F u r t h e r , t h e
a v e r a g e c e l l m e a n s f r o m T a b l e 1 s h o w s t h a t $2
2> [ l~S'o] ( f o r e f f i c i e n c y audit, $2 = 6 2 % a n d So
~- 4 9 % ; for e f f e c t i v e n e s s audit, $2 -- 71% a n d ~'o
-- 5 5 % ) . T h i s i m p l i e s t h a t t h e a b s e n c e o f t h e
c o n t r a s t e f f e c t m u s t b e d u e to (2~t - c t 2) <
( 2 ] 3 - ]32), o r s i m p l y 13 2> ~.
Likewise, under the ( - - + + ) condition (see
A p p e n d i x for d e r i v a t i o n ) :
[$4--,S'21 = ( 1 -S2X2~J - ]32)
a n d [S2-So] = So ( 2 ~ - ~2).
Again, g i v e n t h e c o n t r a s t effect, o n e is e x p e c t e d
to o b s e r v e [$4-$2] 2> 1S2-So]. T h i s n e c e s s i t a t e s
(1 - $2) 2> St, a n d / o r (2~3 - ]32) 2> (2~t -or2).

O u r findings s h o w t h a t t h e c o n t r a s t e f f e c t is

present, a n d t h a t t h e a v e r a g e c e l l m e a n s f r o m
T a b l e 1 s h o w ( 1 - - 8 2 ) 2> So ( f o r e f f i c i e n c y
audit, $2 = 32% a n d So -- 4 6 % ; f o r e f f e c t i v e n e s s
audit, $2 = 4 2 % a n d So = 54% ). T h i s i m p l i e s
that t h e p r e s e n c e o f t h e o b s e r v e d c o n t r a s t
e f f e c t c a n b e c a u s e d b y e i t h e r (1 - $ 2 ) > So, o r
(2]3 - ~2) > (2Gt --ct2), o r b o t h .
Jointly, the preceding data analyses and
analytical derivations imply that the subjects
in this s t u d y e x h i b i t a c o n f i r m a t o r y a t t i t u d e
t o w a r d s e v i d e n c e , at least in t h e ( + + - - )
c o n d i t i o n s . In h i n d s i g h t , w e t h i n k this p h e n o m e n o n s e e m s to r e f l e c t t h e p o s s i b i l i t y t h a t
in m o s t s o c i a l p e r f o r m a n c e audits, o b s e r v i n g
n e g a t i v e r e s u l t s is t y p i c a l r a t h e r t h a n atypical;
t h e r e f o r e , s t a t e a u d i t o r s m a y n o t " o v e r e a c t " to
s u c h results. C o n v e r s e l y , o b s e r v i n g p o s i t i v e
r e s u l t s is a t y p i c a l r a t h e r t h a n t y p i c a l ; t h e r e f o r e
s t a t e a u d i t o r s m a y i n c o r p o r a t e this " s u r p r i s e "
i n t o t h e b e l i e f r e v i s i o n . T o this e n d , a r e c e n t
U.S. d o c u m e n t o n Nonprofit Entities" Performance Measures (AAA, 1 9 8 9 ) s u g g e s t s t h a t it is
difficult to u n a m b i g u o u s l y m e a s u r e t h e i m p a c t s
o f social p r o g r a m s . A c c o r d i n g l y , t h e p e r f o r m a n c e
m e a s u r e s o f s o c i a l p r o g r a m s s h o u l d reinforce

the positive impacts a n d m i n i m i z e the negative impacts o f t h e s e p r o g r a m s ' i n t e r v e n t i o n s .

F u r t h e r , any m e a s u r e o f a s o c i a l p r o g r a m t h a t is
capable of estimating the program's social
benefits should have greater impact on decision
m a k e r s t h a n w i l l a m e a s u r e t h a t is i n c a p a b l e of
g a u g i n g s u c h effects. O n this l a t t e r p o i n t , t h e
r e p o r t c o n t e n d s t h a t d e c i s i o n m a k e r s are
c o n s t a n t l y s e a r c h i n g for p o s i t i v e r e a s o n s w h y
s c a r e r e s o u r c e s s h o u l d b e c o n t i n u e d for a
particular program.

'~To rule out the possibility that the preceding results may be confounded by the unequal subjective weightings between
positive and negative evidence (found in the effectiveness audit earlier by the manipulation check), two expost tests were
performed. For each audit task, a 2 (order) 2 (serial position) ANOVA procedure was performed, using the relative size
of [$2-So] as the dependent variable. The results showed that order was not significant (for efficiency audit F = .207, p =
.653; for effectiveness audit F = .577, p = .453), neither was serial position (for efficiency audit F = .516, p = ,478; for
effectiveness audit F = .337, p = .566) or the interaction term (for efficiency audit F = .274, p = .807; for effectiveness
audit F = .436, p = .728). The absence of significant order effects, in both tasks, suggested that the subjects' belief revisions
for the same pieces of evidence were not "affected by the direction of evidence. Accordingly, the unequal subjective
weightings of evidence flmnd earlier in the effectiveness audit did not appear to be confounding.

AUDITORBELIEFREVISIONS
DISCUSSION AND CONCLUDING REMARKS
Prior to discussing the results, a s u m m a r y of
the p r e s e n t study's findings is provided. Consistent with the belief-adjustment model's prediction, p e r f o r m a n c e auditors' belief revisions
w e r e subject to r e c e n c y effects, although the
p r e d i c t e d effects w e r e m o s t salient in the
effectiveness audit. The results also validated an
important assumption of the belief-adjustment
model. That is, for a relatively short e v i d e n c e
series, the subjects' belief revisions w e r e not
affected by the serial position of e v i d e n c e (i.e.
s(xk) is the same for all pieces o f e v i d e n c e in
the series). Further, in b o t h tasks, the subjects'
intermediate belief changes reflect that the
contrast effects prevail only under the ( - - + + )
condition, but not u n d e r the ( + + - - )
condition. This suggests that the subjects' sensitivity
towards e v i d e n c e may be higher for positive
e v i d e n c e than for negative evidence.
Finding the p r e s e n c e o f r e c e n c y effects is
consistent with the prior studies c o n d u c t e d in
the financial auditing contexts. The present
findings, nonetheless, e x t e n d the m o d e l ' s predictions to a different task e n v i r o n m e n t (i.e.
p e r f o r m a n c e auditing).
Finding the absence o f serial position effects
is important b e c a u s e prior literature suggests
that a decision maker's sensitivity may not be
constant due to an attention d e c r e m e n t effect
(cf. Hogarth & Einhorn, 1990). While the
belief-adjustment m o d e l assumes this p h e n o m e n o n is unlikely to o c c u r for a short e v i d e n c e
series, this assumption has not b e e n explicitly
tested in the literature. Consistent with the
model's assumption, no serial position effects
were found throughout the analyses. Specifically,
the subjects' subjective weightings of evidence
w e r e unaffected by the serial position manipulations in b o t h tasks. In addition, with respect to
belief revisions, the manipulations had no effect
on the magnitude o f the overall revision (i.e.
[$4-So]. This suggests that the model's predicted
r e c e n c y effects are d u e to the a n c h o r i n g and
contrast implications of the m o d e l rather than
serial position effects.
Findings on the subjects' intermediate belief

181

revisions s h o w s o m e preliminary e v i d e n c e that

auditors in a p e r f o r m a n c e auditing c o n t e x t m a y
be m o r e sensitive to positive e v i d e n c e than
negative evidence. This is in line with the
findings o n the subjects' overall belief revisions,
w h i c h reveal that their belief revisions are m o r e
affected b y positive e v i d e n c e than negative
evidence. Based o n the p r e c e d i n g results, w e
c o n j e c t u r e that subjects in o u r study display a
confirmatory attitude t o w a r d s evidence. This
conjecture is further supported by prior evidence
that w h e n e v i d e n c e diagnosticity is low, the
decision m a k e r will be m o r e affected by
positive evidence. Alternatively, an observation
by Ashton & Ashton ( 1 9 8 8 ) also seems to
s u p p o r t o u r conjecture. T h e y argued that
the disconfirmatory attitude t o w a r d s e v i d e n c e
displayed by the auditor subjects may be
attributed to the subjects' p e r c e p t i o n s of
the decision c o n s e q u e n c e s , i.e. potential legal
liabilities for failing to act conservatively.
Comparatively, the p e r f o r m a n c e auditors of the
p r e s e n t study m a y have b e e n less sensitive to
the c o n s e q u e n c e s o f legal liabilities, and hence,
displayed a confirmatory attitude in belief
revision.
Unfortunately, w e c a n n o t u n a m b i g u o u s l y
argue w h e t h e r o r not the study's findings w e r e
the result o f task uncertainty, o r the absence of
legal liabilities, or both. Such a distinction
w o u l d have b e e n possible had our subjects
p e r c e i v e d a differential a m o u n t o f task uncertainty b e t w e e n the effectiveness audit and the
efficiency audit. For b o t h audit tasks, the
manipulation checks s h o w e d that the intersubject reliability ratings of e v i d e n c e w e r e
fairly low, suggesting high task uncertainty;
although intuition and available arguments
in the p e r f o r m a n c e auditing literature (e.g.
Herbert, 1 9 7 9 ) s e e m e d to suggest that the task
uncertainty related to an effectiveness audit
m a y be higher than that of an efficiency audit.
Accordingly, o n e fruitful area o f future research
w o u l d be to investigate w h e t h e r or not the
decision maker's attitude t o w a r d s e v i d e n c e is
driven by task uncertainty, the p e r c e i v e d
decision c o n s e q u e n c e s , or b o t h o f these factors.
A n o t h e r related area for future research is to

182

BUCK K. W. PEI et aL

explore whether
other dimensions of task
uncertainty and decision consequences (and of
d i f f e r e n t m a g n i t u d e s ) affect t h e m o d e l ' s r e c e n c y
effects prediction. In the present study, the
s c o p e o f t h e i n v e s t i g a t i o n is l i m i t e d t o o n l y
o n e a s p e c t o f e a c h v a r i a b l e (i.e. f o r t a s k
u n c e r t a i n t y , e v i d e n c e d i a g n o s t i c i t y ; for d e c i s i o n
consequences,
legal liabilities). Thus, other
a s p e c t s o f t a s k u n c e r t a i n t y (e.g. d e g r e e s o f
t a s k s t r u c t u r e , cf. A b d o l m o h a m m a d i
& Wright
1 9 8 7 ) a n d d e c i s i o n c o n s e q u e n c e s (e.g. differential r e w a r d / p e n a l t y s t r u c t u r e s ) a l s o c o u l d b e
examined. To this end, we wish to point out
that by failing to explicitly manipulate the
d e c i s i o n c o n s e q u e n c e s v a r i a b l e ( e . g . via a l t e r native penalty structures), this study's results

on the state auditors' sensitivity towards evidence

remain tentative.
Finally, t h e p r e s e n t s t u d y ' s f i n d i n g s c o u l d h a v e
potential implications on performance auditing of
social p r o g r a m s . T h e r e s u l t s o f a p e r f o r m a n c e
audit provide an important input to the political
process of resource allocation. Clearly, the
amount of task uncertainty associated with
some social programs (e.g. drug abuse prevent i o n ) m a y b e h i g h e r t h a n t h a t o f o t h e r s (e.g.
s t u d e n t l o a n s ) . If t h e p e r f o r m a n c e a u d i t o r s a r e
c o n f i r m a t i o n - p r o n e in w e i g h i n g e v i d e n c e a n d
t h e i r m a g n i t u d e t e n d s t o increa~sc p r o p o r t i o n a l l y
as t h e t a s k u n c e r t a i n t y i n c r e a s e s , t h e n t h e
resulting resource allocations also could be
s y s t e m a t i c a l l y a f f e c t e d in a s i m i l a r m a n n e r .

BIBLIOGRAPHY
Abdolmohammadi, M. & Wright, A., An Examination of the Effects of Experience and Task Complexity on
Audit Judgments, The Accounting Review (January 1987) pp. 1-14.
American Accounting Association, Committee Report on Nonprofit Entities" Performance Measures,
Measuring the Performance of Nonprofit Organizations: the State of Art. AAA, Government and
Nonprnfit Section, Sarasota, FL 34233 (1989).
Asare, S. K., The Auditors' Going Concern Opinion Decision: Interaction of Task Variable and the
Sequential Processing of Evidence, Working paper, Fisher School of Accounting, University of Florida
(April 1990).
Asare, S. K. & Messier, W., A Review of Audit Research Using "lqae Belief-Adjustment Model, Working
paper, Fisher School of Accounting, University of Florida (May 1990).
Ashton, A. H. & Ashtnn, R. H., Sequential Belief Revision in Auditing, The Accounting Review (October
1988) pp. 623--641.
Deighton, J., The Interaction of Advertising and Evidence, Journal o f Consumer Research (December
1984) pp. 763-770.
Einhorn, H. J. & Hogarth, R. M., Ambiguity and Uncertainty in Prnbabilistic Inference, Psychol~Rical
Review (October 1985a) pp. 433~461.
Einhorn, H. J. & Hogarth, R. M., Judging Probable Cause, Psychological Bulletin (1986) pp. 3-19.
Einhorn, H. J. & Hogarth, R. M., Adaptation and Inertia in Belief Updating: The Contrast-Inertia Model,
Wurking Paper, Center for l)ecision Research, University of Chicago, October (1987).
Fischhoff, B. & Beyth-Marom, R., Hypothesis Evaluation from a Bayesian Perspective, Psychological
Revieu, (July 1983) pp. 239-260.
Gambino, A. H. & Reardon, T. J., Financial Planning and Evaluation f o r the Non-Profit Organization
(New York: National Association of Accountants, 1981 ).
Herbert, L., Auditing the Performance o f Management (Belmont: Lifetime l.earning, 1979 ).
Hoch, S.J. & Ha, Y-W., Consumer Learning: Advertising and the Ambiguity of Product Experience,Journal
o f Consumer Research ( September 1986) pp. 221-233.
Hogarth, R. M , J u d g m e n t and Choice, 2nd Edn (New York: Wiley 1987).
Hogarth, R. M. & Einhorn, H. J., Order Effects in Belief Updating: The Belief-Adjustment Model, Working
paper, Ccnter fi)r Dccisinn Rcsearch, University of Chicago (May 1990).
Kahneman, D. & Tversky, A., Prospect Theory: An Analysis of Decision Under Risk, Econometrica (March
1979) pp. 2263--2291.
Klayman, J. & tla, Y-W., Confirmation l)isconfirmation, and Infi)rmation in Hypothesis Testing,
Psychological Revieu, (April 1987) pp. 211-228.

A U D I T O R BELIEF REVISIONS

183

N a c h m i a s , D., P u b l i c P o l i c y E v a l u a t i o n : A p p r o a c h e s a n d M e t h o d s ( N e w York: St. Martin's Press, 1979).

Nisbett, R. & Ross, L., H u m a n Inference: Strategies a n d S h o r t c o m i n g s o f H u m a n J u d g m e n t ( E n g l e w o o d
Cliffs, NJ: Prentice-Hall, 1980).
Payne, J. W., C o n t i n g e n t D e c i s i o n Behavior, P s y c h o l o g i c a l B u l l e t i n ( 1 9 8 2 ) pp. 3 8 2 - 4 0 2 .
Payne, J. W., B e t t m a n , J. R. & J o h n s o n , E. J., T h e A d a p t i v e D e c i s i o n Makers: Effort a n d A c c u r a c y in C h o i c e ,
in H o g a r t h , R. M. ( e d . ) , I n s i g h t s in D e c i s i o n M a k i n g ( C h i c a g o : U n i v e r s i t y o f C h i c a g o Press, in press).
Reed, S. A., T h e I m p a c t o f N o n m o n e t a r y P e r f o r m a n c e M e a s u r e s U p o n B u d g e t a r y D e c i s i o n Making in the
Public Sector, J o u r n a l o f A c c o u n t i n g a n d P u b l i c P o l i c y ( S u m m e r 1 9 8 6 ) pp. 1 1 1 - 1 4 0 .
Rossi, P. H., F r e e m a n , H. E. & Wright, S. R., E v a l u a t i o n : A S y s t e m a t i c A p p r o a c h ( B e v e r l y Hills: Sage,
1979).
T u b b s , R. M., Messier, W. F. & K n e c h e l , W. R., R e c e n c y Effects in t h e Auditors' Belief-Revision Process, The
A c c o u n t i n g R e v i e w (April 1 9 9 0 ) pp. 4 5 2 - 4 6 0 .
U. S. D e p a r t m e n t of Labor, The W o r k I n c e n t i v e ( W I N ) P r o g r a m a n d R e l a t e d Experience, R & D
Monograph 49 (1977).
U. S. D e p a r t m e n t o f Labor, I m p l e m e n t i n g W e l f a r e - E m p l o y m e n t Programs: A n I n s t i t u t i o n a l A n a l y s i s o f
the W o r k I n c e n t i v e ( W I N ) P r o g r a m , R & D M o n o g r a p h 78 ( 1 9 8 0 ) .
Winer, B. J., S t a t i s t i c a l P r i n c i p l e s in E x p e r i m e n t a l D e s i g n ( N e w York: McGraw-Hill, 1971 ).

APPENDIX

Under the ( + + - - )
condition, the relative size of belief
revisions for positive evidence, [$2-So], can be derived as
follows:
letSt
-(;2

So+~(I-So)S(xl

(A1)

,$'1 + [3( i -- SI )S(x2).

(A2)

Without losing generalizabilit3,, assume s(x~ ) = s(xz) and lx)th

are equal to a constant. By substituting (A1) into (A2),

$2

{So + ~( l -- So)} + [~{ 1 - [So + ~( l - S o ) l }

So + [3(1 -- So) + [~

So + [~{(1 -- St,) + (1 -- So) -- ~(1 -- So)}

= so+(1
Thus,

sz-so

(1

-so)(2~-

,s',,) (213 -

Correspondingly, u n d e r the ( - - + + ) condition, the relative

size of belief revision for negative evidence, [$2-So], can be
derived as follows:
letSt

So ~(;(~s'(X 1)
= $1 - ~$'ls(x2).

$2

(A6)

Further, let s ( x t ) = s(x2), and both equal to a constant. By

substituting ( A 5 ) into (A6),
S2

~,~'o -- ]32(1 -- S0)

Thus, $2-So

~)

(AS)

(S , ) -

So - o3"o - m~'o+ al,','o.

~%)

u(,%

~So)

-So(2Ct - u2), or 1S2 - So] = So(2(t - ctz).

The relative size of belief revis ons fur positive evidence, [$4$2], can be derived ms fifllows:

132).

The relative size of belief revisions fnr negative evidence,

let S~

[$4-$2], can be derived as follows:

S~
let $5
$4

=
:

Sz - ~2,S(X:~)
S:~ -- o,~s(x~).

(A3)
(A4)

Again, assume s(x~) = s(x4), and both are equal to a constant.

By substituting (A3) into (A4),

S4

(Sz

ed2)

c*($2

s~

aS2)

- $ 2 ( 2 ~ - R2), or [$4-$2] = $2(2ct - (12).

(A7)
(A8)

Assume s(x6) = s(x4) and both are equal to a constant. By

substituting (A7) into (AS),

= $2 - oA'2 -- 0S2 + ~ZSz.

Thus, $4-$2

$2 + ~(l -- S2)s(x))
= ,$'~ + ~(1 -- S3)s(x4).

Thus, S4-S2

{s~ + I~(l - s ~ ) } + ~ ( 1 - lS~ + [~(1 - S ~ ) l }

$2 + [3(1 -- $2) + [3 - 13S2 - [~2(1 - $2)

$2 + [3{(1 - $2) + (1 - $2) - [3(1 - $2)}

s~ + ( I - s ~ ) (2[3 -

(1 - $2)(213 - [32).

13~).