Professional Skepticism and Auditors' Workpaper Review

Kathy Hurtt University of Wisconsin Martha Einin and !avid Plumlee University of Utah !raft version March" #$$% &e'ruary" #$$#

Please do not quote without permission of the authors. Comments are welcome. We thank the workshop participants at Arizona State University, University of Utah, University of Wisconsin and entley Colle!e for their comments on earlier versions of this paper.

Professional Skepticism and Auditors' Workpaper Review

'

Professional Skepticism and Auditors' Workpaper Review "eview of su#ordinates$ workpapers #y more e%perienced auditors consumes a si!nificant portion of the effort on an audit en!a!ement & am#er and ylinski '()*+. "esearch into the #ehavior of workpaper reviewers shows that e%perience affects their a#ility to detect certain kinds of errors &e.!., "amsay '((,, am#er and "amsay '((*, -..., /ardin! and 0rotman '(((+. Audit risk and information importance are situational factors positively associated with the accuracy of an auditor$s memory for evidence contained in workpapers &Sprinkle and 0u##s '(()+. eyond e%perience and the situational factors investi!ated #y

Sprinkle and 0u##s &'(()+, our understandin! of the workpaper review process remains remarka#ly incomplete. 1ne une%amined potential influence on the review process can #e found in the audit standards, which state that, 2&s+ince evidence is !athered and evaluated throu!hout the audit, professional skepticism should #e e%ercised throu!hout the audit process2 &AU $ -3..*4(+. 0hus, the due care requirement that auditors e%ercise professional skepticism is a critical mandate that should e%tend to auditors$ workpaper review. 0his paper e%amines the relationship #etween the de!ree of skepticism possessed #y an auditor and four specific #ehaviors e%hi#ited durin! workpaper review. 0he audit profession$s institutionalization of workpaper reviewin! arises from the requirement that assistants #e adequately supervised &AU $ '5...-+. /owever, from a practical perspective, the review process is time consumin! and e%pensive & am#er and ylinski '()*, Asare and 6c7aniel '((8+, and findin! ways of producin! efficiency !ains in workpaper reviewin! are likely #e welcomed #y the auditin! profession. 9arly studies of the review process &0rotman and :etton '()5, 0rotman '()5+ find no performance difference #etween interactin! !roups comprised of two senior auditors and !roups comprised of a senior and a

mana!er. "amsay &'((,+ finds that seniors detect mechanical errors #etter than mana!ers do, while the reverse is true for conceptual errors. A similar relationship e%ists #etween staff and senior auditors, where staff auditors are #etter at detectin! mechanical errors &/ardin! and 0rotman '(((+. 1ther studies show that com#inin! the reviews of mana!ers and seniors results in more effective reviews than either seniors or mana!ers alone & am#er and "amsey '((*+, and composite teams consistin! of a senior and a staff auditor out perform pairs of either staff or senior auditors &/ardin! and 0rotman '(((+. While this research !enerally focuses on the efficiency and effectiveness !ains that could result from specialization within the review process, si!nificant other influences, such as the de!ree of skepticism possessed #y the reviewer, remain une%amined. 0his paper addresses one such influence; the reviewer<s professional skepticism. =n addition to the due care requirement for auditors to #e skeptical, Statement on Auditing Standards (SAS) No. 53 &A=CPA '())c+, No. 57 &A=CPA '())#+, No. 67 &A=CPA '((-+, and No. 82 &A=CPA '((*a+ reinforce the necessity for auditors to e%hi#it professional skepticism in a variety of situations and on different types of en!a!ements. /owever, there is anecdotal evidence that auditors either do not consistently #ehave skeptically or there is some variation in the de!ree of skepticism possessed #y auditors. >ormer chief accountants of the Security and 9%chan!e Commission$s &S9C+ enforcement division enforcement division have indicated that a lack of independence and a lack of professional skepticism were amon! the primary causes of S9C actions a!ainst accountants &0he CPA ?ournal '((8+. "esearch on S9C 9nforcement Actions &'()*4'((*+ #y easley, Carcello and /ermanson concurred with this assessment, indicatin! that 8.@ of the enforcement actions were related to a lack of professional skepticism &A=CPA -...+. and that 2people failure( is the most common cause of

audit failures &Choo and 0an '(()+. =n response to the S9C$s concern a#out the quality of financial audits, the Pu#lic 1versi!ht oard esta#lished a panel that recommends recommended to audit firms that they provide !uidance to their audit personnel a#out the concept of professional skepticism &P1 -...+. 0hus, understandin! the role of professional skepticism in workpaper review may provide important insi!ht into a reco!nized pro#lem within the audit profession. A critical step in conductin! research involvin! professional skepticism is a means of identifyin! individuals who can #e characterized as skeptical. "ecent research #y /urtt &-..'+ has resulted in a 3.4item psycholo!ical scale that measures the de!ree of skepticism possessed #y an individual. =n addition, a model linkin! an auditor$s de!ree of skepticism and certain #ehaviors has #een proposed &/urtt, 9inin! and Plumlee -..'+. 0he model of skepticism, developed from surveys of professional accountants as well as literatures from philosophy and psycholo!y, predicts that four #ehaviors will #e associated positively with increasin! levels of skepticism; &'+ e%panded information search, &-+ increased identification of contradictions, &3+ increased !eneration of alternatives and &,+ increased e%amination of information from and a#out people. 1ur study tests the assertion that auditors with hi!h levels of skepticism will e%hi#it these #ehaviors when reviewin! workpapers. 0he A=CPA accepts the position that professional skepticism results in #ehavioral differences. SAS 53 &A=CPA '())c+, one of the pronouncements intended to reduce the 2e%pectation !ap2 #etween the !eneral pu#lic and pu#lic accountants, re4emphasized the necessity for an auditor to have an attitude of professional skepticism and descri#ed actions that a skeptical auditor would take. An even stron!er position on professional skepticism, and one with more #ehavioral implications, is found in SAS )- which states that, 27ue professional care

requires the auditor to e%ercise professional skepticism 4 that is, an attitude that includes a questioning mind and a critical assessment of audit e idence2 &emphasis added+. SAS )- also modifies SAS ' to more clearly define and e%plain professional skepticism, and lists e%amples of skeptical #ehavior, includin! sensitivity in the selection and e%tent of documentation and increased corro#oration of mana!ement representations &A=CPA '((-#+. We test whether more skeptical auditors e%hi#it these four #ehaviors when reviewin! workpapers as well as the impact of hi!her audit risk on less skeptical auditors in an on4line workpaper review task administered to auditors from a maAor international accountin! firm. 0he e%periment involved two phases. =n the first phase, participants accessed a we# site and completed the Skepticism Scale, 6arlowe4Crowne Social 7esira#ility Scale &Crowne B 6arlowe '(8.+ and demo!raphic questions, which were placed on a we#4server where participants answered the scales #y accessin! a U"C for the site.. 0he second phase consisted of a workpaper review task, which was also performed on4line, where the reviewer was ask to create review notes #ased on the final review of a su#set &de#t and inventory+ of audit workpapers. 9m#edded in the workpapers were two risk treatment conditions &hi!h and normal audit risk indicators+. Usin! the Skepticism Scale score to differentiate #etween those who were more and less skeptical, we find that more skeptical auditors detect more contradictions and demonstrate a sta#le search pattern across risk situations. =n contrast, a hi!h risk audit situation induced less skeptical auditors, to !enerate more information search queries, spendin! more time evaluatin! the information and look at more information from and a#out people, than they did in the low risk situation. /owever, this additional work #y less skeptical auditors does not result in more contradictions #ein! detected. 0his su!!ests that althou!h less skeptical auditors reco!nize the risk present in a hi!h risk situation, they are not

effective in translatin! that awareness of risk into #etter detection of contridictionscontradictions. 0he remainder of this paper is or!anized as follows. 0he ne%t section reviews prior research into #oth workpaper review and professional skepticism alon! with the #ehavioral predictions. 0he fourth section descri#es the e%perimental methods and data collections. 0he data analysis is in the fifth section with the conclusions and implications in the last section. ))* +A,K-R./0! 1)2ERA2/RE A0! H3P.2HESES Research on the workpaper review process am#er and ylinski &'()*+ document the su#stantial portion of total audit hours that audit firms devote to the review process. Certain situational factors have #een shown to affect the review process., Asare and 6c7aniel &'((8+ show that when reviewer$s are familiar with preparers they reperform less of the ori!inal work. 0hey also find that reviewers who are more familiar with preparers detect more conclusion errors in a comple% task relative to routine task, while reviewers were not familiar with preparers are more effective in a the routine task. 1ther e%perimental studies of the review process compareed Aud!ments #etween sequential reviewin! and interactin! !roups, and show that , whether performance is measured #y consensus or accuracy, the sequential review process was not found to #e #etter than the !roup performance &0rotman and :etton '()5, 0rotman '()5+. "esearch involvin! actual review of simulated workpapers e%plicitly reco!nizes that the nature of the review task differs #etween mana!ers and seniors. "amsey &'((,+ finds that mana!ers were #etter at detectin! conceptual errors, while seniors were more accurate at detectin! mechanical errors. /ardin! and 0rotman &'(((+ !eneralize "amsey$s &'((,+ findin! #y showin! that composite !roups with a staff and a senior auditor outperform composite !roups with either two seniors or two

staff auditors. 9%tendin! this research to the question of whether specialization improves reviewer performance, am#er and "amsey &'((*+ compared 2all4encompassin!2 reviews with specialized reviews that focused e%clusively on either conceptual or mechanical errors. Consistent with "amsey they find that mana!ers were more effective at identifyin! conceptual errors, while seniors were more effective at identifyin! mechanical errors. /owever, when e! "ost teams comprised of #oth a senior and a mana!er en!a!e an all4encompassin! review, they are more effective than teams performin! specialized reviews. 9%aminin! review efficiency and reviewer confidence, am#er and "amsey &-...+ find that specialized reviews are less efficient in terms of time, and seniors performin! specialized reviews were more confident when performin! specialized reviews, while mana!ers were not. =n studies involvin! reviewer$s memory for audit evidence, 6oeckel &'((.+ finds that ine%perienced auditors fail to connect related pieces of audit evidence more often than e%perienced auditors were more likely to mentally reconstruct inconsistent pieces of evidence to form a coherent memory of the evidence. 6oeckel and Plumlee &'()(+ correlate confidence in memory with reco!nition accuracy and find that, when the mental representations resulted from inference, reco!nition accuracy and auditor<s confidence in their memories were inversely related, . =implyin! that inferred &i.e., internally !enerated memories+ were seen as valid audit evidence. Cookin! for situational factors that mi!ht miti!ate misplaced confidence, Sprinkle and 0u##s &'(()+ show that audit risk and information importance to miti!ate misplaced confidence. Ci##y and 0rotman &'((3+ demonstrate that the review process can serve as a countervailin! force to the potentially #iased recollections of su#ordinates. am#er, am#er and ylinski &'())+ surveyed audit mana!ers usin! an instrument that focused on four audit areas. 0hey created a ta%onomy of ei!ht workpaper review activities.

>or each activityies, participants indicated whether they would en!a!e in that activity and estimated their anticipated time spent reviewin!. am#er et al. find that = mana!ers en!a!e in

directed search, that is, pursuin! related audit evidence rather than analyzin! information sequentially. am#er et al. descri#e the review process as a sta#le search process where auditors are sensitive to account materiality and risk, #ut that does not chan!e si!nificantly their review strate!ies. 0hey also find that over three4quarters of their su#Aects read all workin! papers for each account and perform active inquiry such as thinkin! of competin! e%planations performin! analytical tests. =n summary, research into the workpaper review process finds that it consumes a lar!e portion a firm$s total audit hours, and situational factors effect performance. Auditors$ level in the firm corresponds to how they mentally view the review task, which, in turn, affects their performance in error detection. Auditors$ search throu!h workin! papers is directed and, typically, thorou!h. "esearch into audit workpaper reviewin! has not e%amined the impact of professionally mandated factors such as skepticism. Research on skepticism 6uch of the skepticism research in accountin! &6c6illan and White '((3, Shau# and Cawrence '((8, Shau# '((8, Choo and 0an '(()+ attempts to measure an individual$s skepticism level then e%amine #ehavioral differences amon! individuals with different levels of skepticism. 6ost of these studies #uild upon the premise that suspicion is the opposite of trust, and skepticism is synonymous with suspicion. /owever, it is difficult to find either theoretical or anecdotal support for equatin! these constructs. Ssuspicion and skepticism are never equated in the philosophical literature on skepticism, ratherwhich shows that skepticism is

portrayed as is a comple% construct, rather than #ein! unidimensional. Also, /urtt &-..'+ indicates that practicin! accountants did not equate skepticism with suspicion. Several other studies employ a manipulation intended to induce skepticism for a treatment !roup, then e%amined the #ehavioral differences #etween !roups &Peecher '((8, 0urner '((*, Shau# and Cawrence '((*+. =n # oth Peecher &'((8+ and 0urner &'((*+ find #ehavioral differences #etween the auditors induced to #ehave skeptically and those in a control condition. 0urner finds that auditors in the skepticism4induced condition e%amined more total items and e%amined more disconfirmatory items &items that would tend to disprove or discount the client$s position+ than did the other auditors. Peecher finds that auditors in a skepticism4induced situation were less likely to rely on client4provided e%planations. Shau# and Cawrence &'((*+ manipulate situational varia#les in ei!ht different scenarios so that part of the information in the scenario either si!naled or did not si!nal a risk of fraud. 0hey ask participants to predict what they would do in each scenario, and find that new hires !enerally e%hi#it !reater skepticism &defined as increased audit testwork and increased questionin! of mana!ement and other personnel+ than do seniors, mana!ers and partners. 0hese studies demonstrate that auditors #ehave differently when induced to do so. =t appears that auditors respond to either situational cues indicatin! increased risk of fraud or to e%plicit instructions to #e more skeptical. =n contrast to the situational view of previous studies, /urtt &-..'+ adopts the view that skepticism is a psycholo!ical characteristic of individuals. She develops an understandin! of philosophical skepticism as a #asis for her multidimensional definition of skepticism, from which she develops an instrument that can #e used to measure individuals$ level of skepticism. instrument She identifies si% su#constructs of skepticism from questionin! professional

accountants and from the philosophical literature on skepticism; a quest for knowled!e &curiosity+, a desire to understand people, a questionin! nature, a slowness in formin! Aud!ments, a reluctance to accept others$ claims, and self4confidence. 0hese constructs were used to develop a 3.4item scale which demonstrated stron! test4retest relia#ility, indicatin! that skepticism is a relatively sta#le individual characteristic./urtt e%amined e%istin! psycholo!ical instruments as potential measures for these su#constructs, in li!ht what each instrument was desi!ned to measure and research related to the use of each scale. 0his e%amination identified e%istin! instruments with items relatin! to the constructs of skepticism, which were e%amined and rewritten for possi#le inclusion in the Skepticism Scale. >or constructs without e%istin! measures, she created new items. After three rounds of pilot testin!, --. potential items were reduced to a 3.4item scale. 0he si% su#constructs emer!ed in a factor analysis and the resultin! scale demonstrates stron! test4retest relia#ility, indicatin! that it a valid measure of the skepticism construct. 0his validated skepticism scale provides useful measure of individual<s$ skepticism level. As /urtt et al. &-..'+ e%plain, research into the writin!s of early philosophers &Annas and arnes '()5, un!e '((', Durtz '((-, 6cEinn '()(, Popkin '(*(+ su!!ests that skepticism results in four #ehaviors; increased information search, increased detection of contradictions, increased !eneration of alternative e%planations, e%panded scrutiny of information from and a#out people. Citin! Durtz &'((-+ they e%plain that skeptics have a $questionin! nature$ characterized #y searchin! and e%aminin!, and skeptics are people who persist in their investi!ations &Annas and arnes '()5+. /urtt et al. also ar!ue that skeptics are

motivated to answer questions that are raised throu!h an e%panded information search and are comforta#le with uncertainty #ecause of their willin!ness to suspend Aud!ment. 0hus, skeptics

'.

e%hi#it more e%tensive information searches than less skeptical individuals. 0he second #ehavior of skeptics asserted #y /urtt et al. is that skeptics are more adept at discoverin! contradictions. 0hey attri#ute this a#ility of skeptics to their willin!ness to suspend Aud!ment, makin! it comforta#le co!nitively for them to delay reachin! conclusions and search more information. 0he third #ehavior /urtt et al. contend skeptics e%hi#it is constructin! alternative hypotheses re!ardin! statements or claims &6cEinn '()(+. Support for this comes for the followin!, 2Skeptics wish to e%amine all sides of a questionF and for every ar!ument in favor of a thesis, they usually can find one or more ar!uments opposed to it2 &Durtz '((-, p --+. So, skeptical individuals can #e e%pected to construct alternate interpretations for the information they o#serve. 0he fourth #ehavior /urtt et al. attri#ute to skeptics is an interest in information a#out people. 0hey claim that skeptics want to understand underlyin! assumptions #ehind claims and #eliefs and that skeptics try to understand people in order to understand the assumptions that were made &Popkin '(*(+. 0his reluctance $to accept others$ claims$ results in an e%panded investi!ation of information provided #y people, since #y understandin! a person skeptics can understand the perceptions and assumptions made #y that person. 0hese philosophical #ehavioral e%pectations are also consistent with the audit procedures required #y auditin! standards. 0he e%panded information search e%pected of skeptics is consistent with the !eneral standard on fieldwork &SAS '+ that requires an auditor to o#tain sufficient competent evidential matter #efore opinin! on the financial statements. 0his mandates that an auditor perform testwork until she or he #elieves that sufficient evidential matter e%ists. =n addition, #oth contradiction detection and alternative !eneration are e%pected of auditors. >or e%ample, SAS 58 &A=CPA '())a+ requires auditors to develop e%pectations a#out a company$s financial statements prior to performin! any analytical review procedures.

''

As part of the procedure, auditors are e%pected to identify and e%plain &!enerate alternative e%planations for+ any une%pected differences. =t is only after differences have #een identified and alternatives have #een !enerated #y the auditor that any inquiry as to mana!ement$s e%planations should #e made. 9valuatin! the e%planations provided #y mana!ement a!ain allows an auditor to detect contradictions in e%planations or amon! analytic review results. =n addition, SAS 58 &A=CPA '())a+ indicates that an auditor must o#tain corro#oratin! evidence for mana!ement$s assertions. SAS )- requires that auditors specifically assess mana!ement$s characteristics #y o#tainin! information a#out mana!ement$s a#ilities, pressures, style and attitude toward internal control. 0his is consistent with the su!!estion from philosophical skepticism that a skeptic will e%ercise increased scrutiny of information #oth from and a#out people. Hypothesis !evelopment Hypothesized effect of skepticism While Peecher &'((8+ and 0urner &'((*+ find that participants instructed to e%ercise professional skepticism e%hi#it skeptical #ehaviors, we #elieve that skepticism is also a personality trait. 0hat is, skeptical individuals will e%hi#it skeptical #ehaviors across situations. Specifically, we e%pect that skeptical auditors will e%hi#it the four #ehaviors found in the model of skeptical #ehavior developed #y /urtt et al. &-..'+; !reater information search, #etter detection of contradictions, !reater !eneration of alternative e%planations, and e%panded scrutiny of information from and a#out people. 0his leads to the followin! research hypothesis; H%4 7urin! work paper review, auditors with hi her levels of skepticism will display a reater levels of information search, contradiction detection, alternati e generation, and scrutin# of information from and a$out "eo"le than those with lower levels of skepticism. Hypothesi5ed effect of risk on individual6s with lower skepticism levels

'-

SAS )- instructs auditors to #e aware of situational cues that indicate a need for a hei!htened sense of professional skepticism. Shau# and Cawrence &'((*+ find that auditors responded to situational risk cues em#edded in several e%perimental scenarios. As EAAS specifies the need for skeptical #ehavior in hi!h risk situations, and research indicates that auditors do respond to situational cues. What is more uncertain is whether there will #e #ehavioral differences in the manner that more and less skeptical auditors #ehave in a 2normal2 audit situation. ?ohnson &'(*)+, indicates that althou!h all people are 2skeptical2 a#out certain thin!s or in certain situations, not all people are skeptics, nor are skeptics skeptical in all situations. We assume that skeptical auditors maintain a hi!her level of skeptical #ehavior at all times, and less skeptical auditors will e%hi#it skeptical #ehavior in hi!her audit risk situations, #ut they will not e%hi#it skeptical #ehaviors when the audit situation is not hi!her risk. 0his leads to the followin! hypotheses; H#4 7urin! work paper review, less skeptical auditors in the low risk condition will e%hi#it lower level of information search, contradiction detection, alternati e generation, and scrutin# of information from and a$out "eo"le than either more skeptical auditors in either condition or than less skeptical auditors in the hi!h risk condition. E7perimental Methods E7perimental desi n 0his research employs a randomized #lock e%perimental desi!n where participants were cate!orized as scorin! hi!her or lower on the Skepticism Scale, then assi!ned to one of two risk treatment !roups. 0he risk treatments result from manipulation of situational cues that indicate either a 2normal2 or a 2hi!h risk2 audit condition. 0he dependent varia#les are information e%amined and time spent &to measure information search #ehavior and scrutiny of information

'3

from and a#out people+, contradictions detected &to measure contradiction detection+, and alternatives !enerated &to measure alternative !eneration+. E7perimental 2ask 0he e%periment involved two distinct phases. =n the first phase, participants completed the Skepticism Scale and the 6arlowe4Crowne Social 7esira#ility Scale &Crowne B 6arlowe '(8.+ and answered demo!raphic questions. 0he scales and demo!raphic questions were coded into /06C and ?avaScript and placed on a we#4server where participants answered #y accessin! a U"C for the site. 0he participants$ responses were collected online usin! a CE= Script. After scorin! each participant$s responses to the Skepticism Scale, we assi!ned the participants alternately to either the hi!h or low risk e%perimental condition #ased on their ranked skepticism score to ensure #alanced cell sizes #etween the e%perimental conditions. 0he second phase of the e%periment was administered an avera!e of '3 days after completion of the pretest, and consisted of a workpaper review task with two treatment conditions &hi!h and normal audit risk indicators+ em#edded in the workpapers. Similar to Shau# and Cawrence &'((*+, one set of workpapers contains several cues that are consistent with a hei!htened audit risk level and the other did not have any specific hi!h risk cues. Appendi% lists the differences #etween the instructions in the two treatment conditions.

0he participants conducted !eneral information search throu!h 2Workpaper2 screens. 0he auditors were e%pected to si!n4off on these screens &these were $normal$ audit workpaper de#t and inventory screens+ and the auditors had the a#ility to write review notes for them. =n contrast, information from and a#out people was contained within 2People2 screens. 0he 2People2 screens contained #rief #io!raphies on the employees, and there was no a#ility to si!n4off or write review notes on these screens.

',

0he e%perimental task was #ased on the one used #y 6oeckel &'((.+. 0he written task instructions task inform the participants that he or she will #e performin! the only review of a su#set &de#t and inventory+ of the workpapers, and they ask the reviewer to document any review and their reasons for makin! the note. >or this study, the ori!inal workpapers were modified sli!htly and were coded into /06C and ?avaScript and placed on a we#4server. 0he modifications included; '+e%pandin! the workpapers to include 2people2 information consistin! of an or!anizational chart and #rief #io!raphies of company employeesF -+ a 2review notes2 te%t#o% frame located on the top portion of every screen replacin! the review note ta#let in the 6BP studyF 3+ tickmarks modified to ones supported #y a key#oardF and ,+ the addition of separate screens for the 2people information.2 7ates were also chan!ed to make the workpapers current. 0he ori!inal workpapers were desi!ned to contain a num#er of contradictions and errors and these remained in the online version. Appendi% A contains an e%ample of #oth a contradiction and an error that were in the workpapers. As a result of pilot testin!, several chan!es were made to the workpapers; '+ inclusion of a screen indicatin! specific inventory testwork that was performed, which replaced physical copies of ei!ht invoices that were included in the ori!inal workpapersF -+ modification of the instructions from one lon! screen to three short screens which are easier to read onlineF and 3+ dividin! the permanent file screen from one lon! to three shorter screens for the same reason. A second pilot test indicated that these chan!es were effective and !ave preliminary indication that participants in the treatment conditions were respondin! as predicted. Participants completed the task #y accessin! a U"C for the online site, e%aminin! on4 line the workpapers and writin! review notes which were captured usin! a CE= Script. An

'5

analysis of the participant$s workpaper review notes provides evidence re!ardin! detection of contradictions, detection of errors, and information search queries. =n addition, unknown to the participants, the pro!ram captured the screens e%amined, and the amount of time spent on each screen. 0his information allowed analysis of the information search #ehavior and the scrutiny of information from and a#out people. 0o e%amine the participant$s !eneration of alternatives, three #rief scenarios were presented to the participants at the conclusion of the workpaper review task. 9ach participant was asked to !enerate written alternative e%planations for each scenario &0a#le )+, and an analysis of the participant$s answers to the scenarios provides evidence re!ardin! alternative !eneration. !ata Analysis and Results Prior to #e!innin! statistical analysis, the alternative !eneration, identification of contradictions and information search varia#les were coded #y two independent raters who were #lind to the treatment conditions and skepticism levels of the participants. =nter4rater relia#ility was !reater than .(. for the codin! on contradiction detection, and !reater than .)5 for alternative !eneration codin! and information search codin!. 7ifferences were resolved #y the researchers. oth the amount of time spent' and the num#er of screens the auditor e%amined are used to measure !eneral information search and the scrutiny of information from and a#out people. 0o o#tain time spent and num#er of screens information, the raw data captured #y the CE= Script was summarized #y screens and time. An initial review of the time spent indicated that, althou!h participants were instructed to complete the entire review without interruptions,
' Participants were not aware that time or search pattern was #ein! captured.

'8

in some cases it appeared that participants had #een interrupted. 0he data was winsorized and all analyses use this winsorized sample data.- 7escriptive statistics for all varia#les are presented in 0a#le '. 444444444444444444444444444444444444444444444444444 %nsert &a$le ' a$out here 444444444444444444444444444444444444444444444444444 Effect of skepticism 1ur initial test was a 6AG1HA I with all 0a#le '3 presents the results of a series of AG1HA procedures used to test for the effect of skepticism. Consistent with philosophical predictions, skepticism level was a si!nificant &pJ..5, n K *5+ predictor of the auditor$s detection of contradictions. Contradictions are the type of errors that are most critical for an auditor to identify and include elements such as contradictory or inconsistent client e%planations. =ncreased identification of this type of error should result in more effective audits. Skepticism level was not a si!nificant predictor of increased information search or increased scrutiny of information a#out people. Geither was it si!nificant in the num#er or type of alternatives !enerated.3 , 444444444444444444444444444444444444444444444444444
- >or e%ample, participants spent &on avera!e+ appro%imately two minutes on workpaper AP>43 &a sin!le te%t pa!e containin! information on client personnel+. /owever, one participant &who wrote no review notes while this workpaper was on his or her screen+ spent over ''5 minutes #efore e%itin! this screen. Con!ruent with the procedure descri#ed #y Winer &'(*'+ on how to handle influential outliers, the sample was winsorized at g ( ', where the hi!hest and lowest time values were replaced #y the second hi!hest and second lowest time values. 3 Additional analyses were completed usin! re!ressions run with the participant$s raw skepticism score. 0his did not result in any meanin!ful chan!es in the results. 0he num#er of contradictions detected remained si!nificant &t K -.888, si! t J ...(F adA "- K ..*8+. , >our auditors never e%amined the trial #alance or su#sequent event workpapers. 0his is si!nificant #ecause of the nine contradictions present in the workpaper, si% of them have part of the contradictory information on either the trial #alance or su#sequent event workpapers. Participants who did not review these workpapers did not have the information to detect the maAority of the contradictions. 0hree of the four auditors who did not e%amine these workpapers were in the hi!h skepticism cate!ory. We reanalyzed the data after eliminatin! these four individuals. 0he new analysis resulted in only one si!nificant chan!e; the num#er of mechanical errors detected was different #etween the two treatment conditions.

'*

%nsert &a$le ' a$out here 444444444444444444444444444444444444444444444444444 Planned comparison with low risk and low skepticism level 0a#le '5 presents the results of a series of AG1HA procedures performed prior to completin! the planned comparison #etween the low skepticism, low risk condition !roup and the remainin! three !roups. 0his analysis e%amines whether differences e%ist amon! the !roups. 7etection of contradictions remains si!nificant as does the scrutiny of information from and a#out people, which was measured #y the num#er of people screens e%amined. 444444444444444444444444444444444444444444444444444 %nsert &a$le ' a$out here 444444444444444444444444444444444444444444444444444 0he results of the planned comparison are shown in 0a#le '8. 0he num#er of information search queries written in the review notes is si!nificantly different #etween the two !roups, as is the num#er of people screens e%amined and the time spent on these. /owever, the detection of contradictions or !eneration of alternatives is not si!nificantly different #etween the !roups. 444444444444444444444444444444444444444444444444444 %nsert &a$le ' a$out here 444444444444444444444444444444444444444444444444444 ,.0,1/S).0 0his research e%amines the important issue of whether professional skepticism affects four #ehaviors in workpaper review. a term widely used in auditin! literature #ut not clearly defined or well understood. 0he purpose of this study was test the theoretical predictions of

')

skeptical #ehaviors utilizin! the instrument to differentiate #etween more and less skeptical individuals. >ollowin! is a #rief discussion of these three areas that hi!hli!hts overall contri#utions, areas of potential limitations, and future research. 0he model predicts that skeptics will e%hi#it four #ehaviors; e%panded information search, detection of contradictions, !eneration of alternatives, and scrutiny of information from and a#out people. Usin! the Skepticism Scale score o#tained from an on4line pretest to differentiate #etween those who were more and less skeptical, the auditors in this study completed an on4line workpaper review task and then responded to scenarios askin! them to !enerate alternatives. 0he most si!nificant findin! from the e%periment is that more skeptical auditors find more contradictions. 6ore skeptical auditors also demonstrate a sta#le search pattern across risk situations. 0his is consistent with my e%pectations that more skeptical auditors maintain a level of skeptical #ehavior in all situations and do not react to chan!in! risk conditions with radical chan!es in #ehavior. =n contrast, less skeptical auditors, when placed in a hi!h risk audit situation, respond #y !eneratin! more information search queries, spendin! more time evaluatin! the information and lookin! at more information from and a#out peopleF however, this additional work does not result in more contradictions detected. 0his su!!ests that althou!h less skeptical auditors reco!nize the risk present in a hi!h risk situation, they are not effective in translatin! that awareness of risk into #etter results. 0he e%panded information search #ehavior that was predicted for more skeptical auditors was not evident in the num#er of screens e%amined or the time spent on those screens.

'(

Upon reflection, the nature of the e%perimental task, a workpaper review which requires that the participants 2si!n4off2 on each workpaper, created a #ias a!ainst findin! si!nificant differences in search #ehaviors. Auditors are trained to review and si!n each of these workpapers which limited the availa#ility of discretional additional information for e%panded information search. 0he #ehavioral model predicts that more skeptical auditors will !enerate more alternative e%planations and a !reater num#er of different types of alternative e%planations. Geither of these #ehaviors was evident from the e%periment. 1nce a!ain, the nature of the e%perimental task, where the participants were answerin! #rief scenarios after completion of a workpaper review, mi!ht have #iased a!ainst findin! si!nificant differences. Additionally, the mean time spent on this e%periment was over seventy4five minutes, and at the end of that amount of time, many participants mi!ht have #een too e%hausted to !ive complete attention and provide alternatives to these scenarios. =n retrospect, a workpaper review task is not one where a lar!e num#er of alternatives are !enerated. 1n the other hand, auditors performin! analytic review procedures are required to !enerate e%planations for variances. 6easurin! an auditor$s skepticism level #efore he or she performs an analytic review task and usin! performance on that task to test the model$s prediction of alternative !eneration mi!ht #e a more appropriate task. 0his study also contri#utes #y usin! a new method of deliverin! the e%perimental materials to the participants. 0he e%periment conducted for this study was administered entirely on4line usin! World Wide We#4#ased technolo!y. 0he on4line nature of this e%periment allowed auditors to complete the workpaper review task at a time and place of

-.

their choosin!, recreatin! the most realistic audit environment. 0his type of e%perimental desi!n offers one alternative to e%pensive and time4consumin! on4site e%periments.

-'

References am#er, 9. 6. and ". ?. "amsay. -.... 0he effects of specialization in audit workpaper review on review efficiency and reviewers$ confidence. Auditing) A *ournal of +ractice and &heory '( &-+;',*4'5*. am#er, 9. 6. and ". ?. "amsay. '((*. An investi!ation of the effects of specialization in audit workpaper review. ,ontem"orar# Accounting -esearch ', &>all+; 5.'45'3. /ardin!, G. and D. 0. 0rotman. '(((. /ierarchical differences in audit workpaper review performance. ,ontem"orar# Accounting -esearch '8 &Winter+; 8*'48),. Ci##y, ". and D. 0. 0rotman. '((3. 0he review process as a control for differential recall of evidence in auditor Aud!ments. Accounting .rgani/ations and Societ# ') &Au!ust+; 55(45*,. 6essier, W. >., and ". 6. 0u##s '((,. "ecency effects in #elief revision; the impact of audit e%perience and the review process. Auditing) A *ournal of +ractice and &heory '3 &'+;5*4*-. 6oeckel, C. C. '((.. 0he effect of e%perience on auditors$ memory errors. *ournal of Accounting -esearch -) &Autumn+; 38)43)*. "amsay, ". ?.. '((,. SeniorLmana!er differences in audit workpaper review performance. *ournal of Accounting -esearch 3- &Sprin!+; '-*4'35. Sprinkle, E. . and ". 6. 0u##s. '((). 9ffects of audit risk and information importance on auditor memory durin! workin! paper review. &he Accounting -e ie0 *3 &1cto#er+; ,*545.0rotman, D. 0. '()5. 0he review process and the accuracy of auditor Aud!ments. *ournal of Accounting -esearch -3 &Autumn+; *,.4*5-. MMMM, and P. :etton. '()5. 0he effect of the review process on auditor Aud!ments. *ournal of Accounting -esearch -3 &Sprin!+; -584-8*.

' 0a#le ' Su#Aect 7emo!raphic Statistics &6ean values are presented first, followed #y median values &italici/ed+ with standard deviation in parentheses+

Risk ,ondition 1ow 1ow Skepticism ;-roup%< n8%= A e -).)a 28.1 &-.(+ E7perience in Months 8-.8 52.1 &-..*+ Skepticism Score '-).8NN '23.1 &8.,+ -ender ;M 8 Male? & 8 &emale<
a

Risk ,ondition Hi h 1ow Skepticism ;-roup :< n8## -8.8a 26.1 &-.,+ ,).8 27.5 &'-.(+ '-).-NN '23.1 &*.-+ '. 6 '- > -).)a 28.5 &3.'+ 5(.* 56.1 &3,.'+ ',*.*NN '26.1 &*.)+ '- 6 *> ' no response Hi h Skepticism ;-roup >< n8#$

n89:

Hi h Skepticism ;-roup #< n8## -*.)a 28.1 &-..+ 8-.) 6'.5 &-3.5+ ',(.8NN '28.5 &*.8+ '3 6 (>

'. 6 (>

Si!nificant difference e%ist #etween !roup 3 and !roups ' and ,, p J ..5 &t4test+

NNSi!nificant difference #etween the hi!h and low skepticism cell means &as planned+, p J ...' &t4test+

0a#le ' 7escriptive Statistics &6ean values are presented with standard deviation in parentheses.+

Risk ,ondition 1ow 1ow Skepticism )nformation search measures Workpaper Screens Count Workpaper Screens 0ime in minutes =nformation search queries ,ontradiction detection measures Contradictions 9rrors Alternative eneration measures 0otal num#er of alternatives 7ifferent types of alternatives n8%= 5'.( &-'.5+ *8.8 &3,.5+ .,* &'..+ n8%= '.&'.,+ .*, &.85+ n8%= -., &-.5+ '.) &'.)+ People search measures People Screens Count People Screens 0ime in minutes n8%= .) &'.'+ .8 &'.,+ Hi h Skepticism n8## 5-.&-,.8+ *5.8 &3*.(+ '.( &3.'+ n8%9 '.8 &'.'+ .8' &.()+ n8%9 3.' &'.5+ -.8 &'.'+ n8## -.) &,.'+ -.&3.5+

Risk ,ondition Hi h 1ow Skepticism n8## 53.8 &-5.*+ *,.* &--.'+ '.8 &-.'+ n8%= '.3 &'.-+ '.' &.)5+ n8%= 3., &-.3+ -.5 &'.5+ n8## ,.) &5.*+ 3.&,.5+ Hi h Skepticism n8#$ 5,.8 &-*.'+ *(.3 &-5.8+ -.' &3.'+ n8%= -.5 &'.)+ '.&'..*+ n8%= -.* &'.8+ -.3 &'.3+ n8## -.5 &3.,+ '.) &3.-+

3 0a#le 3 AG1HA "esults; 9ffect of Skepticism Cevel Panel A4 )nformation Search &/ypothesis 'a+ nK)3 Effect Workpaper screens e%amined Workpaper time &in minutes+ Gum#er of information search queries &n K *,+ Panel +4 !etection of ,ontradictions &/ypothesis '#+ nK*, Effect 0otal lo!ic errors detected 0otal mechanical errors detected Panel ,4 -eneration of Alternatives &/ypothesis 'c+ nK*' Effect 0otal alternatives !enerated 0otal different types of alternatives SS ...5 .('* &@Aalue ...' .'-) Si * of & .(*.5'. SS ''.*.5 ...' &@Aalue 5.*.3 .... Si * of & *$#$ .()( SS 5.-*' 83.,,8 '*.**3 &@Aalue ...( ..8( -.('8 Si * of & .(-8 .*(3 ..(-

Panel !4 Scrutiny of )nformation from and a'out people &/ypothesis 'd+ nK)3 Effect People screens e%amined People time &in minutes+ SS -.-(* .....8 &@Aalue .'-) .... Si * of & .*-.(()

, 0a#le , AG1HA Amon! all >our Eroups Panel A4 )nformation Search nK)Effect Workpaper screens e%amined Workpaper time &in minutes+ Gum#er of information search queries &nK*,+ Panel +4 !etection of ,ontradictions nK*, Effect 0otal lo!ic errors detected 0otal mechanical errors detected SS '(.5-3 3.*'' &@Aalue 3.-53 '.5-( Si * of & *$#B .-', SS (*.,5* -,(.35' -(.,)8 &@Aalue ..53 ..)) '.8'' Si * of & .(), .(88 .'(5

Panel ,4 -eneration of Alternatives nK*' Effect 0otal alternatives !enerated 0otal different types of alternatives SS (.(*3 8.'55 &@Aalue .**3 .()) Si * of & .5'3 .,.3

Panel !4 Scrutiny of )nformation from and a'out people nK)Effect People screens e%amined People time &in minutes+ SS '*-.(,, *..-5&@Aalue 3.5,* -..-Si * of & *$%9 .'')

5 0a#le '8 Planned Comparison #etween Cow Skepticism, Cow "isk Condition and the 1ther Eroups Panel A4 )nformation Search Effect Workpaper screens e%amined Workpaper time &in minutes+ Gum#er of information search queries Panel +4 !etection of ,ontradictions Effect 0otal lo!ic errors detected 0otal mechanical errors detected Panel ,4 -eneration of Alternatives Effect 0otal alternatives !enerated 0otal different types of alternatives Panel !4 Scrutiny of )nformation from and a'out people Effect People screens e%amined People time &in minutes+ t -.,58 -..'( df *( *( Si nificance *$%C *$>B t '.'5( '.5)5 df 8) *' Si nificance .-5. .''* t '.,*, .)53 df *' *' Si nificance .',5 .3(* t .-,3 ..', -.''df *( *( *' Si nificance .).( .()( *$:9

Appendi7 A
,ontradiction or Error e7ample Contradiction Statements in Workpapers 0here were stron! markets at the end of the prior year. Philips Co. had forecast that stron! markets would #ear #oth price and volume increases. 6ana!ement decided to #e very a!!ressive and to do whatever was necessary to take full advanta!e of the situation. . . Second, to !et in on the price increases it had predicted would result from the stron!er market at the end of >: '((*, Philips Co. chan!ed its pricin! policies across the #oard. Philips Co. has plants located in Stillwater, Gew 6e%ico &also its central office+ and Coldton, 6innesota. Philips closed a plant in Sunnyvale, California in 7ecem#er '((* when its lease on the #uildin! e%pired. Sales were down 5.5 @. 0here are two related reasons for this. . . . Second, Philips decreased its prices across the #oard in #oth 6arch and ?une in order to reduce its inventory #uildup &per schedule A4 ', inventory; O-.5,5.. and O'8,,.,. in '(() and '((*+, and to shore up slackenin! sales. )nference or Pro'lem Were prices increased or decreasedP

9rror

0his note was retired with the proceeds from the sale of land and the closin! of the Coldton plant.

Which plant closedP

Appendi7 +
A0ormal" Audit Risk )nstructions E9G9"AC ACDE"1UG7 =G>1"6A0=1G 0erry randon, who prepared the workpapers you have received, has a#out one year of e%perience, includin! a limited amount of work on the prior year$s audit of Philips Co. AHi h@ Audit Risk )nstructions E9G9"AC ACDE"1UG7 =G>1"6A0=1G 0erry randon, who prepared the workpapers you have received, has a#out one year of e%perience with /enderson and >ranks, includin! a limited amount of work on the prior year$s audit of Philips Co. :ou have no reason to question 0erry$s #asic competence, #ut since 0erry was trained #y another firm, there will pro#a#ly #e minor stylistic differences #etween your approaches to the same work. CC=9G0 =G>1"6A0=1G; 0he client, which is new to you, is Philips Co., a desi!ner and manufacturer of water treatment systems. :our firm acquired Philips Co. as a client on 3L3'L() when your firm mer!ed with /enderson B >ranks, a well respected re!ional CPA firm. /enderson B >ranks has audited Philips Co. for the past seven years, and rendered unqualified opinions in all cases.

:ou have no reason to question 0erry$s #asic competence. CC=9G0 =G>1"6A0=1G; 0he client is Philips Co., a desi!ner and manufacturer of water treatment systems. :our firm has audited Philips Co. for the past seven years, and rendered unqualified opinions in all cases.