Journal of Business and Psychology, Vol. 21, No. 4, Summer 2007 (2007) DOI: 10.

1007/s10869-007-9038-9

DO PEOPLE FAKE ON PERSONALITY INVENTORIES? A VERBAL PROTOCOL ANALYSIS Chet Robie

Wilfrid Laurier University

Douglas J. Brown
University of Waterloo

James C. Beaty
PreVisor

ABSTRACT: Research has focused heavily on whether individuals can fake on personality inventories. Research is less clear on whether individuals actually do fake on personality inventories. Verbal protocol analysis was used to trace the motivational processes for 12 participants as they completed a personality inventory in an applicant context. Exploratory analyses suggested that individuals do fake on personality inventories; that individuals can be classed into one of three faking classes (honest responders, slight fakers, and extreme fakers); and that honest fakers take less time to complete and make less corrections to their personality inventories than faking responders. Study implications, limitations, and future research will be discussed. KEY WORDS: faking; personality testing; verbal protocol analysis.

INTRODUCTION Findings from years of research strongly support the use of personality inventories in selection systems. Barrick, Mount, and Judge (2001) quantitatively summarized the results of 15 prior meta-analytic

Address correspondence to Chet Robie, E-mail: crobie@wlu.ca This research was funded in part by Social Science and Humanities Research Council of Canada Grant # 410-2005-0207 awarded to Chet Robie and Douglas J. Brown. 489
0889-3268/07/0600-0489/0 2007 Springer Science+Business Media, Inc.

490

JOURNAL OF BUSINESS AND PSYCHOLOGY

studies that have investigated the relationship between the Five Factor Model (FFM) personality traits and job performance. Their primary results were that conscientiousness was a valid predictor across performance measures in all occupations studied; emotional stability was found to be a generalizable predictor when overall work performance was the criterion, but its relationship to specic performance criteria and occupations was less consistent than was conscientiousness; and the other three Big Five traits (extraversion, openness and agreeableness) did not predict overall work performance, but they did predict success in specic occupations or relate to specic criteria. Moreover, organizations are using personality inventories in their stafng systems. In a survey of 959 organizations in 20 different countries, personality inventories were found to be the most often used method of selection (Ryan, McFarland, Baron, & Page, 1999). One issue that continues to concern practitioners is the issue of faking (i.e., intentional or motivated distortion) on personality inventories. For the purpose of the present study, we dene faking as a conscious attempt to manipulate ones responses to create a positive impression (McFarland & Ryan, 2000; Zicker & Robie, 1999). A survey of 77 assessors who conducted individual assessments for an international consulting rm was recently conducted (Robie, Tuzinski, & Bly, 2005). One of the ndings was that a majority of the assessors (70%) believed that faking was a serious threat to the validity of the personality inventory in the assessment process. The current study is designed to examine whether this concern by practitioners is justied; specically, do people fake on personality inventories? We will rst briey review the literature on faking on personality inventories. We will then provide an overview of the primary methodology we used in the current study (verbal protocol analysis) and how it has been used in prior studies. Finally, we will present our study hypotheses. Faking on Personality Inventories A great deal of research has been conducted on faking on personality inventories. The majority of research has attempted to answer the following research questions: (1) Can people fake on personality inventories?; (2) Does faking affect the measurement properties/validity of personality inventories?; and (3) What can be done to reduce faking or the effects of faking? The research is most clear on the rst research question. Metaanalytic research across 51 studies has found that individuals can fake personality inventories when instructed to do so (Viswesvaran & Ones, 1999). Research also shows that individuals vary considerably in their ability to fake (McFarland & Ryan, 2000; Mersman & Shultz, 1998).

C. ROBIE, D. J. BROWN AND J. C. BEATY

491

The research on the next research question is less clear. Research conducted by (Ellingson, Smith, & Sackett, 2001; Smith & Ellingson, 2002) found that faking does not affect personality structure, although other researchers suggest that faking does affect psychometric properties (Stark, Chernyshenko, Chan, Lee, & Drasgow, 2001). In a multi-sample study, Hough, Eaton, Dunnette, Kamp, and McCloy (1990) found that personality validities remained stable regardless of possible distortion by respondents in either unusually positive or negative directions; whereas, other researchers have found faking to result in decrements to criterion-related validity (Douglas, McDaniel, & Snell, 1996; Topping & OGorman, 1997). Many studies have found prima facie evidence of faking (cf. Bass, 1957; Becker & Colquitt, 1992; Dunnette, McCartney, Carlson, & Kirchner, 1962)that is, a group that one would expect to score higher on a noncognitive test (e.g., applicants) on average does score higher than a control group (e.g., honest group). Several researchers suggest that faking may affect hiring decisions (Christiansen, Gofn, Johnston, & Rothstein, 1994; Mueller-Hanson, Heggestad, & Thornton, 2003; Rosse, Stecher, Miller, & Levin, 1998). The nal research question has arguably led to the most controversy and the least clarity. Research has found that measures that contain items that are ipsative (Bowen, Martin, & Hunt, 2002); forced-choice (Christiansen, Burns, & Montgomery, 2005; Jackson, Wroblewski, & Ashton, 2000);1 or subtle (Holden & Jackson, 1981; Worthington & Schlottmann, 1986) may be better than normative measures for controlling faking. However, issues of applicant comparison for these types of measures complicate hiring decisions compared to items from normative measures. Researchers have also developed indirect measures of personality (James et al., 2005); however, the development of the instruments appears to be onerous and, after years of development, only one or two constructs have been developed. Research fairly conclusively has shown that social desirability measures that are designed to catch fakers do not act as predictors, moderators, or mediators of various job performance outcomes (Ones & Viswesvaran, 1998; Ones, Viswesvaran, & Reiss, 1996). Researchers have also attempted to use item response theory techniques called appropriateness measurement to identify fakers (cf. Zickar & Drasgow, 1996); the results of this line of research have mostly been disappointing. Similarly, use of response latencies to identify or ameliorate the effects of faking has not met with promising results (Holden, Wood, & Tomashewski, 2001). One experimental study found that a warning which identied that faking could be identied and the potential consequences of faking impacted responding (Dwight &
1 A recent study by Heggestad, Morrison, Reeve, and McCloy (2006) calls into question the viability of using forced-choice measures to control faking.

492

JOURNAL OF BUSINESS AND PSYCHOLOGY

Donovan, 2003). Moreover, McFarland (2003) found that warnings do not negatively affect test-taker reactions. All of the above research rests on the premise that at least some individuals do actually fake on personality inventories. However, no research to date to our knowledge has addressed the basic question of whether individuals fake on personality inventories. In the current study, we used a methodology called verbal protocol analysis to provide some preliminary insights into whether and the extent to which individuals fake on personality inventories. Verbal Protocol Analysis Verbal protocol analysis (VPA) is a method used to collect data on how people make decisions from a process perspective (Carroll & Johnson, 1990). When utilizing VPA, experimenters request participants to say whatever comes to mind while performing a task of interest. Concurrent reports are most often utilized because memory and other cognitive biases can distort retrospective reports (Nisbett & Wilson, 1977). It is important not to cue the participant (i.e., decision maker) by asking for certain types of information, particularly if that information would not normally be salient while doing the task (Ericsson & Simon, 1984). VPA has been used for decades in the decision sciences; its use is more recent in the organizational sciences. In one of the earlier organizational applications of VPA, Isenberg (1986) used VPA to examine managerial problem solving. Martin and Klimoski (1990) used VPA to trace cognitions associated with self- and supervisor evaluations of performance. Barber and Roehling (1993) used VPA to investigate the process of deciding whether or not to apply for jobs. Cable and Graham (2000) used VPA to examine the determinants of job seekers reputation. VPA has thus been used to examine a wide variety of organizational issues. VPA has not (to our knowledge) been used to examine faking issues. We will now present the hypotheses of the current study. Study Hypotheses As noted above, one of the basic premises upon which much of the faking literature is based is that at least some individuals do fake on personality inventories. Therefore, our rst hypothesis is: H1: One or more individuals will fake on the personality inventory.

Few studies have speculated on how faking may be distributed within a given population. A multi-sample study by Zickar, Gibby, and Robie (2004) using mixed-model item response theory techniques found that individuals could be classed into three faking classes: regular

C. ROBIE, D. J. BROWN AND J. C. BEATY

493

responding (or honest) class; a slight faking (or mixed) class; and an extreme faking class. H2: Individuals will be able to be classed into honest responders, slight fakers, and extreme fakers. A study on faking in the individual assessment process suggests that individuals who fake on personality inventories compared to those who do not fake may: (a) make more corrections to their personality item responses (i.e., change their responses); and (b) take longer to complete the inventory. Specically, in a study of individual assessors who regularly administer personality inventories for selection purposes (Robie et al., 2005), one assessor answered the open-ended question, Please share additional possible reasons for faking with the following answer: I had a participant who completed the personality inventory and worked it over for two more times before he submitted the questionnaire. This comment suggests that participants who fake both change responses and take longer than those who do not fake. Thus: H3: Individuals classed as honest responders will make fewer corrections to their responses on the personality inventory than individuals classed as fakers.

H4: Individuals classed as honest responders will take less time to nish the personality inventory than individuals classed as fakers.

METHOD Participants Twelve individuals not currently enrolled in university courses (i.e., a community sample) participated in the study. The majority of participants (seven) were female and all participants were Caucasian. Four participants were between 36 and 40 years of age; ve participants were between 41 and 45 years of age; and three participants were between 51 and 55 years of age. One participant possessed a high school degree or equivalent; four participants possessed some college or university; ve participants possessed undergraduate degrees; and two participants

494

JOURNAL OF BUSINESS AND PSYCHOLOGY

possessed post-graduate degrees (M.S., M.A., or M.B.A.). Three participants were employed full-time; six participants were employed part-time; and three participants were unemployed and searching for work. One participant possessed 57 years of full-time work experience; one participant possessed 1113 years of full-time work experience; three participants possessed 1719 years of full-time work experience; and seven participants possessed 20+ years of full-time work experience. The majority of participants (seven) reported holding a part- or full-time retail sales position in the past. No participants currently held a part- or full-time retail sales position. The employed participants held a variety of jobs including: Administrative Assistant; Bus Driver; High School Guidance Counselor; Laborer; Machine Operator; Manager; Packager; Paper Route Deliverer; and Technical Writer. Table 1 lists the demographic data for each participant. Procedure Participants were recruited for the current study via a local daily newspaper advertisement placed in the employment section on a consecutive Wednesday and Saturday. The advertisement read: Wanted

Twelve research participants

On a study examining personality at Wilfrid Laurier University. Three top-scoring participants on the personality inventory will each receive $100. Participants must be between 25 and 55 years of age, possess at least two years of full-time work experience in any industry, and be willing and able to travel to Wilfrid Laurier Universitys Waterloo campus for testing. Compensation for travel to campus will be $10. Participants will be accommodated on a rstcome, rst-served basis. Please contact Dr. Chet Robie at crobie@wlu.ca (519) 884-0710 ext.2965 to set up a one-hour appointment if you are interested in participating. All participants were individually administered all materials by Chet Robie. Participants were rst asked to read and sign a consent form detailing the tasks that they would engage in and the respective rights and responsibilities of the participant and researcher during and after the appointment time. Participants were told on the consent

Table 1 Data Display of All Relevant Study Variables with Participants Rank-Ordered by Personality Score Fake or Honest?

Gender Age No 214 27.27 6 28.75 Extreme faker Honest

Education

Employment Work Past Current Job Personality Memory # of Corrections Time Status Experience Retail Score Score (%) to (min) to (years) Experience? Inventory Complete Inventory

Female 4145 University

Part-time

20+

Female 3640 Some Part-time university Female 3640 Post-graduate Part-time Yes Yes Yes No Yes Yes Yes Yes No No Laborer Manager Packager 183 168 157 90.91 56.82 68.18 34.09 38.64 84.09 45.45 1 0 0 1 0 0 0 34.09 56.82 0 0 63.64 10 41.00 16.50 12.92 12.08 10.33 13.33 16.67 12.08 16.92 10.25

1719

No

High school guidance counselor Bus driver 212 22.73 0 14.42

57

Slightfaker Slight faker Honest Honest Honest Honest Honest Honest Honest Honest

Male 4145 Post-graduate Unemployed 1719 Female 5155 University Part-time 20+

Male

Full-time

20+

Unemployed 1719 Part-time 20+

C. ROBIE, D. J. BROWN AND J. C. BEATY

4145 Some university Male 3640 University Female 5155 Some university Female 4155 Some university Male 5155 University Female 4145 University Male 3640 High school

Unemployed 20+

Technical 199 writer Unemployed 196 Paper route 195 delivery Machine 194 operator Unemployed 192 Administrative 188 assistant Unemployed 188

Part-time Full-time Full-time

20+ 20+ 1113

Note. All participants: were Caucasian; were not currently enrolled in university; and did not currently hold retail sales jobs.

495

496

JOURNAL OF BUSINESS AND PSYCHOLOGY

form that the research project examined how people go about responding to personality inventories in work-type situations. The true purpose of the study was not divulged to participants until a postexperimental debrieng was administered. Participants were then given the following directions (based on typical verbal protocol analysis technique; cf. Ericsson & Simon, 1984): As soon as you begin working on completing the personality inventory please start thinking aloud. The best way to do this is to be as spontaneous as possible. Tell me everything you are thinking as you are thinking it, even details or sidetracks that seem insignicant or embarrassing. If you think aloud spontaneously, you will soon forget that you are speaking at all. There is no need to explain to me why you are thinking what you are. You dont have to interpret or justify your approach to an item. Just tell me what you are thinking at the moment. If you are silent for more than a few seconds, I will remind you by saying: Please tell me what you are thinking or Keep talking.

While taking the inventory, keep in mind that how you score on the questionnaire will determine whether you will be considered for one of the three $100 prizes. Nothing that you say out loud will have any impact on whether you will be considered for one of the prizes. The three $100 prizes will go to the individuals whose personality test scores most closely match the requirements of the job detailed in the following job advertisement: Participants were then shown a job advertisement for the target retail sales positions (see Appendix). The job advertisement was an actual job advertisement from the Human Resources Development Canada Job Bank web page (http://jb-ge.hrdc-drhc.gc.ca/) with identifying information on the employer and address made ctitious. Participants were instructed to take as long they liked to read the job advertisement but that they could not refer back to the advertisement once they began lling out the personality inventory. Participants then lled out a paper-and-pencil personality inventory while thinking aloud. Their verbalizations were audio taped. After participants completed the personality inventory, they were asked to (in the following order): complete a brief post-experimental manipulation check (see Measures); complete a demographic information sheet; read a postexperimental debrieng form that explained the true purpose of the study; read and sign a release form (with an option for opting out) for the

C. ROBIE, D. J. BROWN AND J. C. BEATY

497

use of their quotations in subsequent reports; and complete several forms to ensure that they would be able to be contacted in the event that they won one of the $100 prizes and to release the $10 travel compensation to them. Nine participants provided email addresses. One week after participation, six participants who responded honestly (see Determination of Response Set below) and provided email addresses were sent the following brief post-experimental email-based survey: Dear Participants:

If you feel comfortable with sending me a brief email describing possible reasons as to why you decided NOT to match your personality responses to the hypothetical job description (i.e., fake good) on the personality inventory, I would very much appreciate it!

Sincerely,

Chet Robie, Ph.D. For the three participants that faked to some extent (see Determination of Response Set below) and provided email addresses, the email was changed to the following: Dear Participants:

If you feel comfortable with sending me a brief email describing possible reasons as to why you decided to match your personality responses to the hypothetical job description (i.e., fake good) on the personality inventory, I would very much appreciate it!

Sincerely,

Chet Robie, Ph.D.

498

JOURNAL OF BUSINESS AND PSYCHOLOGY

Measures Personality inventory The personality inventory used in this study was the Criterion International Service InventoryShort Form (CISI-S) (ePredix, 2004). The CISI-S is a brief, standardized, self-report assessment designed to be used along with other job-related information to identify candidates with high potential for success in entry to mid-level positions. Three characteristics of the inventory demonstrate that it was developed specically for use in a work context. First, all items reect a work environment frame of reference. Second, the span of contact in the items includes ones self, coworkers, supervisors, and customers. Finally, the items include work relevant situational demands such as dealing with an angry customer, checking work for errors, or getting to work on time. Test-takers respond to statements by choosing one of four response categories labeled as denitely no, probably no, probably yes, and denitely yes. Meta-analytic research has shown that effective customer service instruments tend to measure three broad personality characteristics: Conscientiousness, Emotional Stability, and Agreeableness (Frei & McDaniel, 1998). The CISI-S was designed to measure two of those three broad personality characteristics: Conscientiousness (31 items) and Agreeableness (26 items). Both factors were drawn from a previously developed and larger personality inventory based on the Five-Factor model of personalitythe Criterion Occupational Behavior Inventory (COBI). Conscientiousness in the CISI-S inventory is dened as being related to the likelihood that individuals will exert effort to complete tasks ahead of schedule, tend to be well-organized and pay attention to detail in tasks, get to work on-time and attend work regularly, have a strong desire to achieve their goals, and tend to be dedicated employees that are committed to the success of the company. An example item from the Conscientiousness scale is: Sometimes when it is especially nice outdoors, I will excuse myself from work. The internal consistency coefcient for the Conscientiousness scale is .84. Agreeableness in the CISI-S inventory is dened as being related to the likelihood that individuals will demonstrate enthusiasm in customer interactions, successfully calm irate customers, react with sensitivity to others feelings, continue to work hard during busy or frustrating circumstances, and making time to nd product information for customers. An example item from the Agreeableness scale was: At work, I tend to offer my assistance to others. The internal consistency coefcient for the Agreeableness scale is .78. Because the two factors were found to be fairly highly correlated in a large customer service applicant sample (r = .66, N = 49,100) (Shepherd, Brown, & Robie, 2005), we elected to simply combine the two factors for scoring purposes and select top-down when drawing for prizes.

C. ROBIE, D. J. BROWN AND J. C. BEATY

499

The highest possible score on the personality inventory was then 57 (sum of 31 items from Conscientiousness scale + 26 items from the Agreeableness scale) 4 (maximum points available per item) = 228. Post-experimental manipulation check Participants were asked to complete a partially blank job advertisement form after completing the personality inventory. The job advertisement form was similar in all respects to the job advertisement form that they were presented with prior to their completing with the personality inventory with the exception that the skill requirement sections had been removed and only the headings remained: Education; Credentials; Languages; Work Setting; Product Setting/Knowledge; Specic Skills; Security and Safety; and Essential Skills. Participants were asked to remember as much as they possibly could from the rst time they were presented with the information and write the information in the appropriate elds. The task was scored by weighting a eld by its number of words. In this manner, the total score for the task was 44 (44 words in all the elds). For example, under Specic Skills which contained 14 words, if a participant only remembered Work with Minimal Supervision, they would receive a score of 4 for that eld. Numbers of corrections to personality inventory The numbers of times that participants made corrections or changes to personality responses on the personality inventory were recorded. Multiple corrections per item could be counted although this did not occur. Time to complete personality inventory The time to complete inventory was recorded in minutes. Conversion to tape counter to time was: (one side of tape = 360 tape count = 30 min). Thus, the tape counter was divided by 360 and multiplied by 30 to derive the time in minutes to complete the personality inventory. Determination of response set Participants were classied as responding in one of the following response sets: (1) honest, (2) slight fakers, or (3) extreme fakers. We coded participants as honest based on the principle of trait activationthat ...behavioral expression of a trait requires arousal of that trait by trait-relevant situational cues (p. 398) (Tett & Guterman, 2000). To be classied in the honest response set, a participant must clearly be responding to all of the items on the personality inventory from the standpoint of situations that appear to be eliciting trait-based information from that individual. Several excerpts from actual audio tapes illustrate the coding principles:

500

JOURNAL OF BUSINESS AND PSYCHOLOGY

...When the need comes, I will offer to work extra hours or weekends or whatever if I feel it is really important.

...[reads item]...Ya, I think I do that.

Yes, I do complete jobs within a reasonable amount of time.

[reads item]...denitely no...because of my attachment to Christ. To be classied in the extreme fakers response set, a participant must clearly be responding to all of the items on the personality inventory from the perspective of a hypothetical ideal applicant (i.e., the personality inventory is not capturing any trait-based information). In other words, it did not appear that the individual was using situations from his or her experience to answer the personality items which then were activating trait-based information. Several excerpts from an actual audio tape illustrate the coding principles: I wonder what type of responses they want?

If I answer denitely yes to all of these, theyll think Im lying.

[reads item]...yes, that would be a plus for the job.

[reads item]...I dont remember reading anything about getting a commission...however, if I say denitely yes, they are going to be thinking that I am going to be cutthroat...um, ya, Ill say yes, denitely. To be classied in the slight fakers response set, a participant must appear to be using themselves as the referent in their initial verbalizations for each item, but switch to how the hypothetical ideal applicant should respond. Several excerpts from actual audio tapes illustrate the coding principles: [reads item]...Generally yes, so Ill say probably yes because its true and it looks good on the sheet.

C. ROBIE, D. J. BROWN AND J. C. BEATY

501

[reads item]...Oh, thats not me...ya, I get distracted, but you would have to go with probablydont say denitely on any of these...ok...[reads same again]...probably no, because that means that at least that you are open to people coming up to you and talking to you in retail. RESULTS Table 1 provides a data display of all relevant variables with participants rank-ordered by personality score. Personality scores ranged from 157 to 214 and averaged 190.50 (SD = 16.09); this compares to a range of 66 to 228 and an average of 209.24 (SD = 11.87) for a large customer service applicant sample (Shepherd et al., 2005). Scores on the post-experimental manipulation check (i.e., memory scores) ranged from 22.73 to 90.91% accuracy and averaged 51.89% accuracy (SD = 21.99) and did not differ signicantly by honest versus faking responders [t(10) = .92, p > .05]. The number of corrections to the personality inventory ranged from 0 to 10 and averaged 1.50 (SD = 3.18). The time to complete the personality inventory ranged from 10.25 to 41.00 min and averaged 17.10 min (SD = 8.99). One individual was classied as responding in the extreme fakers response set; two individuals were classied as being in the slight fakers response set; and nine individuals were classied as responding in the honest response set. Given that at least one individual was classied as a faker on the personality inventory and individuals were able to be classied into the three a priori specied classes, support was found for the rst two hypotheses. The extreme fakers and slight fakers response set categories were collapsed into one category so that a response set category of fakers could be compared to honest for testing the third and fourth hypotheses. The discriminant function for the number of items corrected on the personality inventory was statistically signicant (v2 = 7.16, p < .01), with fakers participants correcting 5.33 items on average and honest participants correcting 0.22 items on average. The discriminant function for the time to complete the personality inventory was also statistically signicant (v2 = 8.94, p < .01), with fakers participants taking 28.75 min to complete the personality inventory and honest participants taking 13.22 min to complete the personality inventory. Thus, support was also found for the third and fourth study hypotheses. Five of the nine participants who were classied as being in the honest response set responded to the post-experimental email-based survey question asking them why they did not fake on the personality inventory. Following are excerpts from each responding participants answers:

502

JOURNAL OF BUSINESS AND PSYCHOLOGY

It never entered my mind to not tell the truth. I answered the questions as if you really wanted to know who I am. (bus driver)

What motivated me to be true and sincere can be summed up in one word experience! I have worked with lots of achievers and other so called ,hard workers and have realized that the truth always comes out in the end. Up front, it may appear as if you have fooled someone, but as I always say, the worm always turns! If I have what you need, then Ill deliver, but I do not falsify just to get a job! (administrative assistant)

It never even entered my mind to fake my responses so as to enhance my chance of winning the money. If it had occurred to me to do so, I most likely wouldnt have, for integritys sake. I already know from experience that I am good with customers in a retail setting, so I was quite condent of being naturally suitable for the hypothetical position. Thus, even if I had been willing to fake my responses, fakery would not have seemed likely to produce any substantial improvement. Finally, I am generally more concerned with process than outcome, so the potential reward of $100.00 would not motivate me more than the intrinsic motivation of my own character just to be true to myself. I am the sort of principled idiot who most likely would stand before a ring squad refusing to tell a white lie that would save my life. Wanting to live would not be enough it matters too much how I live. Wanting the hundred dollars was not enough to entice me into dishonesty. (laborer)

If the prize amount was greater, hypothetically a thousand dollars, then you would likely have seen changes in my results. This is because a hundred dollars would make no appreciative difference in my current situation, but a thousand dollars would help considerably, therefore I would be more motivated to conform to the personality needed for the job. I suspect that if the prize amount was greatly increased there would be a lot more general competition. (unemployed female)

Not sure why I didnt lie to conform to the job description. Integrity I would have to say...seems to trump exaggerations. Hindsight I should have lied...oh well integrity is of more value anyways. (packager)

C. ROBIE, D. J. BROWN AND J. C. BEATY

503

Both of the participants who were classied as being in the slight fakers response set responded to the post-experimental email-based survey question asking them why they did not fake on the personality inventory. The participant who was classied as being the extreme fakers response set did not respond to the post-experimental emailbased survey question. Following are excerpts from each of the responding participants answers: I think the internal ambivalence and struggle I experienced answering the questions was in deciding whether it was the job being applied for or the temporary monetary benet that I was gearing my answers towards. Initially, I approached the task with the mindset that I wanted to best represent myself for the job. I think I felt obligated and it was easier to respond to a question as honestly as possible, if it was a reection of myself (maybe with just a bit of a stretch if it seemed a little more desirable one level up). (technical writer)

Though it was very clear, right upfront, that the ,winners would get prizes of $100, I found that this information somewhat receded into the background as I was continuing the questionnaire. Also, the details of the job also were rather lost on me after I got involved in the questionnaire. I knew it was a retail position based on a Walmart job description, I believe but as I continued through the questions, I actually took most of the answers based on my own personal awareness. However, I was conscious about what employers value in general, so I tended to focus on the positive aspects of my personality in relation to the job, and tempered my personality aspects that were not in keeping with my understanding of employers needs and wants. Now, a couple of caveats: Were this a real position, or if the ,prize money were considerably more, then my answers may have been different. As well, I would have likely taken much more time in going through the questions, probably analyzing them much more (potentially too much!) than I did. (unemployed male)

DISCUSSION This is the rst study to provide direct evidence that individuals fake on personality inventories. Prior studies inferred the existence of faking through higher mean personality inventory scale scores between incumbent and applicant populations (Hough, 1998) or applicants

504

JOURNAL OF BUSINESS AND PSYCHOLOGY

endorsement of nonexistent job analytic task statements (i.e., saying they have done something on a job when could not have possibly done it) (Anderson, Warner, & Spencer, 1984; Pannone, 1984). Past research has found that individuals vary in their ability to fake on personality inventories (McFarland & Ryan, 2000; Mersman & Shultz, 1998). The present research suggests that individuals also vary in their motivation to fake and that individuals may be categorized based upon their faking motivation into one of three faking classes: honest; slight fakers; and extreme fakers. This more direct evidence of classbased faking motivation is consistent with less direct evidence provided by Zickar et al. (2004). We also found that individuals classed as honest responders made fewer corrections and took less time completing their personality inventories than individuals classed as faking responders. The results of the current study support McDaniel and Timms (1990) notion that lying takes time; however, other studies have found faster latencies associated with faking (George, 1990; Hsu, Santelli, & Hsu, 1989). Furthermore, Holden et al. (2001) has found that limiting response time does not affect personality inventory validity. The discrepancies between the current study and previous studies in examining completion time can probably be accounted for the fact that previous studies used the item as the level of analysis and the current study used the scale as the level of analysis. An interesting nding from this study is the large number of respondents that were classed as honest and the concomitant low average personality scale score in comparison to a large applicant sample of customer service representatives (Robie et al., 2005). In fact, the difference between the average personality scale scores in the current study and the applicant sample in Robie et al. (2005) is over one standard deviation. One could infer that the induced motivation in the present study is not as strong as an employment context in which an individual is seeking a job. Several post-experimental respondent comments suggest this to be the case (see Results section). Study Implications Several implications from the current study can be drawn. One major implication is that, given that direct evidence of faking has been found in a fairly low stakes testing context, it is likely that more faking is occurring in applicant contexts. Indirect evidence of such likely faking is provided by the large mean differences in personality scale scores between the experimental sample in the present study and the applicant sample in Robie et al. (2005).

C. ROBIE, D. J. BROWN AND J. C. BEATY

505

Another major implication from the current study is the issue of rank-order change. As can be seen in Table 1, the rank-order of respondents would likely change quite drastically if the individuals who faked chose to respond honestly. As has been suggested by others but not demonstrated directly, selection ratio would also be a determining factor in the degree to which faking impacts validity (Christiansen et al., 1994; Mueller-Hanson et al., 2003; Rosse et al., 1998). At a selection ratio of .25, one would hire two fakers and one honest responder. At a selection ratio of .50, one would hire three fakers and three honest responders. At a selection ratio of .75, one would hire three fakers and six honest responders. Limitations This study has several limitations. One limitation is the small size of the sample which may hinder arguments for generalization. VPA studies generally utilize small samples (1015 participants) because of the time intensiveness and costliness of the methodology. Future replications will surely be needed using multiple methods to ensure generalization. Another limitation of our study was also its strength; namely, that is was experimental in nature. It is very difcult to model the motivational characteristics of an applicant context. Respondent post-experimental comments suggest that with an adult community sample perhaps increasing the motivational press over and above what was introduced in the current study would be warranted. Another limitation to the study was the Time to Complete Inventory variable. As one reviewer of this study noted, some individuals are simply more talkative than others. We did not note any individuals that were off task to any major extent and talking about irrelevant material; that is, all participants were engaged in the task at hand and it appeared to us that the individuals who were attempting to fake the measure simply took longer to verbally work through the process. Future Research One of the most pressing avenues for future research is to examine the extent to which honest responders, slight fakers, and extreme fakers produce differential job outcomes and whether these outcomes may differ by job context. Hogan (1991) has argued that faking does not really exist and is instead an applicants attempt to manage how the applicant would like for him or her to be seen which is likely how he or she will perform on the job. On the other hand, some research suggests that faking behavior may actually be positive for some occupations such as sales personnel (Tull, 1998).

506

JOURNAL OF BUSINESS AND PSYCHOLOGY

One critical limitation of VPA, that individuals are many times self-deceived about their own decision-making processes (cf. Nisbett & Wilson, 1977), suggests a future study to rule out alternative explanations for the results in the present study. For example, did the present study identify more honest individuals than one would expect simply because an experimenter was monitoring the participants responses (cf. Asch, 1961) or is this a true state of affairs? A follow-up study can be designed that includes three matched groups: (a) a group that is administered the VPA via a live experimenter; (b) a group that is administered the VPA via a automated system; and (c) a group that is administered a paper-and-pencil version of the measure. Issues of conformity to experimenter expectations can then be addressed.

CONCLUSION The current study is the rst study to nd direct evidence of faking on personality inventories. People do fake on personality inventories; people can be categorized into one of three classes of responders; and honest responders take less time and make fewer corrections to their personality inventories than do faking responders.

APPENDIX Job Advertisement Advertisement number: 1360658 Title: Sales associateretail Terms of Employment: Permanent, Part Time, Weekend Salary: To be negotiated Anticipated Start Date: As soon as possible Location: xxxxxxxx, xxxxxxx (3 vacancies) Skill Requirements:

Education: Completion of high school Credentials (certicates, licenses, memberships, courses, etc.): Not required Experience: Experience an asset Languages: Speak English, Read English, Write English Work Setting: Department store Product Experience/Knowledge: not applicable

C. ROBIE, D. J. BROWN AND J. C. BEATY

507

Specic Skills: Operate cash register, Customer service oriented, Provide advice about merchandise, Work with minimal supervision Security and Safety: Criminal record check Essential Skills: Writing, Oral communication, Working with others, Problem solving Employer: Walmax Please apply for this job only in the manner specied by the employer. Failure to do so may result in your application not being properly considered for the position. How to Apply: In Person between 9:00 and 16:30: 1105 Felsure Blvd. xxxxxxxx, xxxxxx xxx xxx By Phone between 9:00 and 16:30: (519) 555-7500 Advertised until: 2005/06/03 REFERENCES
Anderson, C. D., Warner, J. L.,, & Spencer, C. C. (1984). Ination bias in self-assessment examinations: Implications for valid employee selection. Journal of Applied Psychology, 69, 574580. Asch, S. (1961). Issues in the study of social inuences on judgment. In I. A. Berg & B. M. Bass (Eds.), Conformity and deviation (pp. 143153). Oxford, England: Harper. Barber, A. E.,, & Roehling, M. V. (1993). Job postings and the decision to interview: A verbal protocol analysis. Journal of Applied Psychology, 78, 845856. Barrick, M. R., Mount, M. K., & Judge, T. A. (2001). Personality and performance at the beginning of the new millennium: What do we know and where do we go next?. International Journal of Selection and Assessment, 9, 930. Bass, B. M. (1957). Faking by sales applicants of a forced choice personality inventory. Journal of Applied Psychology, 41, 403404. Becker, T. E.,, & Colquitt, A. L. (1992). Potential versus actual faking of a biodata form: An analysis along several dimensions of item type. Personnel Psychology, 45, 389406. Bowen, C., Martin, B. A., & Hunt, S. T. (2002). A comparison of ipsative and normative approaches for ability to control faking in personality questionnaires. International Journal of Organizational Analysis, 10, 240259. Cable, D. M.,, & Graham, M. E. (2000). The determinants of job seekers reputation perceptions. Journal of Organizational Behavior, 21, 929947. Carroll, J. S.,, & Johnson, E. J. (1990). Decision research: A eld guide. Newbury Park, CA: Sage Publications. Christiansen, N. D., Burns, G. N., & Montgomery, G. E. (2005). Reconsidering forced-choice item formats for applicant personality assessment. Human Performance, 18, 267307. Christiansen, N. D., Gofn, R. D., Johnston, N. G., & Rothstein, M. G. (1994). Correcting the 16PF for faking: Effects on criterion-related validity and individual hiring decisions. Personnel Psychology, 47, 847860. Douglas, E. F., McDaniel, M. A., & Snell, A. F. (1996, August). The validity of non-cognitive measures decays when applicants fake. Paper presented at the annual meeting of the Academy of Management, Cincinnati, OH.

508

JOURNAL OF BUSINESS AND PSYCHOLOGY

Dunnette, M. D., McCartney, J., Carlson, H. C., & Kirchner, W. K. (1962). A study of faking behavior on a forced-choice self-description checklist. Personnel Psychology, 15, 1324. Dwight, S. A.,, & Donovan, J. J. (2003). Do warnings not to fake reduce faking?. Human Performance, 16, 123. Ellingson, J. E., Smith, D. B., & Sackett, P. R. (2001). Investigating the inuence of social desirability on personality factor structure. Journal of Applied Psychology, 86, 122 133. ePredix. (2004). Criterion International Service Inventory (short form) technical report. Minneapolis, MN: Unpublished technical report. Ericsson, K. A.,, & Simon, H. A. (1984). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press. Frei, R. L.,, & McDaniel, M. A. (1998). Validity of customer service measures in personnel selection: A review of criterion and construct evidence. Human Performance, 11, 127. George, M. S. (1990). The use of response latency to study accuracy and consistency in a computerized lifestyle assessment. Unpublished doctoral dissertation, University of Toronto, Toronto, Ontario, Canada. Heggestad, E. D., Morrison, M., Reeve, C. L., & McCloy, R. A. (2006). Forced-choice assessments of personality for selection: Evaluating issues of normative assessment and faking resistance. Journal of Applied Psychology, 91, 924. Hogan, R. T. (1991). Personality and personality measurement. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (Vol. 2, 2nd ed., pp. 873919). Palo Alto, CA, US: Consulting Psychologists Press, Inc. Holden, R. R., & Jackson, D. N. (1981). Subtlety, information, and faking effects in personality assessment. Journal of Clinical Psychology, 37, 379386. Holden, R. R., Wood, L. L., & Tomashewski, L. (2001). Do response time limitations counteract the effect of faking on personality inventory validity?. Journal of Personality and Social Psychology, 81, 160169. Hough, L. M. (1998). Effects of intentional distortion in personality measurement and evaluation of suggested palliatives. Human Performance, 11, 209244. Hough, L. M., Eaton, N. K., Dunnette, M. D., Kamp, J. D., & McCloy, R. A. (1990). Criterionrelated validities of personality constructs and the effect of response distortion on those validities. Journal of Applied Psychology, 75, 581595. Hsu, L. M., Santelli, J., & Hsu, J. R. (1989). Faking detection validity and incremental validity of response validity of response latencies to MMPI subtle and obvious items. Journal of Personality Assessment, 53, 278295. Isenberg, D. J. (1986). Thinking and managing: A verbal protocol analysis of managerial problem solving. Academy of Management Journal, 29, 775788. Jackson, D. N., Wroblewski, V. R., & Ashton, M. C. (2000). The impact of faking on employment tests: Does forced choice offer a solution?. Human Performance, 13, 371 388. James, L. W., McIntyre, M. D., Glisson, C. A., Green, P. D., Patton, T. W., LeBreton, J. M., Frost, B. C., Russell, S. M., Sablynski, C. J., Mitchell, T. R., & Williams, L. J. (2005). A conditional reasoning measure for aggression. Organizational Research Methods, 8, 6999. Martin, S. L., & Klimoski, R. J. (1990). Use of verbal protocols to trace cognitions associated with self- and supervisor evaluations of performance. Organizational Behavior and Human Decision Processes, 46, 135154. McDaniel, M. A., & Timm, H. W. (August, 1990). Lying takes time: Predicting deception in biodata using response latency. Paper presented at the 98th Annual Convention of the American Psychological Association, Boston, MA. McFarland, L. A. (2003). Warning against faking on a personality test: Effects on applicant reactions and personality test scores. International Journal of Selection and Assessment, 11, 265276. McFarland, L. A., & Ryan, A. M. (2000). Variance in faking across noncognitive measures. Journal of Applied Psychology, 85, 812821. Mersman, J. L., & Shultz, K. S. (1998). Individual differences in the ability to fake on personality measures. Personality and Individual Differences, 24, 217227.

C. ROBIE, D. J. BROWN AND J. C. BEATY

509

Mueller-Hanson, R., Heggestad, E. D., & Thornton, G. C. III. (2003). Faking and selection: Considering the use of personality from select-in and select-out perspectives. Journal of Applied Psychology, 88, 348355. Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84, 231259. Ones, D. S., & Viswesvaran, C. (1998). The effects of social desirability and faking on personality and integrity assessment for personnel selection. Human Performance, 11, 245269. Ones, D. S., Viswesvaran, C., & Reiss, A. D. (1996). Role of social desirability in personality testing for personnel selection: The red herring. Journal of Applied Psychology, 81, 660679. Pannone, R. D. (1984). Predicting test performance: A content valid approach to screening applicants. Personnel Psychology, 37, 507514. Robie, C., Tuzinski, K. A., & Bly, P. R. (2005). Faking in the individual assessment process. Minneapolis, MN: Personnel Decisions International. Rosse, J. G., Stecher, M. D., Miller, J. L., & Levin, R. A. (1998). The impact of response distortion on preemployment personality testing and hiring decisions. Journal of Applied Psychology, 83, 634644. Ryan, A. M., McFarland, L., Baron, H., & Page, R. (1999). An international look at selection practices: Nation and culture as explanations for variability in practice. Personnel Psychology, 52, 359391. Shepherd, W. J., Brown, D. J., & Robie, C. (2005, April). Equivalence of tests administered on computer versus interactive voice response. In F. Drasgow (Chair), Innovations in computerized assessment: Research on practical issues. Symposium conducted at the annual meeting of the Society for Industrial and Organizational Psychology, Los Angeles, CA. Smith, D. B., & Ellingson, J. E. (2002). Substance versus style: A new look at social desirability in motivating contexts. Journal of Applied Psychology, 87, 211219. Stark, S., Chernyshenko, O. S., Chan, K., Lee, W. C., & Drasgow, F. (2001). Effects of the testing situation on item responding: Cause for concern. Journal of Applied Psychology, 86, 943953. Tett, R. P., & Guterman, H. A. (2000). Situation trait relevance, trait expression, and crosssituational consistency: Testing a principle of trait activation. Journal of Research in Personality, 34, 397423. Topping, G. D., & OGorman, J. G. (1997). Effects of faking set on validity of the NEO-FFI. Personality and Individual Differences, 23, 117124. Tull, K. T. (1998). The effects of faking behavior on the prediction of sales performance using the Guilford Zimmerman temperament survey and the NEO Five Factor Inventory. Unpublished doctoral dissertation, University of Akron, Akron, OH. Viswesvaran, C., & Ones, D. S. (1999). Meta-analyses of fakability estimates: Implications for personality measurement. Educational and Psychological Measurement, 59, 197210. Worthington, D. L., & Schlottmann, R. S. (1986). The predictive validity of subtle and obvious empirically derived psychological test items under faking conditions. Journal of Personality Assessment, 50, 171181. Zickar, M. J., & Drasgow, F. (1996). Detecting faking on a personality instrument using appropriateness measurement. Applied Psychological Measurement, 20, 7187. Zickar, M. J., Gibby, R. E., & Robie, C. (2004). Uncovering faking samples in applicant, incumbent, and experimental data sets: An application of mixed-model item response theory. Organizational Research Methods, 7, 168190. Zickar, M. J., & Robie, C. (1999). Modeling faking good on personality items: An item-level analysis. Journal of Applied Psychology, 84, 551563.