Available online at www.sciencedirect.

com

International Journal of Project Management 28 (2010) 650662 www.elsevier.com/locate/ijproman

Do project managers practice what they preach, and does it matter to project success?
Karen E. Papke-Shields *, Catherine Beise, Jing Quan
Information Systems and Decision Sciences Department, Frankin P. Perdue School of Business, Salisbury University, Salisbury, MD, USA Received 18 March 2009; received in revised form 26 October 2009; accepted 3 November 2009

Abstract Organizations increasingly use projects to achieve business objectives but report that results often fall short of goals. A number of formal standards for managing projects have been developed intending to improve such project outcomes. However, research examining this assumption is scarce and has yielded mixed results. This paper presents the development and use of a set of metrics for assessing the use of project management (PM) practices in an empirical assessment of the relative use of dierent practices, and the link between the use of those practices and project success. Our analysis shows widely varying usage of dierent PM practices and dierences in use depending on the context of the project. Further, our study indicates that the level of use of PM practices is indeed related to project success. Finally, the results suggest that the PM practices that make a dierence may not be the most frequently used. 2009 Elsevier Ltd and IPMA. All rights reserved.
Keywords: Project management methods; Project success; Practice; Empirical research

1. Introduction Over the past several decades, organizations have increased their use of team-based projects to achieve business objectives. However, along with the increased use of projects, the complexity and cost of projects have also increased. For example, a KPMG survey of 600 organizations across 22 countries showed that project complexity, in the IT domain, increased in 88% of organizations and budgets increased in 79% of organizations (KPMG, 2005). KPMG also reported that 86% of respondents had project outcomes that fell short of planned expectations (KPMG, 2005). Although some improvement has been seen in terms of project success, a relatively high frequency of project failures has been reported elsewhere as well (e.g., Cicmil and Hodgson, 2006; Lee and Xia, 2005; Pich et al., 2002). Project management has evolved over the past couple decades as both researchers and practitioners have attempted to identify the causes of project failure and the
*

Corresponding author. E-mail address: kepapke-shields@salisbury.edu (K.E. Papke-Shields).

various factors that lead to success. Standards have been developed throughout the world that attempt to codify what has been observed in this on-going research and practice (Cicmil and Hodgson, 2006). Such standards have been developed and disseminated by the Project Management Institute (PMI, 2004), the Association for Project Management (APM, 2006), the Australian Institute of Project Management (AIPM, 2009), and the International Project Management Association (IPMA, 2009), as well as others. The spread of these standards demonstrates evidence of worldwide growth in awareness and acceptance of the need for formal PM methods. However, to what extent are these standards actually used and what eect does their usage have on project performance? Although the body of PM research continues to grow, there is still only limited research evidence that links adherence to these project standards to better project performance (Thomas and Mullaly, 2007). The purpose of this study is to contribute to both research and practice by providing additional evidence for this linkage as well as a better understanding of the relative use of dierent PM practices and the eect of context on such use.

0263-7863/$36.00 2009 Elsevier Ltd and IPMA. All rights reserved. doi:10.1016/j.ijproman.2009.11.002

K.E. Papke-Shields et al. / International Journal of Project Management 28 (2010) 650662

651

2. Background and motivation In attempting to understand the causes of project failure, researchers have explored numerous dimensions of PM, including how projects are done as well as the internal and external context in which projects are executed. One research stream, which began several decades ago (Fortune and White, 2006), has focused on identifying critical success factors (CSFs) for projects. More recently, researchers have begun to address the use of specic PM methods. Both perspectives are important, and are reected in the various sets of standards that are currently deployed globally. The following sections review research in these areas with the emphasis on PM methods given the focus of the current study. 2.1. Project management critical success factors From a PM perspective, CSFs are characteristics, conditions, or variables that can have a signicant impact on the success of the project when properly sustained, maintained, or managed (Milosevic and Patanakul, 2005, p. 183). This section reviews prior research that has focused on identication of which of the factors are considered to be most critical, the relationship between the context and the CSFs, and the relationship between presence of CSFs and project performance, the latter reecting the continuing desire to establish that relationship between what is actually done and project success. As might be expected, dierent studies have identied dierent CSFs, and a lack of consensus of opinion among researchers and authors on the criteria for judging project success and the factors that inuence that success (Wateridge, 1995, p. 171) has been cited as a problem (Fortune and White, 2006). However, an extensive review of 63 publications addressing CSFs resulted in a list of 19 CSFs that were cited from 10 to 39 times in the 63 articles (Fortune and White, 2006). The most frequently cited include support from senior management, clear realistic objectives, strong/detailed plan kept up to date and good communication/feedback. In addition, several studies addressing CSFs have observed the impact of context on which factors are considered most critical as well as whether certain CSFs are indeed related to success. For example, organization size (Hyvari, 2006), project size (Belassi and Tukel, 1996; Hyvari, 2006), project manager experience (Hyvari, 2006), and project structure (Belassi and Tukel, 1996) aect the relative importance of CSFs. Finally, a positive relationship between the presence of CSFs and project performance has been demonstrated in several studies (e.g., Milosevic and Patanakul, 2005; Pinto and Slevin, 1988; Cooke-Davies, 2002). 2.2. Project management methods Methods, as examined in prior research, dier from CSFs in having a narrower, more specic focus. However,

what has been termed as PM methods or practices varies considerably. Prior research addressing methods has included specic tools or techniques, established methodologies, practices associated with published standards, artifacts that result from planning practices, or broad practices such as problem identication (see details in Table 1). In addition, PM maturity identies an organizations maturity level based on the extent of use of specic PM practices (Ibbs and Kwak, 2000; Kwak and Ibbs, 2000). Similarly to CSFs, prior research addressing PM practices has focused on the extent of use of dierent practices, the role of the context in which projects are executed, and the relationship between use of practices and project performance (Table 1). Although there is not yet one single document capturing the formal body of project management knowledge (Cicmil and Hodgson, 2006, p. 114), there is some commonality among standards in terms of PM practices. Yet we do not have a complete picture of which PM practices are being used and the relative use of those practices. Prior work has addressed the use of various PM tools, techniques, and methodologies (Besner and Hobbs, 2006; White and Fortune, 2002) and observed substantial variation in the relative use of dierent methods. But what about the specic practices formalized in PM standards? Previous work has examined planning and observed variation in the relative use of practices in dierent knowledge areas (Zwikael and Globerson, 2004). Other work examined practices across dierent project phases and also observed variation in relative use (Ibbs and Kwak, 2000; Kwak and Ibbs, 2000). However, the ndings varied in terms of which practices are most widely utilized (see Table 1). The underlying assumption in the eld is that the use of commonly accepted PM practices will enhance project performance. Although prior research provides initial evidence of the relationship between use of PM practices and improved project performance, a more complete examination is needed (Thomas and Mullaly, 2007). The few studies that have examined the link between PM practices and project success have been criticized for a narrow focus (a particular process group in PM or a subset of PM practices), a sample size too small to be broadly relevant and statistically credible (Thomas and Mullaly, 2007, p. 74), or mixed results, as indicated in Table 1. Dvir et al. (2003) and Zwikael and Globerson (2004) focused exclusively on planning in PM and had conicting results. Gowan and Mathieu (2005) focused on just ve PM practices. Crawford (2005) reports no relationship between use of PM practices and success, but success was captured narrowly as the supervisors perception of workplace eectiveness of personnel. Ibbs and Kwak (2000) report a positive relationship between PM maturity and process success, but with only 17 and 15 observations for cost and schedule performance, respectively, no strong conclusions could be drawn (Thomas and Mullaly, 2007). Finally, Ling et al. (2009) report 46 signicant relationships out of the 468 combinations of practices and success that were

652 Table 1 Prior PM methods research.a Research

K.E. Papke-Shields et al. / International Journal of Project Management 28 (2010) 650662

What methods

Use p

Relationship to: Context Success p

Findings

Ibbs and Kwak, 2000Kwak and Ibbs, 2000

Specic practices captured in the Berkeley PM process maturity model

Shenhar et al. (2002)

Hundred and seventy managerial variables derived from literature Methodologies, tools, and techniques PM infrastructure (17 items) PM planning quality (PMPQ) index (PMBOK-related planning practices and organization support) PMBOK-related knowledge and practices Five broad practices p

White and Fortune, 2002 Dvir et al. (2003) Zwikael and Globerson, 2004

p p p

Crawford, 2005 Gowan and Mathieu, 2005

p p p

Besner and Hobbs, 2006

Tools and techniques

Crawford and Pollack, 2007 Ling et al. (2009)

PMBOK-related knowledge and practices Seventy eight PMBOK-related practices

p p

Variation in maturity based on knowledge area (highest to lowest: cost, communications, scope, time, procurement, HR, quality, risk) Positive but not signicant relationship between PM maturity and cost or schedule index Ninety six of 170 managerial variables signicantly related to project success Seventy six managerial variables signicant depending on project context (complexity or uncertainty) Substantial variation in frequency of use of methodologies, tools, and techniques No signicant relationship between PM infrastructure and success Variation in PMPQ scores based on knowledge area (highest to lowest: integration, time, scope, HR, cost, procurement, quality, risk, communications) Positive relationship between PMPQ and success as cost, schedule, technical performance and customer satisfaction No signicant relationship between knowledge or practices and perceived workplace eectiveness Positive relationship between project size and PM practices; no relationship between project size and meeting target date Positive relationship between PM practices and meeting project target date Substantial variation in level of use Dierences depending on project size, PM maturity, project duration PM knowledge varied by country and application area PM practices varied by industry Forty sis signicant relationships (representing 23 practices) out of 468 combinations of practices and success (budget, schedule, quality, owner satisfaction, protability, public satisfaction)

a Practices research included addresses PM as a whole rather than specic areas, such as risk management or HR management, or practices specic to one industry, such as the construction industry.

tested. Further, their study focused on operations of Singaporean rms in China, limiting generalizability. The internal organizational and external environmental context in which a project is executed can also inuence both project processes and outcomes. Traditional thinking (Gowan and Mathieu, 2005, p. 237) suggests that large and/or complex projects require more formalized PM methods to control the projects. For example, formal PM methods can mitigate the negative impacts of project size and complexity on project success (Gowan and Mathieu, 2005). Which PM tools are valued, although mostly consistent across contexts, can vary with project size and duration (Besner and Hobbs, 2006). Industry type can also aect the use of dierent PM practices (Crawford and Pollack, 2007). Thus the context of the project needs to be considered when examining the use of PM practices and project outcomes. 2.3. Research questions Our study addresses these issues by focusing on the actual use of specic practices based on formal standards

as well as the link between usage and project success. The following research questions guided our study: (1) Which practices within the formal PM standards are being used by practitioners and is there variation in the extent of use of dierent practices? (2) Does the use of PM standards vary with the context in which the project exists? (3) Is there a relationship between the use of PM standards and project success? In order to address these questions, we rst developed a set of metrics based on A Guide to the Project Management Body of Knowledge (PMBOK Guide) (PMI, 2004)1 reecting four of the ve dierent formal processes of a project (initiation, planning, executing, controlling closing was not included). We then used this set of metrics to examine the relative use of dierent PM practices, the relationship

1 PMI has recently released the newest version of PMBOK Guide, 2008. Our study was done prior to this release and is based on the 2004 version.

K.E. Papke-Shields et al. / International Journal of Project Management 28 (2010) 650662

653

between use and context, and nally, the relationship between use and success. 3. Measures Past research provides us with potential measures for a number of the constructs in these research questions. The context has been captured previously in terms of the project itself (e.g., project size by Saeed Bani Ali et al. (2008), complexity or technical uncertainty by Shenhar et al. (2002)), the organization (e.g., organization size by Hyvari (2006)), and the external environment (e.g., indus try by Crawford and Pollack (2007), culture by Zwikael et al. (2005), country by Crawford and Pollack, 2007). Of the numerous aspects of each of these levels of context, we selected one from each that has been used in prior work. The context of the project is commonly captured as project size, which has been measured by the project budget, number of people on the project, and project duration (Hyvari, 2006; Saeed Bani Ali et al., 2008). The organization context is also commonly measured as the size of the organization, which has been measured as the number of individuals employed and the sales of the organization (Hyvari, 2006; Saeed Bani Ali et al., 2008). Finally, the external context has been captured as the industry (Crawford and Pollack, 2007). We capture the industry using the North American Industry Classication System (NAICS). 3.1. Project success Traditionally, measures of project success reected the three aspects of the triple constraint or iron triangle (Atkinson, 1999): cost, time, and quality/performance. These dimensions are still considered central to measuring project success. Agarwal and Rathod (2006, p. 360) state: cost, time, functionality and quality remain the important criteria for assessing performance of software projects in the mind of professionals and have been used in numerous studies, either in conjunction with other measures or alone. However, there has been general agreement that project performance goes beyond these aspects (Baccarini, 1999; De Wit, 1988; Munns and Bjeirmi, 1996; Morrison and Brown, 2004; Pinto and Slevin, 1988; Shenhar et al., 1997, 2001). Much of this work focuses on dierentiating between project management success and project success (Baccarini, 1999; De Wit, 1988; Morrison and Brown, 2004; Shenhar et al., 1997). As explained by Munns and Bjeirmi (1996), the primary dierence lies in the denition of project achievement of a specic objective, which involves a series of activities and tasks which consume resources (p. 81) compared to project management the process of controlling the achievement of the project objectives by applying a collection of tools and techniques (p. 81). Thus PM success is considered measurable during and at the end of the project (including meeting budget, schedule, technical and quality goals) while project success

goes beyond that in focusing on longer-term and customeroriented results. Important dimensions in project success include customer satisfaction (Dvir, 2005; Pinto and Slevin, 1988; Shenhar et al., 2001; Thomas and Fernandez, 2008) and/ or business success (Atkinson, 1999; Freeman and Beale, 1992; Pinto and Slevin, 1988; Shenhar et al., 2001; Thomas and Fernandez, 2008; Wateridge, 1998). Customer satisfaction has been widely used (Belassi and Tukel, 1996; Milosevic and Patanakul, 2005; Ling et al., 2009; Raz and Michael, 2001; Zwikael and Globerson, 2004). Business success generally includes measures of prot (Atkinson, 1999; Ling et al., 2009), market share or a related measure of commercial success (Atkinson, 1999; Shenhar et al., 2001), and meeting organizational objectives (Thomas and Fernandez, 2008; White and Fortune, 2002). The appropriate measure of project performance diers depending on the progress of the project. For example, according to Shenhar et al. (1997) eciency (meeting budget and schedule) applies during and immediately following execution of the project, impact on the customer (meeting operational and technical specications, customer satisfaction) applies immediately following the project up to a few months after delivery to the customer, direct success (commercial success, market share) is applicable 1 2 years after project completion, and future potential (opened a new market, developed a new technology) is the long term and not applicable until usually 35 years afterward. We incorporated aspects of PM success (meeting cost goals, time goals, technical specications, and quality requirements) and project success (achieving client satisfaction and achieving business objectives). We selected achieving business objectives as a general indicator of business success, encompassing protability and market share. Given that respondents were asked about projects within the last two years, these measures are appropriate since they are associated with shorter- and medium-term success (Shenhar et al., 1997). 3.2. PM practices As argued above, current standards now encapsulate the PM knowledge gained through practice and research over the past several decades. In this study we developed a list of items derived from the standards captured in the PMBOK Guide (PMI, 2004). There are several reasons for this choice. First, our items are an extension of those developed by Zwikael and Globerson (2004), who based their planning products on PMBOK. In addition, a number of other studies also opted to use practices based on PMBOK (e.g., Ling et al., 2009). Further, we had the cooperation of and access to a large regional chapter of PMI. While recognizing the dierences between PMBOK and other standards, we think any one particular set of standards is likely to capture a majority of what are considered best practices across the global PM landscape.

654

K.E. Papke-Shields et al. / International Journal of Project Management 28 (2010) 650662

We used the product or artifact from a particular PM practice, rather than the practice itself, following the work of Zwikael and Globerson (2004). That is, rather than asking respondents whether a particular PM activity had been undertaken (e.g., if scope planning had been done), we focused on whether the corresponding artifact had been generated (e.g., the frequency with which a scope plan was produced as part of a project). Thus, our survey uses each of these artifacts as evidence that a particular practice was applied. Zwikael and Globersons (2004) work was limited to planning; we extend it by including items addressing the initiating, executing, and controlling processes (see Table 2). In order to limit the survey to a feasible length, we opted to leave closing processes for future research. 4. Survey The initial list of 58 PM items was distributed to six local project managers who were asked to indicate whether they understood the nature of each item. Based on these results, 6 items were dropped because at least half the project managers were unsure of the meaning. A total of 52 items, representing the artifacts of applying specic PM practices, were then used in the nal survey (see Table 2 and Appendix). Respondents were asked to indicate the frequency with which each output was created for projects in which they had been involved during the last two years. Respondents also supplied information about the industry type based on NAICS, organization size captured as sales volume and number of employees, and the project size captured as typical cost, duration (number of months) and number of people. Our sample was drawn from a large regional chapter of the PMI in the eastern United States. A link to the survey was posted on the chapters web site. In addition, the chapter leadership sent initial and follow-up e-mails to all members of the chapter with information about the survey and a link to the survey. A total of 142 responses were obtained, representing approximately 10% of the chapter membership. A test for potential non-response bias that compared early and late responders (Armstrong and Overton, 1977) found no signicant dierence for any of the contextual or performance variables. 5. Results The respondents represented a variety of industries and company sizes and were involved in projects of varying costs, durations, and number of people (Table 3). Each individual PM practice was used to some extent (Table 2). Using the average frequency of artifact generation, practice use ranged from a high of 4.68 for project schedule to a low of 2.17 for HR change requests. This range is similar to the degree of variation observed by Besner and Hobbs (2006).

Given that our list of PM artifacts is an extension of Zwikael and Globersons (2004) list of planning products, we rst compared our results to theirs. To do this we identied the individual planning artifacts in the current study that matched those reported by Zwikael and Globerson (2004). In those cases where they used a single entity to represent a knowledge area, we used the average of the artifacts in that knowledge area. We compared the ranking of the dierent knowledge areas across the two studies and found exact matches in rank order and magnitude of dierence in all cases except the two lowest ranking areas, which were simply reversed between the two studies. For all analysis, the PM practices were evaluated rst based on knowledge area (e.g., integration, scope or time), consistent with previous studies (Crawford and Pollack, 2007; Ibbs and Kwak, 2000; Zwikael and Globerson, 2004). The internal consistency of the measures was assessed via Cronbachs alpha. In all cases the coecient alpha provided support for the internal reliability of the measures, although the items for integration were slightly below the generally accepted cut-o for new measures (Table 2). Following the analysis by knowledge area, each PM practice was examined individually to gain a deeper understanding of both relative practice use and the relationship to success. Status review meetings was an exception to this approach. It was originally included as an aspect of risk management but had a relatively low item-to-total correlation when included with the other risk practices. Further examination of the correlation between this practice and all other practices showed a somewhat stronger relationship with practices in other knowledge areas including time, cost, HR and communication. This is not surprising given that the use of status meetings is mentioned in these other areas in PMBOK as a time to review dierent aspects of the project or promote the team environment and communication. Thus, status review meetings was treated separately for all analysis. The individual measures of success, each representing a dierent aspect of performance as discussed previously, were initially combined into a single index. This approach was used based on both the strong correlations between the dierent dimensions (Table 4) and prior use in research (Milosevic and Patanakul, 2005; Raz and Michael, 2001). The Cronbachs alpha for this construct demonstrated good internal consistency among these dierent dimensions of success (a = 0.82). Consistent with past research (Dvir et al., 2003; Ling et al., 2009), additional analysis was done for each of the individual dimensions of performance to gain a better understanding of that relationship. 5.1. PM practice usage The results show the greatest use of PM practices associated with time, scope, and cost knowledge areas, while practices associated with integration, HR, and procurement are utilized moderately, and those associated with

K.E. Papke-Shields et al. / International Journal of Project Management 28 (2010) 650662 Table 2 PM practices knowledge area, process group, and individual frequency of use. Knowledge area Integration PM practice Project plan Project charter Stakeholder analysis Feasibility study Project deliverables list Scope statement WBS Scope change proposal WBS update Scope statement update Project schedule Schedule update Schedule baseline PERT or Gantt chart Project activities list Activity duration estimates Activity list update Cost baseline Cost estimate updates Cost performance reports Activity cost estimates Cost baseline updates Time-phased budget plan Quality checklists Dened quality metrics Quality management plan Quality metric results Quality audit Quality change proposals Project sta assignments Roles and responsibilities list Responsibility assignment matrix Team-building event HR change requests Communication management plan Information gathering and retrieval system Information distribution plan Communication requirements analysis Communication change request Risk management plan Contingency plan Risk register Quantitative risk analysis Risk register updates Pre-planned risk response mechanism Contract statement of work Bid documents Supplier proposal evaluation Supplier evaluation criteria Procurement management plan Status review meetings Process groupa P I I I P P P M&C M&C M&C P M&C P P P P M&C P M&C M&C P M&C P P P P E E M&C P P P E M&C P E P P M&C P P P P M&C E P E E P P M&C Average useb 4.47 3.49 2.82 2.75 4.40 4.27 3.78 3.55 3.28 3.25 4.68 4.32 4.04 3.78 4.25 4.16 3.64 3.84 3.69 3.49 3.45 3.39 3.29 3.08 3.07 3.01 2.87 2.80 2.46 4.07 3.74 3.34 2.88 2.17 3.39 3.30 2.92 2.65 2.44 3.31 3.13 2.60 2.59 2.51 2.43 4.00 3.18 3.02 2.96 2.87 4.64 3.36 Construct mean and std. dev 3.41 (0.82)

655

Coecient a 0.69

Scope

3.76 (0.65)

0.80

Time

4.03 (0.69)

0.77

Cost

3.51 (1.05)

0.89

Quality

2.90 (1.03)

0.89

Human resources

3.27 (0.85)

0.80

Communication

2.97 (0.99)

0.83

Risk

2.79 (1.03)

0.89

Procurement

3.24 (1.08)

0.83

Risk Overall
a b

4.62 (0.54)

na

I = initiating, P = planning, E = executing, M&C = monitoring and controlling. 1 = never, 2 = seldom, 3 = sometimes, 4 = frequently, 5 = always.

communication, quality, and risk are used less frequently (Fig. 1). Kwak and Ibbs (2000), looking at PM maturity

in 38 organizations, had similar ndings in that time, scope and cost were highest and quality and risk were lowest. The

656

K.E. Papke-Shields et al. / International Journal of Project Management 28 (2010) 650662

Table 3 Context: industry, organization size and project size. Context Industry (NAICS) Utilities Construction Manufacturing Transportation and warehousing Information Finance and insurance Professional, scientic and technical services Management of companies and enterprises Education services Health care and social assistance Arts, entertainment and recreation Other services (except public administration) Public administration Organization size: sales volume <$50 M $50100 M $100250 M $250500 M $500 M$1 B >$1B Unsure Organization size: # of employees <100 1001000 10005000 500010,000 10,00025,000 >25,000 Unsure Project size: project cost <$100 K $100325 K $325500 K $500 K$1 M >$1 M Project size: duration (months) 112 1324 2536 >36 Project size: No. of participants 110 1150 51100 >100 Responses 4 6 10 1 29 14 33 1 3 10 1 17 7 34 18 9 9 14 26 22 26 39 19 12 9 23 7 20 19 18 18 47 83 27 10 9 54 47 15 6

limited to extensive use group and those below the average were in their from very limited to limited use group. Status review meetings was one of the most widely utilized PM practices along with project schedule (Table 2) and was the most extensively used tool as observed by Besner and Hobbs (2006). Although there is some variation of use within each of the knowledge areas, the results for individual practices tends to reect the same pattern as by knowledge area. However, there is a greater dierence between the most and least widely utilized HR practices than for other knowledge areas. Three of the HR practices, project sta assignments, roles and responsibilities list, and responsibility assignment matrix, are relatively widely utilized while team-building events and HR change requests are in the lowest third of all practices. Although not as extreme, such variations are also observed for procurement and, to a lesser extent, scope practices. 5.2. Context and PM practice usage Given the nature of the data, ANOVA was used to compare use of each PM practice across the dierent categories for industry, organization size, and project cost, while regression was used to examine the relationship between PM practice use and both project duration and number of people on the project. The results show no signicant dierence in use of PM practices depending on the industry (F-values from 2.20 (p = 0.06) to 0.38 (p = 0.86)). Similarly, there was no signicant dierence in use of each PM practice across dierent organization sizes, measured as either sales (F-values from 1.89 (p = 0.09) to 0.57 (p = 0.76)) or number of employees (F-values from 2.10 (p = 0.06) to 0.22 (p = 0.97)). However, dierences were observed in PM practice use depending on project size (see Table 5). Four PM practices varied signicantly based on all three aspects of project size (cost, duration, and number of people on the project): time, cost, procurement, quality and risk. In addition, two other PM practices varied based on one or two of the context variables: integration and scope. With respect to the project cost, the signicant dierences were between the highest (>$1 M) and lowest (<$100 K) cost categories while those in between did not dier signicantly. 5.3. PM practice usage and project success

current results are also similar to the ndings reported by Zwikael and Globerson (2004) for planning practices where they concluded that integration, scope, time, and human resources were high quality areas, cost, procurement and quality were medium quality areas, while risk and communication were poor quality areas. Examining individual PM practices, the pattern for relative use is similar to that observed by Besner and Hobbs (2006). Of the practices common to the two studies, those practices in the current study with use greater than the overall average were in Besner and Hobbs (2006) from

To examine the possible relationship between the use of PM practices and project success, the sample was divided into three groups based on the success index. Given an overall mean of 3.74 for the success index, a cut-o of 3.5 was used to dierentiate the low and moderate success groups, and a cut-o of 4.0 was used to separate the moderate and high performers. All comparisons were then made between the low and high performers. For all PM practices the average use was higher in the high success group than in the low success group

K.E. Papke-Shields et al. / International Journal of Project Management 28 (2010) 650662 Table 4 Descriptive statistics for success measures. Success measures Mean and std. dev. Pearson correlation coecientsa Cost target 0.59 0.35 0.15 0.46 0.47 0.72 Time target Technical specication Quality requirement Client satisfaction

657

Business objectives

Cost target Time target Technical specications Quality requirements Client satisfaction Business objectives Success index
a

3.54 3.58 3.83 3.55 3.91 4.00 3.74

(1.00) (0.89) (0.89) (0.93) (0.69) (0.75) (0.62)

0.30 0.15 0.47 0.56 0.71

0.52 0.51 0.51 0.74

0.42 0.41 0.62

0.59 0.77

0.80

Bolded values all signicant at p < 0.0005 level.

Table 6 Dierences in PM practice use between low and high success groups. PM Practice Success Low MANOVA Status review Time Scope Cost Integration HR Procurement Communication Quality Risk Fig. 1. Relative use of PM practices by knowledge area. High
b

Dierencea

Pillais trace value = 0.4551 4.29 3.64 3.31 3.03 3.14 2.77 2.93 2.50 2.48 2.42 4.90 4.33 4.19 3.96 3.82 3.73 3.58 3.50 3.32 3.30

0.61 0.69 0.88 0.93 0.68 0.96 0.65 1.00 0.84 0.88

a All dierences between low and high success groups signicant at p < 0.0001 level except procurement, which is signicant at p < 0.005 level. b (Prob > F) < 0.0001.

Table 5 Relationship between project context and PM practice use. PM practice F-valuea Project costb (df = 121) Status review Time Scope Cost Integration HR Procurement Communication Quality Risk
a b

Project duration (df = 128) 2.40 6.36* 2.46 11.04** 4.20* 1.45 11.12** 0.00 5.87* 3.95*

Number of people (df = 121) 2.02 7.76** 2.27 5.76* 7.09** 2.00 6.95** 0.73 7.39** 7.07**

1.62 3.76** 3.03* 4.16** 1.49 0.72 3.57** 0.81 3.32* 3.12*

p < 0.01; p = 0.010.05. MANOVA results: Pillais trace value = 0.5998, (Prob > F) = 0.0008; individual F-values for univariate test.

(Table 6). Given the number of dierent PM practices to evaluate, MANOVA was used to determine if there was a dierence between the low and high groups for all of the PM practices as a set. After ensuring that multicollinearity among the PM practices was not an issue (variance ination factors well below 10 and tolerances well above

0.1), the results of the MANOVA indicate a signicant difference between the low and high success groups for the group of PM practices (Table 6). The univariate results indicate that PM practices in each knowledge area account for the dierence in success. This provides support for the relationship between the use of PM practices and increased project success. In addition to this overall view of the relationship between use of PM practices and success, the relationship between each PM practice and each dimension of success was examined via correlation to gain a deeper understanding of these relationships. Given the large number of relationships being examined (312 combinations), a Bonferroni correction was used to control the experiment-wide Type I error. An overall criterion of a = 0.10 was used, compensating for the conservative nature of this correction. A number of relationships between PM practices and dierent dimensions of success were signicant (see Table 7). To ensure that success was indeed related to use of PM practices rather than the context in which the project occurred, tests were also conducted for the relationship between success and organization size and project size. Given the categorical nature of sales volume, number of employees, and typical cost of the project, ANOVA was

658

K.E. Papke-Shields et al. / International Journal of Project Management 28 (2010) 650662

Table 7 Signicant relationships between individual pm practices and success dimensions. PM practice Status review meetings Project charter Project plan Feasibility study Stakeholder analysis Project deliverables list Scope statement Scope statement update Scope change proposal WBS WBS update Project activities list Activity duration estimates Activity resource requirements Activity list update Project schedule PERT or Gantt chart Schedule baseline Schedule update Time-phased budget plan Activity cost estimates Cost estimate updates Cost baseline Cost baseline update Cost performance report Quality management plan Dened quality metrics Quality metric results Quality checklists Quality audit Quality change proposal Roles and responsibilities list Responsibility assignment matrix Project sta assignments Team-building event HR change request Communication management plan Communication requirements Information gathering and retrieval system Information distribution plan Communication change request Risk management plan Risk register Risk register update Quantitative risk analysis Contingency plan Pre-planned risk response Procurement management plan Contract statement of work Supplier evaluation criteria Supplier proposal evaluation Bid documents Cost target Time target Technical specication p Quality requirement p Client satisfaction p Business objectives p

p p p p p p p p p p p p p p p

p p p p p p p p p p p p p p p p p p p p p p

p p p p

p p p p p p

p p p

p p p p

p p p p p p p

p p p

p p

p p p

used to evaluate their relationship to success. No signicant relationships were found (sales volume: F = 0.72 (p = 0.63); number of employees: F = 0.79 (p = 0.58); typical project cost: F = 1.67 (p = 0.16)). The same was found for the continuous variables typical duration of the project and typical number of people on the project success (F = 0.20 (p = 0.66); F = 1.33 (p = 0.25), respectively).

6. Discussion 6.1. Are PM practices being used? The good news is that formal PM practices are indeed being applied in practice. However, they are not being applied equally or consistently across all the knowledge

K.E. Papke-Shields et al. / International Journal of Project Management 28 (2010) 650662

659

areas. Respondents reported that PM practices associated with time, scope, and cost were widely used, practices associated with integration, HR, and procurement were used somewhat less, and nally, practices related to communication, quality, and risk tend to be used least frequently. This is not surprising for a couple of reasons. First, the knowledge areas where practices are utilized most reect the three areas that are the most well-established in PM: time, cost and scope. This triple constraint or iron triangle (Atkinson, 1999) reects the traditional core goals of managing projects. As such, these aspects are most likely to be measured, and, as many practitioners note, what is measured gets the most attention. Further, given the long-standing importance of these areas, seasoned project managers may have more training, experience, and comfort with these traditional dimensions of PM. However, as the eld of PM continues to evolve, what constitutes important goals in PM has been expanding. The quadruple constraint, for example, includes quality along with scope, time, and cost (Schwalbe, 2009). Atkinson (1999) transformed the iron triangle into the square root, adding aspects of the product (maintainability, reliability, etc. of information systems), and organizational and stakeholder benets. Shenhar et al. (2005) propose that it is no longer appropriate to focus on just eciency, operational performance, and meeting time and budget goals (p. 8) and a more strategic perspective is needed. In addition to scope, time and cost being the traditional areas of PM, they can also be characterized as more easily quantiable, harder aspects of PM, while quality, communication, and risk areas could be considered softer, more people-related and/or more dicult to quantify. Cooke-Davies (2002), for example, notes the absence of people factors in historical CSFs for projects. Risk management practices are applied relatively less, which is not surprising given that risk management is frequently overlooked (Schwalbe, 2009, p. 177), perhaps in part because project managers may tend to avoid thinking about what could go wrong in a project. In the future, as PM standards continue to evolve, and as education and training in these areas becomes more widespread, we should expect to see a greater focus on the softer practice areas, as well as better metrics being developed to track them. The fact that practices associated with integration were in the middle range of usage suggests a similar growing understanding that without high-level initial planning (e.g., project charter, project plan, stakeholder analysis), a project is more likely to fail. If we consider individual PM practices, we see that the use of status review meetings, which in our analysis did not t into one of the knowledge areas, was second only to use of a project schedule. In addition, although there is variation among the practices in each of the knowledge areas, the degree of variation is greater for the HR knowledge area. The three practices associated with HR that were used substantially more reect the planning process group, while the other two reect later processes. We found

a similar pattern in other knowledge areas as well. A greater emphasis on up-front planning apparently results in greater use of planning practices, compared to those associated with the other process groups. Although this could not be tested in our study, future research should examine whether there are dierences based on process group. 6.2. Does the context inuence PM practice use? We found that the use of dierent PM practices did not vary depending on the industry type or organization size, consistent with Crawford and Pollacks (2007) ndings that showed no dierences in PM knowledge between industry sectors. However, our study does show dierences in PM practice use depending on the project context as measured by typical cost, duration, and number of people on the project (see Table 5), which is consistent with Besner and Hobbs (2006). Similar to Gowan and Mathieus (2005) ndings for ve broad PM practices, our results provide evidence that larger, costlier projects increase the usage of practices focusing on greater control (time, cost, and integration). There is more at stake with such projects, leading to more formal deployment of quality and risk management practices. Finally, larger projects are likely to need more resources from external sources, leading to greater use of procurement management practices. 6.3. Are PM practice use and project success related? Perhaps most importantly, our study provides empirical support for the assumption that deploying formal PM practices will increase project success. These results vindicate the eorts of professional associations such as the Project Management Institute, the Association for Project Management, the International Project Management Association, and others, in advocating the use of formal PM standards. Further, although PM practices associated with the more established knowledge areas had greater use, on average, the use of practices across all knowledge areas varied signicantly between the low and high success groups. Thus it is not just a few specic areas of PM, such as the triple constraint, that are associated with success. Looking more closely at the relationships between specic practices and dimensions of success (Table 7), we note a number of signicant relationships. Some practices related to the dierent dimensions of success are intuitively obvious (such as use of cost-related practices in meeting cost targets, or use of quality-related practices in meeting quality requirements). However, of note are practices related to scope, integration, risk, HR, and communication, all signicantly related to various success dimensions:  Creating, updating and formalizing changes to the scope statement are important in meeting cost and time targets as well as satisfying clients. These practices likely reduce scope creep and keep everyone aware of any changes.

660

K.E. Papke-Shields et al. / International Journal of Project Management 28 (2010) 650662

 Development of a project deliverables list, another aspect of scope, is related to all success dimensions except meeting time and cost targets. It reects clear realistic objectives (Fortune and White, 2006) and increases awareness of desired outcomes. The use of status review meetings was related to the same success dimensions. Such meetings seem important in keeping the project on track to achieve the project objectives.  Several HR practices associated with a number of success dimensions reect dening who is responsible for what aspects of a project. Clearly dening responsibilities helps keep a project on budget as well as on schedule. It could also help in meeting technical specications because it ensures that everything that needs to get done actually gets done.  Another HR practice associated with several success dimensions is the use of team-building events. This practice enhances group cohesion, which in turn has a number of positive eects on project performance (Jones and Harrison, 1996). For example, a more cohesive team may lead to a better output from the project because the team members can eectively work together on problem-solving.  Communication with customers has been recognized as one of the CSFs associated with PM (White and Fortune, 2002). In our study, gathering and distributing information are related to client satisfaction. It would be dicult to meet cost and time targets, as well as achieve business objectives, without good communication among those involved in the project so there is a clear understanding of where the project stands.  Stakeholder analysis, an aspect of project integration, is associated with meeting quality requirements as well as achieving greater customer satisfaction. Doing a stakeholder analysis allows for a better understanding of who the important stakeholders are as well as their expectations, which should allow the project to better meet the quality requirements as well as satisfy the customer.  Several risk-related practices are associated with a number of success dimensions. Quantitative risk analysis is related to meeting technical specications as well as quality requirements and client satisfaction. This practice allows for a more thorough understanding of potential risks in terms of the possibility that risks will occur as well as the potential impact if they do. This, in conjunction with developing a contingency plan, better prepares for addressing such risks if they arise, so the project can be completed as close to planned as possible. The chance of achieving project success seems to be enhanced by understanding what is involved in the project, maintaining an understanding of any changes to that scope, consistently monitoring the progress of the project, keeping those interested in the project informed, and having an eective team to work on the project. These practices reect several of the CSFs that Fortune and White

(2006) identied as most frequently cited, including clear realistic objectives, strong/detailed plan kept up to date and communication/feedback. In addition, Thomas et al. (2008) found that team building was signicantly related to project success. As PM practitioners note, having a good plan and the right people only works when everyone is aware of what is happening on the project so that adjustments can be made if needed. We also note the apparent absence of relationships between time-related practices and meeting time targets, which is consistent with Ling et al. (2009). Why is not developing a schedule, updating that schedule, etc. not related to improved schedule performance? Additional research will be needed to understand this counter-intuitive result. Further research could also study whether increased use of the softer practices, currently used less, could have a proportionally greater impact on success than the harder practices, currently more commonly used. Finally, PM maturity, although not included in our study, has recently been recognized as a potentially important factor (Morrison and Brown, 2004; Thomas and Mullaly, 2007), is reected in formal standards such as Organiza tional Project Management Maturity Model (OPM3 ) (PMI, 2009), and should be included in future studies. 7. Conclusions Much has been done in the eld of PM in the last several decades, both in terms of practice and research. In practice, a number of standards have been developed that incorporate critical success factors and capture best practices in PM. The assumption associated with these standards is that by following the critical success factors and implementing the associated practices, project performance will improve. The results of research addressing these topics have not been as consistent as might be expected with such an important assumption. This study provides additional support for the link between use of PM practices and increased project success. In addition, this study extends prior work in developing, testing, and validating an extended and reusable instrument that measures PM constructs. Use of PM practices, which was based on formal standards, is based on observable artifacts as was done by Zwikael and Globerson (2004). The measure of the outcome incorporates multiple dimensions of project success, which extend beyond the more traditional triple constraint. Further, the measures were tested with practitioners in the eld who were members of a professional PM standards organization. The fact that the respondents were from a specic geographic region may have limited the variability of project domains, as well as the variability of practices, used that might have occurred with a more heterogeneous population. However, a more homogeneous sample also reduces the eect of potential extraneous factors. The number of respondents, although adequate for the statistical tests applied, may also limit the generalizability of the results. However, the

K.E. Papke-Shields et al. / International Journal of Project Management 28 (2010) 650662

661

consistency of our ndings with prior research increases its validity and potential generalizability. Finally, this study was cross-sectional rather than longitudinal, so although logic leads to interpreting the relationship between PM practice use and project success as practice use leading to success, causality cannot be established. Another contribution of this study is the insight gained into the relative use of dierent PM practices. These results should provide incentive for greater application of less frequently used, softer practices associated with communication, quality, and risk. Finally, the signicant positive relationship between PM use and project success provides empirical support for the assumption that adoption of formal PM practices improves project performance. This vindicates the eorts of professional organizations promoting PM standards, enterprises adopting the standards, and individual practitioners seeking to improve their own skills and knowledge through study and certication. Acknowledgement The authors wish to thank the Project Management Institute Baltimore Chapter for its support of this research. Appendix A. Measurement Items for PM practices and project success A.1. PM practice items In these projects, please indicate how often the following items (see Table 2 for list of PM practice items) were obtained or used. Please select Unsure if you are unsure about what the item in that row means (1) (2) (3) (4) (5) (6) Never Seldom Sometimes Frequently Always Unsure (dropped prior to analysis)

References
Agarwal, N., Rathod, U., 2006. Dening success for software projects: an exploratory revelation. International Journal of Project Management 24, 358370. Armstrong, J., Overton, T., 1977. Estimating nonresponse bias in mail surveys. Journal of Marketing Research 14 (3), 396402. Atkinson, R., 1999. Project management: cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria. International Journal of Project Management 17 (6), 337342. Association for Project Management (APM), 2006. APM Body of Knowledge, fth ed. APM Publishing, Buckinghamshire. <http:// www.apm.org.uk/> (accessed 1.07.09). Australian Institute of Project Management (AIPM), 2009. <http:// www.aipm.com.au/> (accessed 1.07.09). Baccarini, D., 1999. The logical framework methods for dening project success. Project Management Journal 30 (4), 2532. Belassi, W., Tukel, O., 1996. A new framework for determining critical success/failure factors in projects. International Journal of Project Management 14 (3), 141151. Besner, C., Hobbs, B., 2006. The perceived value and potential contribution of project management practices to project success. Project Management Journal 37 (3), 3748. Cicmil, S., Hodgson, D., 2006. New possibilities for project management theory: a critical engagement. Project Management Journal 37 (3), 111122. Cooke-Davies, T., 2002. The real success factors on projects. International Journal of Project Management 20, 185190. Crawford, L., 2005. Senior management perceptions of project management competence. International Journal of Project Management 23, 7 16. Crawford, L., Pollack, J., 2007. How generic are project management knowledge and practice? Project Management Journal 38 (1), 8796. De Wit, A., 1988. Measurement of project success. International Journal of Project Management 6 (3), 164170. Dvir, D., 2005. Transferring projects to their nal users: the eect of planning and preparations for commissioning on project success. International Journal of Project Management 23, 257265. Dvir, D., Raz, T., Shenhar, A.J., 2003. An empirical analysis of the relationship between project planning and project success. International Journal of Project Management 21, 8995. Fortune, J., White, D., 2006. Framing of project critical success factors by a systems model. International Journal of Project Management 24, 53 65. Freeman, M., Beale, P., 1992. Measuring project success. Project Management Journal 23 (1), 817. Gowan, J., Mathieu, R., 2005. The importance of management practices in IS project performance: an empirical study. Journal of Enterprise Information Management 18 (1/2), 235255. Hyvari, I., 2006. Success of projects in dierent organizational conditions. Project Management Journal 37 (4), 3141. Ibbs, C., Kwak, Y., 2000. Assessing project management maturity. Project Management Journal 31 (1), 3243. International Project Management Association (IPMA), 2009. <http:// www.ipma.ch/> (accessed 1.07.09). Jones, M., Harrison, A., 1996. IS project team performance: an empirical assessment. Information and Management 31 (2), 5765. KPMG, 2005. Global IT Project Management. <http://www.kpmg.com/> (downloaded 15.01.06). Kwak, Y., Ibbs, C., 2000. Calculating project managements return on investment. Project Management Journal 31 (2), 3847. Lee, G., Xia, W., 2005. The ability of information systems development project teams to respond to business and technology changes: a study of exibility measures. European Journal of Information Systems 14 (1), 7592. Ling, F., Low, S., Wang, S., Lim, H., 2009. Key project management practices aecting Singaporean rms project performance in China. International Journal of Project Management 27, 5971.

A.2. Project success items In these projects, please indicate how often the following goals were met.       Cost targets Time targets Technical performance specications Required quality standards Satised clients or end-users Business objective (1) Never (2) Seldom (3) Sometimes (4) Frequently (5) Always

662

K.E. Papke-Shields et al. / International Journal of Project Management 28 (2010) 650662 Shenhar, A., Dvir, D., Milosevic, D., Mulenburg, J., Patanakul, P., Reilly, R., Ryan, M., Sage, A., Sauser, B., Srivannaboon, S., Stefanovic, J., Thamhain, H., 2005. Toward a NASA-specic project management framework. Engineering Management Journal 17 (4), 816. Shenhar, A., Levy, O., Dvir, D., 1997. Mapping the dimensions of project success. Project Management Journal 28 (2), 513. Shenhar, A., Tishler, A., Dvir, D., Lipovetsky, S., Lechler, T., 2002. Rening the search for project success factors: a multivariate, typological approach. R&D Management 32 (2), 111126. Thomas, G., Fernandez, W., 2008. Success in IT projects: a matter of denition? International Journal of Project Management 26 (7), 733 742. Thomas, J., Mullaly, M., 2007. Understanding the value of project management: rst steps on an international investigation in search of value. Project Management Journal 38 (3), 7489. Thomas, M., Jacques, P., Adams, j., Kihneman-Wooten, J., 2008. Developing an eective project: planning and team building combined. Project Management Journal 39 (4), 105113. Wateridge, J., 1995. IT projects: a basis for success. International Journal of Projects Management 13 (3), 169172. Wateridge, J., 1998. How can IS/IT projects be measured for success? International Journal of Project Management 16 (1), 5963. White, D., Fortune, J., 2002. Current practices in project management an empirical study. International Journal of Project Management 20 (1), 111. Zwikael, O., Globerson, S., 2004. Evaluating the quality of project planning: a model and eld results. International Journal of Production Research 42 (8), 15451556. Zwikael, O., Shimizu, K., Globerson, S., 2005. Cultural dierences in project management capabilities: a eld study. International Journal of Project Management 23, 454462.

Milosevic, D., Patanakul, P., 2005. Standardized project management may increase development projects success. International Journal of Project Management 23, 181192. Morrison, J., Brown, C., 2004. Project management eectiveness as a construct: a conceptual study. South African Journal of Business Management 35 (4), 7394. Munns, A., Bjeirmi, B., 1996. The role of project management in achieving project success. International Journal of Project Management 14 (2), 8187. Pich, M., Loch, C., De Meyer, A., 2002. On uncertainty, ambiguity, and complexity in project management. Management Science 48 (8), 1008 1023. Pinto, J., Slevin, D., 1988. Project success: denitions and measurement techniques. Project Management Journal 19 (1), 6771. Project Management Institute, 2004. A Guide to the Project Management Body of Knowledge, forth ed. Project Management Institute, Newtown Square, PA. Project Management Institute, 2009. OPM3, second ed. <http://www. pmi.org/BusinessSolutions/Pages/OPM3.aspx> (accessed 1.07.09). Raz, T., Michael, E., 2001. Use and benets of tools for project risk management. International Journal of Project Management 19, 917. Saeed Bani Ali, A., Anbari, F., Money, W., 2008. Impact of organizational and project factors on acceptance and usage of project management software and perceived project success. Project Management Journal 39 (2), 533. Schwalbe, K., 2009. An Introduction to Project Management, second ed. Course Technology Cengage Learning, Boston, MA. Shenhar, A., Dvir, D., Levy, O., Maltz, A., 2001. Project success: a multidimensional strategic concept. Long Range Planning 34 (6), 699 725.