Increasing Project Flexibility: The Response Capacity of Complex Projects

(http://www.chairegp.uqam.ca/en/research/axes/8-les-modes-de-gouvernance-engestion-de-projet.html).

1

Understanding the Response Capacity of Complex Projects

1. Introduction

Since it was realized that the survival and performance of organizations depend on their capacity to adapt to changes in their market, technological, social and regulatory environment, researchers sought to understand what enables adaptation, and why some organizations adapt more successfully than others do. The answers to this question have been set apart by their implicit position on what sociologists call the problem of action and structure (Giddens, 1984; Emirbayer & Mische 1998). On the one hand, answers emphasized the deliberate action of individuals with singular qualities such as foresight and leadership (Cockburn & Henderson, 2000) or the purposeful, ad hoc self-organizing by groups of individuals (Jarzabkowski & Seidl, 2008). On the other hand, successful adaptation is attributed to enduring structural properties in the organization (Teece et al., 1997; Rindova & Kotha, 1998), in the form of centralized routines (Zollo & Winter, 2002) or distributed processes (Eisenhardt & Martin, 2000). However, these theorizing efforts have often turned into fruitless debates, not in the least because in ongoing organizations it is difficult to disentangle deliberate, creative action from structurally conditioned reaction.

However, a class of organizations enables researchers to observe, distinguish, and understand the interaction of deliberate forethought and structural conditioning. This class consists of complex projects, such as those designing and building major airport terminals, high-capacity extraction or energy-production facilities and land transportation infrastructures, as well as those developing information systems, pharmaceutical drugs, and highly innovative products. These projects are sizable enough and different enough from other projects and from the normal activities of sponsor organizations to require a special planning effort, which involves the design of dedicated organizational and contractual structures. Because of the unique nature of projects, planners aim to start with a clean slate and break with past routines when designing such structures. Despite taking a fresh start, the duration and number of participants in such projects are high enough to sustain endogenous processes that lead to emergence of new, project-specific routines and other unintended structural elements. Unlike in ongoing organizations, where routines can be lost in the fog of history, the complex structures of projects can be traced to decisions made in planning as well as to other occurrences. In addition, most complex projects have to respond at some time during their execution or early exploitation phases to major unexpected events. Such events force participants to react in ways that are neither planned initially nor considered business as usual. However, the urgency of the situation increases the likelihood that, in spite of a desire to be unencumbered by the past, the response to such events will be influenced obviously or subtly by emergent project structures. Together, project planning, the emergence of specific structures, and the need to respond to unexpected events provide a quasi-experimental setting for disentangling the role of deliberate foresight from that of structural conditioning in the response of organizations to changes in their environment and to other issues with which they are confronted.

Project management research started paying attention to responding to unexpected events after the 1980s and the early 1990s, a period during which practitioners experimented with innovative institutional and contractual arrangements for the development of infrastructure projects, such as project finance, build-operate-transfer (BOT), public private partnerships (PPPs) and turnkey contracts (Miller & Lessard, 2001; Miller, 2000). These approaches shared a belief that long-term contracts could freeze the future for the life of the project and limit the owners’ and financial backers’ exposure to uncertain events that could affect their net revenues. Planning became the design of contractual structures that pass most risks to other participants via different forms of fixed price contracts. While this approach enabled even small entrepreneurial entities to sponsor large infrastructure projects, such structures not only proved difficult to design and negotiate but also revealed that attempting to freeze the future reduces the subsequent ability to respond to unexpected events. The result were many project failures, which were caused not as much by the market or technical issues raised by the unexpected events as by the ensuing conflicts between participants, by the disintegration of project organizations, etc. (Floricel & Miller, 2001).¹

Practitioners responded to this unexpected outcome of anticipatory planning by resorting to the opposite approach. For example, owners agreed to bear most, if not all, risks, and started to emphasize the creation of a collaborative environment rather than the reduction of project costs (Hobbs & Andersen, 2001, Davies, Gann, & Douglas, 2009). However, these approaches, which go back to some extent to more traditional contractual arrangements, raise the possibility of a return to the cost overruns that had plagued complex projects in the past (see e.g., Merrow, 1988) and had stimulated contractual innovation rooted in the tenets of economic sciences. Faced with this dilemma, researchers undertook a concerted effort to lift the veil of the instrumental and prescriptive rationality that project management and economics had cast on these issues and to study the raw reality of the social processes that occur in projects (Lundin & Söderholm, 1995). Researchers attempted to grasp the nature of unexpected events (Hälgren & Wilson, 2008) and the organizational response to them (Söderholm, 2008) to understand how planning for anticipated activities and risks combine with cultivating an organizational flexibility to respond to unexpected events (Verganti, 1999; Floricel & Miller, 2001; MacCormack et al., 2001), as well as to uncover how all these aspects influence the success of complex projects.

These contributions produced numerous insights about the social processes that shape complex projects and their reaction to unexpected events, but their disparate nature precludes them from providing practical guidance to project planners. This chapter attempts to integrate some of these insights in a comprehensive theoretical framework, built around the concept of response capacity, which captures structurally conditioned ability of complex projects to respond to unexpected events. The focus of our theoretical effort, social processes in organizations, was inspired by the evolution of the strategy field, which used to be dominated by instrumental and prescriptive rationality views (Ansoff, 1965; Boston Consulting Group, 1968), but more recently moved toward views that give a much larger place to understanding the organizational processes that occur in firms. In particular, business strategy is now conceptualized primarily as nurturing organizational capabilities, such as routines and competencies, and, more recently, innovative and dynamic capabilities, that give firms a competitive advantage and enable them to sustain this advantage by adapting to changes in their competitive environment (Wernerfelt, 1984; Barney, 1991; Teece et al., 1997). We were inspired above all by the view of dynamic capabilities as embedded across many ordinary organizational processes (Eisenhardt & Martin, 2000) and by a related stream of the innovation management literature (Henderson & Clark, 1990; Dougherty, 2001), which highlights the rigidities induced by such processes. However, because of the particular nature of the organizations that we studied and of our research question, we went beyond the strategy literature and sought to ground our theoretical development in more fundamental sociological theories about the emergence and nature of social structures and their relation to social action (Giddens, 1984; Callon, 1986; Luhmann, 1995).

The response capacity concept and the theoretical framework built upon these premises enable us to argue that organizational structures and unexpected events make planning more subtle than what current instrumental and normative models imply. Simply put, the latter imply that planning consists of programming project activities with as much precision and detail as possible and then creating structures that control the implementation of these activities. The project organization, if considered explicitly, is reduced to the role of a tool for project leaders, and its structure is seen as an iron cage that restricts the participants’ freedom and punishes deviations from the plan. We argue that the project planning and the processes that lead to the emergence of organizational structures, in short, structuring processes, interact in many other ways beside the rigidity of an iron cage in case of unexpected events. In planning, project participants try to jointly build a set of stable frameworks, which enables them to obtain resources, gain legitimacy, and initiate action. In doing so, planners gather or produce knowledge, but also take into account some preexisting rules and principles, as well as various internal and external interests. They also need to present an acceptable front to resource investors and to stakeholders (Goffman, 1958; Flyvbjerg, Holm, & Buhl, 2006) as well as ensure that responsibilities and risks are shared in ways that would ensure project success. However, as the project advances toward execution, new participants who were not involved in the initial planning enter the project organization and unforeseen circumstances occur as well. Participants try to sort out other participants and start to develop working relationships and, eventually, stable interaction patterns, as well as a language with specific meanings, which enables them to understand each other and coordinate their actions (Weick, 1979; Barley, 1986). The result of this process is an evolving social structure, which includes many aspects ignored by planners and others that are more complex than expected (Emirbayer & Mische, 1998). These structures not only supplement but also deviate from the planners’ intent. This means that project organizations become social systems with a significant autonomy from the planners’ will. We argue that the deviation, which takes place during the structuring process, means that the planners’ intent has the unforeseen consequence of channeling project organizations in certain trajectories, often undesirable. In line with Giddens’ (1984) idea that structuring processes are fraught with unintended consequences, we also argue that emerging structures shape the response to unexpected events in ways that deviate from, or even totally contradict, the anticipations included in the plan (Floricel & Miller, 2001). Overall, instead of a planning that impacts outcomes as a rational blueprint, pulling action through a sort of teleological engine (Van de Ven & Poole, 1995), we postulate a trajectory that influences the project success via two deviating processes, which together amount to the structuring detour (Figure 1-1).

Another crucial assumption we make is that project outcomes can be, at least in part, traced back to the emergent organizational structure, and, eventually, to elements in planning. In other words, although largely unintended, the outcome of the structure formation process is not entirely indeterminate (see e.g., Barley, 1986). This traceability holds out the perspective that measures can be taken as early as the planning stage, as well as during project execution, to improve the response to unexpected events and the chances of project success. However, the key for this is understanding the characteristics of the planning process and of the resulting plans that influence the nature of the subsequent structuring processes, as well as the characteristics of the emergent structure, most notably its response capacity properties. These properties shape the reaction to unexpected events in ways that affect the project performance.

By means of this theoretical development, we hope to make two contributions. The first is to provide a framework that moves beyond description and aims to enable prediction and hence the elaboration of practical recommendations for project planners. Among others, we show that some aspects of planning, which are rarely considered directly, may have a significant influence on emergent project structures and on subsequent performance. Second, we hope to contribute more broadly to the fields of strategy and organization theory, by clarifying how certain mechanisms contribute to the emergence of social structures and the role of the actors’ deliberate intent in these processes. The chapter proceeds as follows. In section 2, we explain the theoretical bases by presenting three kinds of mechanisms that contribute to the creation of social structures and by explaining in more detail the overall process described in Figure 1-1. In section 3, we detail the variables that capture two important aspects of this process, planning and response capacity, as well as the project performance. In section 4, we present a series of testable hypotheses about influence trajectories that originate in a certain aspect of planning, pass through aspects of the response capacity, and influence project performance. In section 5, we discuss the main implications of this theoretical development.

2. Theoretical Background

The implicit paradigm that informs the conceptualization of organizational structure in the project management literature could be called instrumental functionalism (Hodgson, 2004; Winter, Smith, Morris, & Cicmil, 2006). Like other functionalist views, it assumes that organizational differentiation—the grouping and subordination of actors, and the breakdown and scheduling of their tasks—together with the means for coordinating their activities, follow hierarchically (top-down) from the imperative of achieving project goals (Packendorff, 1995). Contrary to ecological functionalist authors (see e.g., Nelson & Winter, 1982; Stinchcombe, 1990), who argued that such structural properties emerge from actors’ random trials and nonpurposeful selection by the organizational system, instrumental functionalism assumes that structure is a matter of deliberate rational design informed by normative rules (Kerzner, 2009). A corollary of this view is an emphasis on control, which is rooted in a belief that this design will only achieve project goals if participants’ actions are kept within tight bounds, and random deviations banished. These beliefs led project management research along two routes. One route took up refining the normative rules by accounting for the limited rationality of planners and actors (Simon, 1978; March & Shapira, 1992), and for the uncertainty affecting their activities (Krishnan, Eppinger, & Whitney, 1997; Winch, 2004; Chapman & Ward, 1994). The other, inspired by incomplete contracts (Williamson, 1981; Hart, 1988) and principal-agent (Holmstrom, 1979) theories, focused on improving the control over actors’ behavior, by considering, beside limited rationality and uncertainty, their opportunism and limited ability to assess the skills and monitor the behavior of other actors, as well as the ambiguity of contracts and other ties between them. Practice-oriented contributions, the bulk of this literature, focus on developing normative methodologies rooted in the belief that naming, mapping and arranging project activities in specific ways creates better tools for controlling actors’ behavior (Packendorff, 1995). At least 70 such tools are widely used in project practice (Besner & Hobbs, 2008).

So far, few design rules and tools have been linked systematically to project performance. As the introduction suggests, some novelties may have even harmed performance. Their persistent use and development may be due to self-interested promotion by a strong professional field (Hodgson, 2004) or to symbolic manipulation by project participants (Sapolsky, 1972). As a result, many researchers decided to take a fresh look at projects as organizations (Ludin & Soderholm, 1995). Using a descriptive (non-normative) approach, they produced interesting insights about project processes (Denis, Lamothe, & Langley, 2001), their embeddedness in broader social contexts (Miller & Floricel, 2001; Engwall, 2003), and even about the underlying rationalities (Lawrence & Phillips, 2004). However, in the quest for realism, the early interest in projects as unitary entities was replaced by a focus on two kinds of micro-level processes. One direction, drawing upon the practice turn in organization theory and strategy (Whittington, 1996), describes ad hoc project practices and contrast them with normative models (Soderholm, 2008; Hälgren, 2007). The other process, inspired by the actor-network theory, focuses on the real role of objects, such as the tools mentioned previously as well as documents, drawings, databases, etc. (Carlile, 2002; Sapsed & Salter, 2004; Papadimitriou & Pellegrin, 2007; Sergi, 2009). While further eroding the hold of instrumental functionalism, this micro focus provides little insight about project organizations seen as a whole, leaving normative tools as the only practical guideline. Yet, this growing body of empirical results could become a catalyst for new conceptualizations of projects as social entities, accounting for their spatiotemporal unity without reification,² and rethinking their structures in ways that accommodate social construction, objects, routine interactions, and situated