The current issue and full text archive of this journal is available at www.emeraldinsight.com/0959-3845.

htm

Investigating the design process: participatory design in agile software development

Karlheinz Kautz
Copenhagen Business School, Department of Informatics, Frederiksberg, Denmark
Abstract
Purpose This paper aims to explore a case of customer and user participation in an agile software development project, which produced a tailor-made information system for workplace support as a step towards a theory of participatory design in agile software development. Design/methodology/approach Based on an integrated framework for user participation derived from the participatory design literature the research was performed as a case study and semi-structured, open-ended interviews were conducted with about a third of the development team and with a representative sample of key players and future users in the customer organization. The interview data were supplemented with company and project documents. Findings The paper found genuine customer and user participation carried out by onsite customers and by other operational staff in the form of direct and indirect participation and with functional and democratic empowerment. The onsite customers played informative, consultative and participative roles. The analysis revealed that planning games, user stories and story cards, working software and acceptance tests structured the customer and user participation. This form of user participation supported a balance between exibility and project progress and resulted in a project and a product which were considered a success by the customer and the development organization. The analysis showed that the integrative framework for user participation can also fruitfully be used in a new context to understand what participatory design is and how, when and where it can be performed as an instance of a design process in agile development. As such the paper contributes to an analytical and a design theory of participatory design in agile development. Furthermore the paper explicates why participatory design contributes to the successful completion of the investigated project. By drawing on innovation theory it was found that participatory design in agile development bears the characteristics of a successful organizational innovation. Grounding further explanations in complex adaptive systems theory the paper provides an additional argument why participatory design despite some identied challenges fosters project staff to successfully carry out the agile development project. Originality/value The paper presents an exploratory, empirical study of an understudied phenomenon and contributes to theory building. Keywords Design science, Design process, Participatory design, Agile development, Design, Customer and user participation, Case study research, Software engineering Paper type Research paper

Investigating the design process

217
Received 21 January 2010 Revised 6 May 2011 Accepted 18 May 2011

The author would like to thank the participants of the OMS project for their time and insights in agile development practice. The author is also indebted to Ralf Klischewski who was a member of the original interview team and to Sabine Matook (nee Zumpe) who performed parts of the data analysis with the author as well as to Helen Sharp who directed the authors attention to the body of knowledge and provided literature which documents empirical studies of user participation in agile development. The guest editors and reviewers of this special issue provided valuable feedback on earlier versions of this manuscript, which helped clarify the relation between the empirical ndings and their theoretical grounding.

Information Technology & People Vol. 24 No. 3, 2011 pp. 217-235 q Emerald Group Publishing Limited 0959-3845 DOI 10.1108/09593841111158356

ITP 24,3

218

Introduction Investigating the design process is part of design science research in the information systems eld (Venable, 2006; Wastell et al., 2009). While design science research both deals with the process, the method and the result of design, the information technology artefact, there is a strong tendency to focus on the design artefact and less on the design process (Walls et al., 1992; March and Smith, 1995; Hevner et al., 2004; Germomprez et al., 2007). Bratteteig (2007) therefore calls for studying design practice and emphasizes the need for practice studies, especially with regard to user participation and participatory design. Participatory design (PD) is a design approach in which the participation of people in the co-design of the information systems and information technologies (IS/IT) they are supposed to use themselves is a central tenet (Kensing and Blomberg, 1998). Although user participation can also be discussed in the context of standard product development and with regard to those who have to use the output produced by IT, participatory design focuses on user participation in the development of dedicated, tailor-made IS/IT, which are used in the workplace. In this context Markus and Mao (2004) put forward a need to revisit the concept and to study it in novel environments. Agile software development, although not that different from approaches discussed in the 1980s under the label prototyping (Merisalo-Rantanen et al., 2005), is such an environment. Agile development principles insist that the customer takes control and constantly participates in the development process in a collaborative partnership, which is based on daily interaction between the developers and the customer (Highsmith, 2002). Agility in information systems development according to Conboy (2009) is characterized by the continual readiness of an information systems development method to rapidly or inherently create change, proactively or reactively embrace change, and learn from change while contributing to perceived customer value (economy, quality and simplicity), through its collective components and relationships with its environments, where interface, functional and technical design and the more technical development activities such as coding and testing are much more intertwined then in traditional information systems development (Gross et al., 2008). Dyba and Dingsyr (2008) however nd that agile development is an understudied empirical phenomenon. This is especially true for agile software development and its relationship to user participation and participatory design. This is surprising in light of the fact that Cockburn (2002), a prominent member of the agile movement, has stated how much his approach is indebted to Ehns work, a prominent proponent of participatory design, by explicitly relating to Ehns (1992) writing on Scandinavian design: on participation and skill as a major source of inspiration. On this background we aim at contributing to the three areas design science, participatory design, and agile software development in a consistent way. We pose the research question how customers and users participate in the design activities in agile development in practice and study participatory design in agile development as a particular instance of a design process in information systems development. In this way, we make a contribution to the establishment of a theory of participatory design in agile development, which comprises elements of an analytical, an explanatory, and a design theory (Gregor, 2006).

In the next section we provide a literature review of related research publications and describe the theoretical background for the study, which comprises an integrated framework based on classic concepts of user participation. We then introduce our research method and the setting of our study, present our analysis and discuss our ndings and nish with some conclusions. Related work and theoretical background We are interested in the relation between participatory design activities and agile software development in the development of tailor-made IS/IT, which are utilized in the workplace, thus our literature search concentrates on studies which focus on both areas. First, however, we clarify the concepts of customer and user. Dening customers and users The concepts of customer, client and user have been discussed in the information systems literature (Friedman and Cornford, 1989) since the inception of the eld. Customers and clients are the ones who have ordered the information system, who will pay for it, and whose requirements are initially specied. Customers might also be users of the information system, those who are directly using the information system and deploying it in operation. But, the users, are normally not the customers; customers, and users, therefore are two different stakeholder groups. In the agile literature, the distinction is often not that clear. Some methods, such as Dynamic systems development methodology and, Crystal, explicitly use the concept user, others such as Scrum, use the concepts product owner, and customer, the method Adaptive Software Development, uses only customer, while extreme programming (XP), the method, which has been applied in the case, which we have analyzed, and which will be presented later, uses both customer, and user nearly interchangeable (Highsmith, 2002). We agree with the overall distinction between customer and user and will therefore in the following always indicate whether we are relating to the one or the other. Participatory design and agile development in the literature Three reasons for user participation in the design activities in information systems development are usually given: (1) Improving the knowledge on which information systems are built. (2) Enabling people to develop realistic expectations, and reducing resistance to change. (3) Increasing workplace democracy by giving the members of an organization the right to participate in decisions that are likely to affect their work (Bjrn-Andersen and Hedberg, 1977). Our literature search of the participatory design literature only led to few contributions related to the question of user participation in agile design and development activities. Some research originating in the area of agile development studies the task of the customer and the relationship of the customer and user roles, but less the actual participation of customers and users in the design and development activities of projects which use an agile development approach. These studies have shown that customer representatives might have decision power, but only a limited understanding of the users needs, they might not be the actual users of the software to be developed

Investigating the design process

219

ITP 24,3

220

who in turn might have the necessary knowledge, but not the authority to decide on system features (Robinson and Sharp, 2005). Users rarely take the role of the customers (Martin et al., 2004). The customer role might even been carried out by substitutes from the development organization such as product managers or marketing staff (Robinson and Sharp, 2003). In the only explicit participatory design paper we found on the topic Rittenbruch et al. (2002) provide a conceptual comparison of the agile development method XP (for more information on XP see subsection The case setting), and participatory design. On this basis they developed their own approach, which integrates techniques from both agile development and PD. They tested their method in a research project where the customers were fellow, IT literate researchers to provide a proof of their concept. Their research provides a number of practical recommendations to deal with the from their perspective limited conception of user participation in XP. However, their work does not provide any insights about user participation in practice. Hansson et al. (2006) present a practice study of how to include users in the development of off-the-shelf software and claim theirs is a case of complementing PD with agile development. However, the organization under investigation neither explicitly uses an agile method nor participatory design. The authors map the practices they nd, namely the users feedback on the product through meetings, support sessions and courses onto the two concepts. But they also admit that the users never actually participated in any design activities and that the developers totally controlled the process of selecting any of the made proposals for design and development. Thus, while this is an interesting description of practice and a valuable account of the lack of user participation in the further development of packaged software, the work does not investigate user participation in agile practice. Chamberlain et al. (2006) set out to develop a framework for integrating agile development and user-centered design. Analyzing the similarities and differences between the two approaches they conclude that these are compatible. However, in a practice study which investigated the use of agile methods and user-centered design they identied a clear distinction between designers, who were responsible for user-centered activities and developers, who produced code. They also found no or little customer and user participation, if at all, in the form of users being interviewed and asked for their opinion in limited tests sessions. On this basis they propose ve principles for successful integration of the approaches including an explicit demand for user participation. Unfortunately the study does not provide a proof of concept based on empirical data to verify the viability of the framework. There seems to be a lack of empirical studies of participatory design activities in agile software development. We attempt to contribute to lling this gap and subsequently provide a case study of an agile software development project in which actual customer and user participation took place. We analyze our case based on the following theoretical backdrop. A framework for analysing participatory design in agile development To study participatory design and within the design activities more precisely customer and user participation in agile software development we will apply the concept of user focus introduced by Iivari and Iivari (2006) who distinguish between individual, average and ctive user focus: With an individual focus the designer tries to satisfy

each actual user and emphasizes each individuals capabilities and needs. With a focus on an average user interaction designers typically apply some heuristics or general human factor principles. Focusing on a ctive user HCI specialists or interaction designers base their design (proposals) on personas, which are descriptions of hypothetical archetypes of actual users. Our further analysis will then be based on the following constitutive concepts of user participation: . forms of participation; . user roles in user participation; and . purpose of user participation as presented by Mumford (1983), Damodaran (1996) and Clement (1994), respectively. Clement (1994) argues that the purpose of user participation is user empowerment and he distinguishes between functional empowerment and democratic empowerment. The former means that the users should be able to carry out their work to their own satisfaction and in an effective, efcient and economical manner. Their participation in the design process supports to reach this objective. Democratic empowerment means that they should have the mandate to participate in decision-making in their workplace including the design and development of software and IT-based systems. Damodaran (1996) differentiates three user roles in the design and development process. The user can play an informative, consultative or participative role. As informants users merely provide information about their work and might be the objects of some observation. In a consultative role they are asked to comment on preset design solutions. In a participative role they actively participate in the design process and have decision-making power regarding the solution. Mumford (1983) who provided the rst distinction of user roles further classies two different forms of participation, namely direct and indirect user participation, where the user is represented by some kind of intermediary. Direct and indirect participation are dened through the users direct participation in the project (team) or their direct or indirect contact with project staff from the development organization. The participatory design literature has traditionally advocated workplace democracy, a participative role for the users, and direct participation (Bjerknes and Bratteteig, 1995), while in large parts of the information systems literature, functional empowerment with users in informative or consultative roles, directly or indirectly involved, has been the focus of research (Kujala, 2003). The concepts have separately been used by Iivari (2006) to study user participation in open source software development. We consider them jointly and have integrated them in a comprehensive, analytical framework for understanding participatory design in agile development in the context of the development of tailor-made, workplace information systems. Table I summarizes this framework of user participation, which we apply for our analysis. Research method Our research follows the approach of engaged scholarship (Van de Ven, 2007), which is a participative form of research for seeking advice and perspectives of key stakeholders to understand and theorize about a complicated problem. Given the limited literature concerning our research topic, understanding the participatory

Investigating the design process

221

ITP 24,3

User focus

Individual user Average user Fictive user Direct participation Indirect participation Informative role Consultative role Participartory role Functional empowerment Democratic empowerment

Forms of participation

222
Table I. Integrated framework for user participation

Roles of participating stakeholder

Purpose of participation

design activities and customer and user participation in agile software development, our investigation is based on an exploratory, qualitative, single case study (Creswell, 2003) of an agile software development project. While it is often stated that it is not possible to generalize and certainly not to theorize from a single case study, Walsham (1995) suggests that it is possible to generalize case study ndings among others in the form of a contribution of rich insight. So inspired, we have used our integrated theoretical framework which consists of salient concepts that make up user participation as one background for our data analysis. The research presented is part of a larger project, which aims at understanding agile development in practice and at contributing to sustainable theories of information systems development in general and agile development in particular (Kautz and Zumpe, 2008; Matook and Kautz, 2008). In the context of this paper we focus on the role of the customers and users and how they participate in the design and development process. Our research is based on an empirical case study of a commercial agile development project in a large German public sector organization, called WaterWorks, performed by a German software company, called AgDev, which has specialized on agile development. The empirical data for the case study was collected in semi-structured, open-ended interviews, which were conducted by a team of two researchers in a three days period on the development site. The research team performed 12 interviews with 11 individuals the AgDev project manager was interviewed twice. This included nearly a third of the development team covering project managers and developers with a wide range of experience and skills (for further information on the development team see subsection The case setting), a representative sample of key players and future users in the customer organization. The interviews were tape-recorded and subsequently transcribed. The interview data were supplemented with company and project documents such as method, requirements and release descriptions, as well as project plans. The data collection, the coding of the data, and the data analysis have been guided by the four pairs of values underlying agile development which are: (1) Individuals and interactions over processes and tools. (2) Working software over comprehensive documentation. (3) Customer collaboration over contract negotiation. (4) Responding to change over following a plan (see www.agilemanifesto. org). For the qualitative data analysis a software tool was used.

As stated previously for the research presented here, the data were in particular analyzed with regard to participatory design, customer and user participation during the different development and design activities, which were supported by the applied development method XP. For example the statement But its not just that the subproject manager develops his requirements at his own discretion, he holds a strong contact with the people from his division. He, quasi, sucks the requirements out of the division and carries them into the project was coded in the context individual and interactions and associated with the creation of user stories as part of planning and preparing an iteration, while the assertion Well, at latest when an iteration is nished, sometimes already in the middle of it, or whenever, presentations are run for users. Well, not always in front of many users, but the customer subproject manager gets some people together and says: Here, look, do we develop in the right direction? was coded as relating to working software as a feedback mechanism for the customers and users during the development phase of an iteration. As a part of the analysis we produced an account of our case setting including a structural prole of the investigated project. The structural prole consists of descriptions of the information system under development, of the formalized method (Fitzgerald et al., 2002) as opposed to the method-in-use to be used, of the projects structural context and of the structural characteristics of the involved development team, its members and associated staff such as onsite customers and end users. It is presented in the next section. The case setting: the OMS project The project under investigation was concerned with the development of an operations management system (OMS) for the WaterWorks of a large German city. Founded 150 years ago the organization is now partially privatized with the city council holding 50.1 per cent of the ownership. The system was developed with a web-based graphical user interface and a backend to interface the technical infrastructure as dened by an underlying ERP system. The project was organized in four subprojects to provide IT support for the employees of WaterWorks ranging from customer management to the maintenance of the sewer and duct system. After several attempts of traditional information systems development based on a standard ERP system, which had not led to the desired results the organization opened a tendering process. It was won by a small software company: AgDev. At the time of the project AgDev consisted of about 25 employees, 20 of them being developers, and based its development approach on the agile method XP (Beck and Andres, 2004). The formalized method includes planning techniques for releases and iterations called planning games, user stories and story cards to specify user requirements (in XP formally the customer writes the stories onto simple index cards), onsite customers to support customer-developer communication, short daily stand-up meetings of the whole project team to support team communication, pair programming, re-factoring, collective ownership, continuous integration and testing of code to develop the software proper and tuning workshops to improve the development processes regularly. The company extended the method with some project management processes to cater for larger projects such an elaborate overall project plan, formal reporting mechanisms and a formal contract based on a requirements specication called realization concept which had been produced by the customer.

Investigating the design process

223

ITP 24,3

224

The project was organized in two phases. In a rst 12 months exploration phase prototypes catching requirements and possible solutions were developed. This led to the development of the realization concept by the customer organization and their decision to contract AgDev also for the development of the OMS proper. In this main development phase, a team of about 12 development staff with multiple roles such as project manager, subproject manager, analyst, customer contact, and developer worked onsite in a building owned by WaterWorks. A sophisticated management structure with one subproject manager acting as contact person from AgDev and one subproject manager acting as onsite customer from WaterWorks for each of the four subprojects was, in addition to two overall project managers, one from each company, established. The WaterWorks subproject managers and onsite customers were managers and team leaders in the operational divisions of the application areas and as such also actual users of the OMS. These customers were by and large, however not the whole time, onsite. The project also comprised a varying number of other users, representing operational staff, from the different divisions. These participated mostly in feedback and testing activities as will be described and discussed in more detail in the next section. The WaterWorks staff council had approved the overall project and the set up for the staff participation. When this study was performed phase one had been successfully closed and after a break of over a year due to internal politics at WaterWorks phase two had been going on for four months. Responding to an inquiry call during our analysis the AgDev project manager stated that the project ended ten months later on time and budget with all parts of OMS being operational. The collected data are mostly related to phase two. With this structural prole, which is summarized in Table II in mind, we will now analyze the project in more detail with regard to participatory design and user participation. An analysis of the OMS project The OMS project was described by both the customer and the development organization as a success. The emerging information system afforded, in the words of one of WaterWorks subproject managers, to identify synergies among the various departments and in particular in the duct department it enabled improved planning that resulted in the possibility to dispose cleaning vehicles and reduce related staff. With regard to the focus of this paper one of the WaterWorks subproject managers explicated that their end users had been very satised. However, various comments were made about reaching the right balance of customer collaboration and user participation with regard to the degree of agility, project progress and product quality. AgDevs project manager commented on the onsite customer behaviour and said: Yesterday he said something and today he says something else. Requirements have to be clear at the beginning of an iteration, and cannot change right in the middle of it. We are agile, but not on a daily or hourly change request rate. To this end, customer and user participation took place on an ongoing basis during what the AgDev project leader himself, called preparation, development, and evaluation of project iterations. The planning games and story cards, the (presentation of) working software as well as the acceptance tests structured in parts the continuous day-to-day-contacts, communication and collaboration. In addition, the project showed one very particular instance of user participation. One

Information system

Operations management system (OMS) with web-based graphical user interface and ERP back end by a small development company (AgDev) for the employees of a large organization (WaterWorks) XP: short releases and iterations of three to six months/three to six weeks planning games, user stories, story cards, onsite customers, pair programming, collective ownership, stand-up meetings, continuous integration, testing, re-factoring Supplemented with further management processes (e.g. overall project plan, formal reporting) Two overall project managers (one AgDev, one WaterWorks) AgDev: up to 12 staff with multiple roles: Project manager, analyst, customer contact, and developer highly motivated and educated, varying XP experience Four subproject development leaders also as customer contacts, analysts, developers WaterWorks: four customer subproject leaders also as customer and operational staff user representatives At least one additional operational staff user representative for each subproject mostly, however not the whole time onsite Agile development method had been clearly communicated to customer, AgDev won a tender General requirements document as basis for contract after failed ERP implementation Four subprojects (CRM to sewage cleaning) AgDev: Limited experience with large agile development projects WaterWorks: 1. agile development project Poject team onsite in a WaterWorks building

Investigating the design process

Formalized method

225

Involved development team and developers

Structural context

Table II. Structural prole of the OMS project

AgDev subproject manager reported on one user who based on his background and experience had become a full member of the development team. As we focus on the design process we will not discuss this phenomenon any further. In the following we distinguish between types of project participants from WaterWorks, namely onsite customers who were managers and team leaders with both managerial and operational assignments and full-edged project members as well end users of the OMS and other WaterWorks operational staff who were also users of the OMS, but only intermittent project participants, and we use the planning games, user stories, and story cards, the (presentation of the) working software and the acceptance tests to arrange our description and analysis of the user participation in the projects development and design activities. Planning games, user stories and story cards At the start of phase two a number of different documents existed which were all comparable short and concise. From a customer perspective these were related as follows: An overall realization concept built the basis for the development contract with the customer. The realization concept was rened into requirements lists. These lists governed what should be the outcome of an iteration, and what should be the basis for the acceptance tests. Individual requirements or groups of requirements were then described as a story and each story was written down on a story card. The story cards

ITP 24,3

226

represented the nal detailed plans and specications for the developers work tasks and processes. The planning games, at the beginning of each iteration, were based on the overall realization concept, and the requirements lists. These were largely produced by AgDev, both their project manager, and the subproject managers. They developed these documents with input from the onsite customers. The story cards were produced and estimated by the AgDev subproject managers and developers in developer team work sessions, because the WaterWorks subproject managers as well as other possible end users at WorkWorks were mostly operational staff which had some impact on their abstraction capabilities and their willingness and capacities to write conceptual texts. Yet, the AgDev subproject managers and developers and the onsite customers together prioritized the cards. In this context the WaterWorks subproject managers saw their role as facilitators and communicators. To back up the development of the OMS based on an agreement with the staff council and with management, some of them had been assigned fulltime to the project to be available and involved in the project whenever necessary. The WaterWorks subproject managers did not develop the requirements at their own discretion, but held a strong contact with the employees in their divisions and carried the requirements from their divisions into the project. They also prepared the prioritization of the requirements according to their importance and the available budget. An AgDev subproject manager talked about the difculties of converting the requirements into design and declared that making design proposals was the task of the AgDev subproject managers and developers: [. . .] it nearly becomes our design task as contact partners [. . .] its not easy to nd out from the WaterWorks people what they want; when I say do you want it this way, they say yes, and when I ask do you rather want it that way, they also say yes. [. . .] and they say we have this and this problem, but to design an interface out of this information is our problem. However, the design was always developed with close participation of the WaterWorks subproject managers and other users and always under the mandate of the WaterWorks subproject managers. While the AgDev subproject managers had the liberty to make proposals the onsite customers could always say no, and with really important issues, AgDev would always get back to the onsite customers before they would go forward. This also included direct contact between those developers who did not act as contact persons or subproject managers and the onsite customers. Some AgDev subproject managers even felt that through the WaterWorks engagement the project participants from the WaterWorks somehow developed the OMS themselves. One of them thus explained that AgDev had not developed something which they had invented themselves and without communication with WaterWorks. This form of customer collaboration provided some structure to cope with the complexities of a comparatively large agile development project, while leaving room for less structured, but necessary collaboration as well. That is to say, when implementing the story cards, it became obvious that some additional collaboration was needed. One WaterWorks subproject manager stated that contact with the onsite customers was necessary for nearly every story card. He put forward that maybe 60 per cent of a cards contents was clear, but that the other 40 per cent had to be directly coordinated 20 per cent in the middle of an iteration and 20 per cent in the end when the onsite customers and additional operational staff end users

looked at the working software and found that their requirements werent understood the way they meant them. This illustrates the importance of the working software for the development and design process, which we will discuss in more detail in the next subsection. Working software The presentations of working software were identied as a signicant basis for customer and user participation. In the OMS project a rst software release was provided after three months with the others to be delivered every three to six months. Each release was organized in iterations of three to six weeks duration, meaning that at the time of our investigation each subproject had at least gone through two iterations. Feedback about and change requests for the software design were brought forward by the onsite customers in weekly feedback loops which were built into an iteration based on presentations and demonstrations of working software. This always led to smaller changes. The AgDev project manager described how the working software, which was produced story card by story card attracted the WaterWorks subproject managers and how they seamlessly participated in the development process. He conrmed that beyond the scheduled weekly meetings for all subproject managers some of the WaterWorks subproject managers turned up at the project nearly on a daily basis while others came at least on a regular basis and looked over the shoulders of the developers. This allowed for xing problems and providing working software already, before an iteration was ofcially released. This was very important for the WaterWorks subproject managers, as they continuously could see progress. The developers however had to get acquainted to this close interaction as they were not that used to sitting at the customer site and having customers pass by every day, which they sometimes considered as quite disturbing. Beyond these contacts with the onsite customers the working software was also presented, as presentations to one onsite customer were not considered sufcient, to larger groups of prospective operational staff end users. Thus the set up with at least 4 subproject managers who also acted as onsite customers was supplemented with other user representatives. The AgDev project manager summarized the situation: Well, at latest when an iteration is nished, sometimes already in the middle of it, or whenever, presentations are run for users. Well, not always in front of many users, but the customer subproject manager gets some people together and says: Here, look, do we develop in the right direction?. In addition, using a similar format, the onsite customer representatives regularly performed road shows with the working software in the user departments to collect feedback and ideas and proposals for improvements. The AgDev subproject managers also sought direct contact with the operational staff end users and one of them reported that he had seated himself for two weeks in the duct operation station with the objective to extensively put the software on trial onsite and to look how well it actually tted the operation. This resulted in the direct participation of those employees onsite who arranged or actually performed the cleaning the ducts. Another AgDev subproject manager had chosen the same strategy and even engaged some of his developers in the process. After installing a release at one division, they went there several times for some days, discussed with the users, registered bugs, and then re-built the software accordingly.

Investigating the design process

227

ITP 24,3

228

The frequent feedback loops were taken very serious and immediately responded to with action. This had the effect that minor misunderstandings were caught and dealt with as changes early before they could grow into larger problems. As a consequence the onsite customers and the other operational staff end users developed trust and a feeling that they had an impact on the development of the information system and even the employees in the divisions who did not directly participate in the development team got quickly integrated into the project. But the working software also brought to light some initial problems related to the distribution of roles in the project. The WaterWorks subproject managers expected that AgDevs developers would bring more of their own ideas into the project and that they would come up with smart technical solutions. They were frustrated about that the AgDev developers always just did what the onsite customers told them to do, because they saw this as a sign that the developers were not competent enough to develop their own proposals. After a clarication of the roles in an agile development project the cooperation between the different groups of project participants then continued without further problems. Acceptance tests The AgDev project manager explained that in the project between two iterations there was always a test phase which was a post activity of the preceding iteration. He conrmed that the acceptance tests were run by the onsite customers, meaning that the onsite customers had the responsibility, and decision power, for, and in, these tests. The tests were similar to the formal presentations of the working software, but they were performed according to a protocol and they always comprised end users. Thus, customer and user participation also took place during and in form of the acceptance tests. The requests for changing the software design which came up during the scheduled acceptance test sessions were dealt with in the next iteration. Before an acceptance test was performed less formal preparations took place often triggered by a WaterWorks subproject manager who would try out the new functionalities as described on the story cards. As valid feedback was considered important operational staff end users, not just the WaterWorks onsite customers as represented by the subproject leaders, were performing the tests. An acceptance test, was then always led by a responsible WaterWorks subproject leader, who also approved, or rejected, the new version of the system. A typical acceptance test lasted just one day where four to six people participated. Two or three divisional managers and other employees who owned the task and had to work with it, but who were not members of the project team were present and tested. There was an ofcial test protocol where the requirements for the iteration were listed. There the responsible WaterWorks subproject managers approval, conditional approval or rejection as a test leader was documented together with what was missing, and where, to achieve a full approval. The AgDev subproject managers were always present during the tests which otherwise as mentioned previously were the responsibility of the WaterWorks. The AgDev subproject managers also monitored the acceptance test and recorded the errors and bugs on private bug lists. Other developers usually did not participate. There were however exceptions from this rule as we learned from an AgDev subproject manager [. . .] in the end, we participated in the test, me, our project manager and the respective WaterWorks subproject manager and in this case another developer as well.

One AgDev subproject manager described that this way many mistakes were found and that encouraging feedback was provided. The employees involved in the tests stated that they could imagine to work with the system, and that they liked it better than the earlier proposed ERP-based solution. The WaterWorks subproject managers declared that this approach with the acceptance tests before an iteration was approved and the iterations as such helped them to quickly and regularly get something tangible and to get in touch with operational staff end users. They conrmed that in this context the test sessions, which were also coordinated with the staff council, were decisive. Yet, despite the general positive reception of the acceptance tests, still more participation and more tests were also demanded as can be seen by the following statement made by one WaterWorks subproject manager: The tests were good, but there were too few [. . .]. By and large, however, the WaterWorks subproject managers were content with this form of participation. It gave them the feeling of being a full and vital part of the development team and process. They also reported on content operational staff end users and based this appraisal on the feedback they had received from those after the tests. As such, the acceptance tests had a signicant inuence on the further design of the system components as request for changes and new requirements always came up as a result of a test and were dealt with in the next iteration. Table III summarizes the roles and forms of user participation in the OMS project for the two types of project participants from WaterWorks, namely onsite customers and other operational staff. Discussion Prior research into user participation in agile development has shown that end users rarely take the role of the customers (Robinson and Sharp, 2003, 2005; Martin et al., 2004). In contrast we found in our case genuine customer and user participation in the design activities in an agile development project. We empirically conrm and extend
Preparation Onsite customers Participative role: in requirements and story card prioritization in design approval Informative role: in requirements list elaboration Direct participation Operational staff Informative role: in requirements list elaboration Indirect participation Development Onsite customers Participative, consultative, informative roles: during daily informal visits weekly feedback meetings during working software presentations Direct participation Operational staff Consultative and informative roles: during working software presentations in trials in the divisions during road shows in the divisions Direct and indirect participation Evaluation Onsite customers Participative and consultative roles: in preparation and performance of acceptance tests Direct participation

Investigating the design process

229

Operational staff Consultative role: during performance of acceptance tests Direct participation Table III. User participation roles and form during the OMS project

ITP 24,3

230

Chamberlain et al.s (2006) conceptual work that agile development and participatory design are compatible. Our analytical, integrative framework of user participation supports the description of what participatory design is and how, when and where in agile development it can and is carried out: . In the OMS project the focus was not on any average user nor was it on a ctive user. It was on the actual onsite customers and operational staff end users either as individuals or as individual representatives of a professional stakeholder group. Our empirical data show that they even if they were not always content with the fact that they had to take that role through their continuous feedback acted as designers themselves. . In terms of forms of participation (Mumford, 1983) in the OMS project we found both direct and indirect participation: the onsite customers were WaterWorks staff who themselves would work with the future system to a certain extent. As such they exerted direct participation. The operational staff indirectly participated when they commented during presentations or when they provided their viewpoints or descriptions of their work processes to the onsite customers. They however also participated directly when they tested the results of the iterations or when they were observed or conferred with the onsite customers who were in contact with them directly in the divisions as for example the developers who visited and stayed in the duct net division while the result of an iteration was in operation. . Analyzing the different user roles (Damodaran, 1996) we see that those operational staff users in the divisions who were not also onsite customers had the role of informative or consultative users. They provided both the onsite customers and the development staff at AgDev with information about their work, their needs and their preferences. Some of them were observed during their work by some developers and they provided their comments during presentations and tests. This information and comments were taken into account and many of them were realized. The onsite customers had a decision making mandate. They, thus, as so far as their positions as future users were concerned, were clearly in a participative role although they did not themselves write the user stories and the story cards. However, no design decision could be taken without their agreement. Beyond their participative role, the onsite customers also had informative and consultative roles. . Regarding empowerment (Clement, 1994) we can determine that functional empowerment was achieved as all stakeholder groups at the customer organization reported that they were content with the outcome of the project. With regard to democratic empowerment we nd the customer organizations staff council had some authority and exerted power through the approval of the overall project and of the set up for the staff participation and the onsite customers had the authority to take decisions concerning their staffs and their own workplaces. Wastell et al. (2009) strongly argue that a subset of design theory concerns the efcacy and aptness of design methods. As such the presented work also adds to the studies and theory of the design process in agile software and information systems development. As a contribution to the establishment of a design theory of participatory

design in agile development our proposition consists of all structural elements, which determine a theory (Gregor, 2006): . as a means of representation it comprises a verbal description including a prescriptive statement of how to execute participatory design in agile development to achieve a successful project and product; . it includes constructs such as user focus, user role, form of user participation, user empowerment; . it contains statements of relationship, for example that participatory design leads to successful projects in terms of scope, quality, resources and time; and . it denes as a scope participatory design in agile software development for tailor-made workplace information systems. Wastell et al. (2009) also contend that design science generates knowledge of direct practical relevance and in doing so narrows the gap with practice that is acknowledged as imperative for the future viability of the information systems discipline. In this respect our work has practicable bearings, too. It shows how actual user participation in the design activities in agile development can be organised in a project to result in a process and product that all stakeholder groups appreciate. All involved stakeholders in the case were satised with the outcome of the OMS project. The identied form of user participation supported the achievement of a balance between exibility and project progress and resulted in a in project which was considered successful in terms of scope, quality, resources and time by both the customer and the development organization and a product which enabled the users to carry out their work to their own satisfaction and in an effective, efcient and economical manner. In excess to our contribution to a descriptive, analytical (what is) and an actionable, design (how to do) theory our previous discussion also contains elements of an explanatory theory by providing answers to how, when and where questions concerning participatory design in agile development. To accomplish a more exhaustive explanatory theory and to answer why participatory design in agile development played out the way it did and was successful in the presented and analyzed case and to draw more general lessons learnt, we have to move beyond mere description. Kautz (2004) and Madsen et al. (2006) emphasize the signicance of organizational structure, individual participants characteristics as well as the interplay between social context and social process for the successful enactment of information system development methods as organizational innovation and drawing on these innovation theories we nd that participatory design in agile development bears the characteristics of such an organizational innovation. From this perspective participatory design, worked out because of the two distinctly identiable categories of users. The onsite customers acted as individual leaders, champions and agents with regard to the chosen development and design approach, a responsibility, which has been identied as decisive for the successful utilization of innovations. The onsite customers were intermediaries for the second group of users, the other operative staff in the WaterWorks divisions. They comprehended their assignment as true representatives of the operational staff and backed up by the staff council their understanding as facilitators and communicators strongly supported the operational staffs pronouncing of opinion and gave them an inuential and potent voice.

Investigating the design process

231

ITP 24,3

232

This had an impact on and lead to an effective social process, another facet of thriving innovation, which focuses on the interaction of the engaged stakeholders. The interaction between the different stakeholder groups went well: The onsite customers sought contact with the operational staff users, the development organizations subproject managers and the rest of development team who in turn sought contact with the onsite customers and the other users on informal occasions, in feedback meetings, in road shows, during test sessions, and during system operation in the divisions. The distribution of power, the second characteristic of a well functioning social process, provides further explanation: The onsite customers held a clear mandate and authority to decide on all design matters and used it to the satisfaction of all corporative actors. This led to true participation and controlled a possible dominance of the development team. In this setting good social relations and a social infrastructure consisting of a broad range of different participants made up the social context in which participatory design could strive. This was ultimately supported by a structural context in which the development method had been clearly communicated to the customer organization, and in which a sophisticated project structure with subprojects, partnering subproject managers, as well as a communication structure with weekly and daily meetings and co-location of the development team had been set up. This environment helped to overcome the identied challenges: lack of user produced user stories and story cards, high change rate, doubt about designer competences, and limited number of tests. The onsite customers close interaction with the other operational staff and with the development team compensated for the lack of writing user stories and story cards and for the limited number of tests and helped resolving the issues concerning the customer organizations doubts of the developers competences. Together with the communication structure, it also managed the high change rate. Further explanations for why participatory design worked in the investigated case can be found in complex adaptive systems theory which has been used as a theoretical scheme to understand and explain agile development (Vidgen and Wang, 2009; Kautz and Madsen, 2010). On the background of complex adaptive systems theory Vidgen and Wang (2009) identied matching co-evolutionary change rate, optimizing self-organization, and synchronizing concurrent exploitation and exploration as the guiding principles of agile software development. From this perspective the projects time pacing consisting of iteration tests in 3-6 weeks periods, of weekly feedback sessions, and of daily meetings provided the appropriate intervals to match and deal with the changes which were brought up continuously in a participatory manner during the project. In this context, using the existing requirement lists to write user stories while investigating future options with information from and in consultation with operative staff, balancing at the edge of time in terms of complex adaptive systems theory, facilitated synchronizing concurrent exploitation of existing knowledge and exploration of new knowledge and strengthened the participative role of the onsite customers. Finally as interconnected autonomous agents the onsite customers had a mandate to make decisions and exerted power, which allowed for the self-organization of the project and the participatory design process.

Conclusion In this paper we investigated the question how customers and users participate in agile development and design activities in practice as a step towards a theory of participatory design in agile software development. We found genuine customer and user participation carried out by onsite customers who were managers and team leaders with operational tasks in the customer organization and by other operational staff. Our analysis shows that the integrative framework for user participation which consists of well established concepts can be fruitfully used in a new context to understand what participatory design is and how, when and where it can be performed as an instance of a design process in agile development. As such we contribute to an analytical and a design theory of participatory design in agile development and follow Markus and Maos (2004) call, revisits the concepts and show that they are also useful in a novel environment. We also contribute with a practice study of a design process as requested by Bratteteig (2007) to broaden the perspective on design science research and we provide a sound, empirical study of agile development as demanded by Dyba and Dingsyr (2008). Furthermore we explicate why participatory design contributes to the successful completion of the investigated project and provide elements of an explanatory theory. By drawing on innovation theory we found that participatory design in agile development bears the characteristics of a successful organizational innovation. Grounding further explanations in complex adaptive systems theory we provide an additional argument why participatory design despite some identied challenges fosters project staff to successfully carry out the agile development project. Our research adds to the body of knowledge in information systems development with rich insight (Walsham, 1995) about participatory design as a possible and vital element of agile software development and provides a link between the otherwise often disconnected research areas and research communities of participatory design, agile development and design science. Further research is necessary to allow for more theorizing about the relation of the three elds and for a viable theory of participatory design in agile software development and information systems development.
References Beck, K. and Andres, C. (2004), Extreme Programming Explained: Embrace Change, 2nd ed., Addison Wesley Professional, Boston, MA. Bjerknes, G. and Bratteteig, T. (1995), User participation and democracy: a discussion of Scandinavian research on system development, Scandinavian Journal of Information Systems, Vol. 7 No. 1, pp. 73-98. Bjrn-Andersen, N. and Hedberg, B. (1977), Designing information systems in an organizational perspective, Studies in the Management Sciences Prescriptive Models of Organizations, Vol. 5, pp. 125-42. Bratteteig, T. (2007), Design research in informatics: a response to Iivari, Scandinavian Journal of Information Systems, Vol. 19 No. 2, pp. 65-73. Chamberlain, S., Sharp, H. and Maiden, N.A.M. (2006), Towards a framework for integrating agile development and user-centred design, Proceedings of XP, pp. 143-53. Clement, A. (1994), Computing at work: empowering action by low-level users, Communications of the ACM, Vol. 37 No. 137, pp. 52-63.

Investigating the design process

233

ITP 24,3

234

Cockburn, A. (2002), Agile Software Development, Addison-Wesley, Boston, MA. Conboy, K. (2009), Agility from rst principles: reconstructing the concept of agility in information systems development, Information Systems Research, Vol. 20 No. 3, pp. 329-54. Creswell, J.W. (2003), Research Design - Qualitative, Quantitative and Mixed Methods Approaches, Sage Publications, Thousand Oak, CA. Damodaran, L. (1996), User participation in the systems design process a practical guide for users, Behaviour and Information Technology, Vol. 15 No. 6, pp. 363-77. Dyba, T. and Dingsyr, T. (2008), Empirical studies of agile software development: a systematic review, Information and Software Technology, Vol. 50 Nos 9-10, pp. 833-59. Ehn, P. (1992), Scandinavian design: on participation and skill, in Adler, P.S. and Winograd, T.A. (Eds), Usability: Turning Technologies into Tools, Oxford University Press, New York, NY, pp. 96-132. Fitzgerald, B., Russo, N.L. and Stolterman, E. (2002), Information Systems Development, Methods in Action, McGraw Hill, London. Friedman, A.L. and Cornford, D.S. (1989), Computer Systems Development: History and Implementation, Wiley & Sons, New York, NY. Germomprez, M., Hovorka, D. and Collopy, F. (2007), A theory of tailorable technology design, Journal of the Association for Information Systems, Vol. 8 No. 6, pp. 351-67. Gregor, S. (2006), The nature of theory in information systems, MIS Quarterly, Vol. 30 No. 3, pp. 611-42. Gross, J.B., Daughtry, J.M. III and Lee, J.C. (2008), Heurists of the world unite! Merging agile methods in software and interaction design, Agile Journal, Vol. 3 No. 2. Hansson, C., Dittrich, Y. and Randell, D. (2006), How to include users in the development of off-the-shelf software: a case for complementing participatory design with agile development, Proceedings of the 39th Hawaiian International Conference on System Science, IEEE Computer Society Press, Washington, DC. Hevner, A.R., March, S.T., Park, J. and Ram, S. (2004), Design science in information systems research, MIS Quarterly, Vol. 28 No. 1, pp. 75-105. Highsmith, J. (2002), Agile Software Development Ecosystems, Addison-Wesley, Boston, MA. Iivari, N. (2006), Constructing the users in open source software development an interpretative cases study of user participation, Information Technology & People, Vol. 22 No. 2, pp. 132-56. Iivari, J. and Iivari, N. (2006), Varieties of user-centeredness, Proceedings of the 39th Annual Hawaii International Conference on System Sciences, IEEE Computer Society Press, Washington, DC. Kautz, K. (2004), The enactment of methodology the case of developing a multimedia information system, Proceedings of the International Conference on Information Systems, Washington, DC, December 12-15. Kautz, K. and Madsen, S. (2010), Understanding agile software development in practice, Proceedings of the 2010 International Conference on Information Resources Management, Rose Bay, May 16-18. Kautz, Z. and Zumpe, S. (2008), Just enough structure at the edge of chaos: agile information systems development in practice, in Abrahamsson, P. et al. (Eds), Agile Processes in Software Engineering and Extreme Programming, Proceedings of the International Conference XP, Limerick, pp. 137-46. Kensing, F. and Blomberg, J. (1998), Participatory design: issues and concerns, Computer Supported Cooperative Work, Vol. 7, pp. 167-85.

Kujala, S. (2003), User participation: a review of the benets and challenges, Behaviour and Information Technology, Vol. 22 No. 1, pp. pp1-p16. Madsen, S.K., Kautz, K.R. and Vidgen, R. (2006), A framework for understanding how a unique and local IS development method emerges in practice, European Journal of Information Systems, Vol. 15 No. 2, pp. 225-38. March, G. and Smith, G. (1995), Design and natural science research on information technology, Decision Support Systems, Vol. 15, pp. 251-66. Markus, M. and Mao, Y. (2004), User participation in development and implementation: updating an old tired concept for todays IS contexts, Journal of the Association for Information Systems, Vol. 5 Nos 11-12, pp. 514-44. Martin, A., Biddle, R. and Noble, J. (2004), The XP customer role in practice: three studies, Proceedings of the 2nd Agile Development Conference, Salt Lake City, UT. Matook, S. and Kautz, K. (2008), Mindfulness and agile software development, Proceedings of the 19th Australasian Conference on Information Systems (ACIS), Christchurch, pp. 638-47. Merisalo-Rantanen, H., Tuunanen, T. and Rossi, M. (2005), Is extreme programming just old wine in new bottles: a comparison of two cases, Journal of Database Management, Vol. 16 No. 4, pp. 41-61. Mumford, E. (1983), Designing Human Systems for New Technology: The ETHICS Method, Manchester Business School, Manchester. Rittenbruch, M., McEwan, G., Ward, N., Manseld, T. and Bertenstein, D. (2002), Extreme participation: moving extreme programming towards participatory design, Proceedings of the Participatory Design Conference (PDC), pp. 29-41. Robinson, H. and Sharp, H. (2003), XP culture: why the 12 practices both are and are not the most signicant thing, Proceedings of the Agile Development Conference, Salt Lake City, UT. Robinson, R. and Sharp, H. (2005), The social side of technical practices, Proceedings of the XP Conference, pp. 100-8. Van de Ven, H.A. (2007), Engaged Scholarship A Guide for Organizational and Social Research, Oxford University Press, New York, NY. Venable, J. (2006), A framework for design science research activities, Proceedings of the 2006 Information Resource Management Association Conference, Washington, DC, pp. 184-8. Vidgen, R. and Wang, X. (2009), Coevolving systems and the organization of agile software development, Information Systems Research, Vol. 20 No. 3, pp. 355-76. Walls, J.G., Widmeyer, G.R. and El Sawy, O.A. (1992), Building an information systems design theory for vigilant EIS, Information Systems Research, Vol. 3 No. 1, pp. 36-59. Walsham, G. (1995), Interpretive case studies in IS research: nature and method, European Journal of Information System, Vol. 4, pp. 74-81. Wastell, D., Sauer, J. and Schmeink, C. (2009), Time for a design turn in IS innovation research? A practice report from the home front, Information Technology & People, Vol. 22 No. 4, pp. 335-50. Corresponding author Karlheinz Kautz can be contacted at: Karl.Kautz@cbs.dk

Investigating the design process

235

To purchase reprints of this article please e-mail: reprints@emeraldinsight.com Or visit our web site for further details: www.emeraldinsight.com/reprints