Você está na página 1de 16

Benchmarking: An International Journal

R&D portfolio management practices in Brazilian electric power utilities


Eduardo K. Yamakawa, Thayla T. Sousa-Zomer, Paulo A. Cauchick-Miguel, Catherine P. Killen,
Article information:
To cite this document:
Eduardo K. Yamakawa, Thayla T. Sousa-Zomer, Paulo A. Cauchick-Miguel, Catherine P. Killen,
(2018) "R&D portfolio management practices in Brazilian electric power utilities", Benchmarking: An
Downloaded by Universidad Peruana de Ciencias Aplicadas Lima At 18:08 29 April 2019 (PT)

International Journal, Vol. 25 Issue: 6, pp.1641-1655, https://doi.org/10.1108/BIJ-10-2016-0159


Permanent link to this document:
https://doi.org/10.1108/BIJ-10-2016-0159
Downloaded on: 29 April 2019, At: 18:08 (PT)
References: this document contains references to 53 other documents.
To copy this document: permissions@emeraldinsight.com
The fulltext of this document has been downloaded 199 times since 2018*
Users who downloaded this article also downloaded:
(2018),"Organizational logic to prioritize between the elements of triple bottom line", Benchmarking:
An International Journal, Vol. 25 Iss 6 pp. 1626-1640 <a href="https://doi.org/10.1108/
BIJ-02-2017-0027">https://doi.org/10.1108/BIJ-02-2017-0027</a>
(2008),"Project portfolio management for product innovation", International Journal
of Quality &amp; Reliability Management, Vol. 25 Iss 1 pp. 24-38 <a href="https://
doi.org/10.1108/02656710810843559">https://doi.org/10.1108/02656710810843559</a>

Access to this document was granted through an Emerald subscription provided by emerald-
srm:536606 []
For Authors
If you would like to write for this, or any other Emerald publication, then please use our Emerald
for Authors service information about how to choose which publication to write for and submission
guidelines are available for all. Please visit www.emeraldinsight.com/authors for more information.
About Emerald www.emeraldinsight.com
Emerald is a global publisher linking research and practice to the benefit of society. The company
manages a portfolio of more than 290 journals and over 2,350 books and book series volumes, as
well as providing an extensive range of online products and additional customer resources and
services.
Emerald is both COUNTER 4 and TRANSFER compliant. The organization is a partner of the
Committee on Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for
digital archive preservation.

*Related content and download information correct at time of download.


The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/1463-5771.htm

R&D portfolio
R&D portfolio management management
practices in Brazilian electric practices

power utilities
Eduardo K. Yamakawa and Thayla T. Sousa-Zomer 1641
Federal University of Santa Catarina, Florianopolis, Brazil
Received 16 October 2016
Paulo A. Cauchick-Miguel Revised 2 April 2017
Downloaded by Universidad Peruana de Ciencias Aplicadas Lima At 18:08 29 April 2019 (PT)

Accepted 17 April 2017


Department of Production and Systems Engineering,
Federal University of Santa Catarina, Florianopolis, Brazil and
University of Technology Sydney, Sydney, Australia, and
Catherine P. Killen
University of Technology Sydney, Sydney, Australia

Abstract
Purpose – Project portfolio management (PPM) has been recognized as critical for the productivity of
research and development (R&D) investments, but empirical research on PPM use and outcomes in
non-commercial R&D environments is limited. The purpose of this paper is to investigate PPM processes and
outcomes in a unique R&D context within Brazilian electric power utilities.
Design/methodology/approach – An exploratory best practice survey was used to collect data on PPM
processes, methods and performance results in the power sector. Analysis of the data employs descriptive
statistics and comparative analysis in the light of the literature.
Findings – The findings emphasize the importance of strategic value and the need for PPM to be customized
for the specific context. The results also demonstrate the importance of adopting selection criteria and
measures in accordance with the organizations strategic goals.
Practical implications – The findings may help organizations better understand how PPM can be tailored
for the environment. PPM managers in utilities and other non-commercial R&D environments may find
guidance in tailoring and improving their PPM approaches.
Originality/value – The contributions of this paper are twofold. First, it provides empirical findings to
support PPM concepts on strategic alignment and the importance of context by demonstrating how PPM
works to deliver strategy in a unique environment. Second, it contributes to the management of R&D projects
and portfolios in power utilities, providing an example and analysis that may offer guidance. The
contributions from this study may also offer insights that are valuable for R&D management in other utilities,
or for R&D management in general.
Keywords Project portfolio management, Power utilities, R&D, Survey
Paper type Research paper

1. Introduction
Companies engage in research and development (R&D) as an investment to their future
product and service offerings (Martinsuo et al., 2014), and managing a set of R&D projects
as a portfolio has become a common practice (Daim et al., 2013). Since organizations tend to
run several projects concurrently, project portfolio management (PPM) is a key competence
for flexibly adjusting to environmental changes and maintaining competitive advantage
(Teller and Kock, 2013). Indeed, portfolio techniques allow R&D projects to be analyzed in a
systematic manner, providing an opportunity for the optimization of company long-term
growth and profitability (Mikkola, 2001).
Benchmarking: An International
PPM is an essential part of the operational planning process for both private and public Journal
organizations (Solak et al., 2010). Nevertheless, previous research on PPM has focused Vol. 25 No. 6, 2018
pp. 1641-1655
primarily on commercial contexts, and project portfolios in public utilities have not received © Emerald Publishing Limited
1463-5771
as much attention (Martinsuo and Dietrich, 2002; Plessis and Killen, 2013). In public utilities, DOI 10.1108/BIJ-10-2016-0159
BIJ the goals for R&D are different from those in commercial environments. Whereas
25,6 the function of R&D is measured in terms of a company internal return on investment in the
private sector, the benefits of public R&D tend to be more diffused with respect to both
type and impact (Bozeman and Rogers, 2001). Certainly, public utilities R&D projects are
assessed in broader terms that involve multiple dimensions (Plessis and Killen, 2013).
However, there is still very little guidance provided by the literature for public sector
1642 organizations to understand and measure R&D project value (Killen and Plessis, 2012).
Since public sector organizations usually differ from private businesses, these
differences may imply new requirements to the support and organization of PPM
(Martinsuo and Dietrich, 2002). Indeed, PPM needs to be applied appropriately to each
situation and its success is dependent on the context (Martinsuo, 2013). Moreover,
Downloaded by Universidad Peruana de Ciencias Aplicadas Lima At 18:08 29 April 2019 (PT)

successful PPM depends on organizationʼs structural alignment with the needs of PPM
(Kaiser et al., 2015).
Thus, since a better understanding of PPM in practice and context is needed (Martinsuo,
2013), and little is known about PPM practices in public sector organizations (Martinsuo and
Killen, 2014; Nielsen and Pedersen, 2014), an empirical investigation was conducted to
explore PPM practices in power utilities located in an emerging economy (Brazil). The goal
is also to address the following research questions:
RQ1. How do public utilities handle portfolio management?
RQ2. Are there differences between public and private organizations in the electric
utilities sector?
There is a need for effective R&D programs for the electric utility industry, and it is also
necessary to develop uniform management practices for these projects (Daim et al., 2013);
therefore, this sector may be valuable for analysis. In addition, the context of a developing
country was chosen because public utilities play a vital role in demonstrating the pace of
economic and social progress of a developing nation (Prashar, 2016). Moreover, the Brazilian
R&D program in power utilities offers a valuable context for PPM investigation. The power
utilities in Brazil are required to invest a percentage of their net operating income (NOI) in
annual R&D projects. This is regulated by a national law and the utilities must ensure that
the projects meet the guidelines set by a regulating body, the Brazilian Electricity
Regulatory Agency (ANEEL). If the projects do not meet the guidelines, financial penalties
apply. Therefore, reducing the risk of non-compliance is of paramount importance. Through
improving project decision making, PPM may be valuable in aiding the utilities in meeting
ANEEL audit requirements. In fact, a specific model for PPM in the analyzed context may
be valuable to minimize the different types of risks (e.g. economic feasibility). Other ANEEL
requirements can also be included to optimize the project selection process considering
risks, budget and strategic alignment of the projects.
Through an exploratory survey, the objective is to investigate PPM practices in the
utilities in the country, to collect primary data about PPM practices and outcomes, and
to compare the findings among the utilities and with other studies. The purpose is to add to
PPM empirical body of knowledge, since empirical evidence on the use, outcomes and most
important success drivers of PPM is still scarce (Spieth and Lerch, 2014), especially
regarding public contexts.
The remainder of this paper is structured as follows. Section 2 presents a brief
literature review, drawing on the PPM body of knowledge. The context under analysis is
also outlined at the end of the section. Section 3 describes the methods applied for
gathering and analyzing data. Section 4 presents the survey results and discusses the
main findings in the light of the literature. Finally, conclusions are drawn in Section 5
concerning the usefulness and implications of this research, its limitations and directions
for future research.
2. Project portfolio management literature R&D portfolio
Project portfolios can be defined as collections of single projects that run concurrently management
(Archer and Ghasemzadeh, 1999). PPM is the art and science of applying a set of knowledge, practices
skills, tools and techniques to a collection of projects to meet or exceed the needs and
expectations of an organization’s investment strategy (Dye and Pennypacker, 1999). PPM
focuses on the execution of the right projects in an effective way (Teller et al., 2012). It has
been gaining more and more importance in theory and practice (Teller and Kock, 2013). 1643
The initial foundation for PPM derives from the financial portfolio management
discipline (Markowitz, 1952), which relies on mathematical optimization models and
financial data. However, due to the complexity of the project environment and the need to
Downloaded by Universidad Peruana de Ciencias Aplicadas Lima At 18:08 29 April 2019 (PT)

incorporate multiple types of information and assist optimization decisions along several
dimensions, these models have not gained support in management environments and new
approaches and methods have been developed (Killen, 2008).
Key PPM tasks are to select projects, manage their alignment with strategy and allocate
resources to projects taking into consideration the optimization of the entire portfolio
(Archer and Ghasemzadeh, 1999). Over the past four decades, project selection has gained
increasing attention in the literature (Cooper, 1981; Danila, 1989; Hall and Nauda, 1990).
Indeed, early studies defined portfolio management as project selection (Baker, 1974). Later
on, the term portfolio management was applied to portfolios for new product development
(Cooper et al., 1997, 1999). Some literature has viewed PPM primarily as a means of project
selection drawing upon portfolio concepts of balancing risk and return, but nowadays it
refers to a broader set of activities and considers a wider range of factors (Kaiser et al., 2015;
Killen and Hunt, 2010). Some of the typical activities associated with PPM are (Blichfeldt
and Eskerod, 2008; Martinsuo, 2013): gathering of possible projects, screening, prioritization
and selection according to strategy and available resources; evaluation of the running
projects and the concurrent reprioritization of projects in the portfolio; and reallocation of
resources to projects according to priority.
The PPM literature has been evolving and covers multiple aspects. Much of the early
research attention has focused on the tools and techniques for portfolio evaluation and
prioritization (Amiri, 2010; Henriksen and Traynor, 1999), resource management solutions
(Hansen et al., 1999; Hendriks et al., 1999) and holistic frameworks (Archer and Ghasemzadeh,
1999; Cooper et al., 2001a). Research on PPM has expanded into a complete managerial
approach, focusing on aspects that cover not only tools, techniques and methods, but also
includes aspects on how PPM is practiced by companies (Blichfeldt and Eskerod, 2008).
The empirical research on PPM has increasingly viewed PPM as more than a toolset for
rational decision-making processes (Martinsuo, 2013). Although the research agenda for
PPM studies has already moved beyond the state of reliance on untested principles (i.e. the
development of tools and methods for PPM that were not tested for their practical
relevance), and the research has moved toward studies that also look upon PPM as it is
actually carried out by companies (Blichfeldt and Eskerod, 2008), more empirical studies are
still needed. Empirical studies are especially needed in response to literature that
emphasizes how PPM is customized to align with the context and calls for further
context-based studies. PPM implementation in project-based organizations is not merely a
matter of defining project selection techniques, instead it shapes the organizational structure
as a whole (Kaiser et al., 2015). Indeed, R&D portfolio management is a challenging task for
most organizations (Cooper et al., 1998). In the public sector, for instance, the methods
necessary for utilities to successfully manage new or renewing R&D efforts need further
study (Daim et al., 2013; Menke, 2013). In fact, more attention needs to be paid to the
assumptions associated with PPM in practice in real contexts (Martinsuo, 2013). This
research gap is addressed through research based in power utilities. The next section
outlines the specific context under analysis in this study.
BIJ 2.1 R&D portfolio management in Brazilian power utilities
25,6 Electric power utilities have not developed many working methods for managing R&D
projects and funding (Daim et al., 2013). The principles of PPM, however, may well apply
in public sector organizations (Martinsuo and Dietrich, 2002). As stated earlier, the
Brazilian power utility context is explored in this study because it may be valuable to
investigate the relationship between PPM and the context. Moreover, only a limited
1644 number of studies have focused on PPM in Brazilian companies (Cauchick Miguel, 2008)
and in developing countries in general, and this work intends to contribute to the body of
knowledge on this subject as well.
Since 2000, the Brazilian electric utility industry has been receiving a large amount of
investment due to the creation of a national law that determines the allocation of financial
Downloaded by Universidad Peruana de Ciencias Aplicadas Lima At 18:08 29 April 2019 (PT)

resources in R&D. The power utilities must invest 0.5 percent of their NOI in R&D projects
every year, according to a federal law issued by the Presidency of the Republic (2016). In
Brazil, the power utilities R&D projects are usually conducted by external organizations
such as R&D institutes, universities or private companies, and the power utilities manage
the portfolio of projects. Once completed, the projects are audited by ANEEL using the
following assessment criteria: projects originality; scope and application potential; relevance
(training, technical training, social and environmental impacts) and cost (the economic
impacts resulting from the application of the project results).
If the project originality is not proven or there is non-compliance related to the project
deadline, budget, scope, documentation, etc., ANEEL may not approve part or even the
whole project. When this occurs, the project budget is financially updated (by a Brazilian
base interest rate named “SELIC,” usually high), and this amount must be returned to the
power utilities’ R&D fund. This represents a financial loss for the utility, since the money
that was invested in the project came from the power utilities’ regular budget and then must
be returned to the R&D fund.
Thus, since the demand for effective R&D programs for the power utilities continues to
increase and this is regulated by the Brazilian Government, the power utilities must
develop effective management practices to reduce the risks of problems satisfying the
criteria of the ANEEL audit and, as a consequence, having to face financial losses.
Anecdotal evidence suggests that R&D projects are managed by power utilities using
PPM techniques in some cases, with different degrees of formality and a variety of
approaches. In other cases, the projects are managed individually without being
considered as a portfolio. Thus, development of a PPM approach adapted to the public
power utility context is proposed. The first step of this work is then to investigate the
existing practices of managing R&D projects in the Brazilian power utilities taking into
consideration its context.
The next section outlines the research design and procedures used to collect and analyze
survey data of PPM practices in the Brazilian power utilities.

3. Survey design
An exploratory-descriptive survey was carried out to investigate PPM practices and
outcomes in Brazilian power utilities. This kind of survey approach is valuable when the
objective is to gain preliminary insight on a subject as well as to provide preliminary
evidence of association among concepts (Forza, 2002). Moreover, surveys are also suitable
for comparing responses across groups (Gillham, 2000).
When conducting exploratory surveys, generally, there is not a specific theoretical model
(Forza, 2002). The survey was adapted from a questionnaire instrument developed and used
to capture data on PPM practices and outcomes in Australian companies (Killen, 2008), and
also aligns with similar studies conducted in North America (e.g. Cooper et al., 1999). These
surveys form a basis for much of the literature on PPM. Moreover, they represent initial
steps in the maturity journey of portfolio management implementation in companies, R&D portfolio
similarly to what is occurring in the context under analysis in this study. Figure 1 management
summarizes the survey process. practices
The survey questions were designed for a specific audience, but attempts were made to
explain the terms and to limit assumptions about the respondents’ levels of knowledge. A
five-point Likert scale was adopted, where the anchored endpoints for responses were
defined by statements relevant to each particular question. This technique was applied to 1645
ensure that the respondent clearly understood the meaning of the high and low Likert
ratings, and to provide an additional level of confidence that the question statement had
been interpreted correctly.
The majority of the questionnaire consists of closed questions which required the
Downloaded by Universidad Peruana de Ciencias Aplicadas Lima At 18:08 29 April 2019 (PT)

respondent to check a box or circle a number, although some open-ended questions were
also included. The questions were divided into six blocks (see Table A1) in order to address
different aspects of PPM process and outcomes. The final survey instrument consisted of
105 questions in addition to some sub-questions.
The survey targeted Brazilian power utilities. To select potential participants, a list of
R&D projects published by ANEEL in 2013 was examined. In all, 124 companies that were
conducting or have conducted R&D projects in the past were then targeted. These
companies fit within three electric power sub-sectors: 50 from the power distribution, 47
from the power generation and 27 from the power transmission. A pilot test of the
questionnaire was conducted with two utilities: one with considerable experience in PPM
and another one with no established PPM process. The goal of the pilot test was to check
the questionnaire for both form and content, in order to enhance it according to the
respondents comments.
The questionnaire was revised, and a final version was then sent by e-mail to all selected
power utility companies. Explanation about the research objectives was also provided to the
target respondents. The instrument was available to be completed online for up to four
weeks. After the second and fourth weeks, e-mails and telephone contacts were conducted to
follow-up with the remaining non-respondents, in an attempt to increase the response rate.
Data were then analyzed using descriptive statistical measures. The results were also
compared among the utilities (public and private), in the light of the literature. The survey
results and a discussion of the main findings are presented next.

Public and private utilities identified


Selection of companies
from ANNEEL R&D report

Questionnaire adapted from previous


Development of survey instrument
studies (e.g. Killen, 2008)

Test of the questionnaire with two


power utilities
Conduct pilot test
Review/refinement of
the survey instrument

Requesting data from 124 utilities


Collect data
from three power sub-sectors

Figure 1.
Descriptive analysis of 30 respondents
Data summary and analysis Survey structure
based on similar works and literature
BIJ 4. Survey results and discussion
25,6 The findings are presented in the following subsections according to the instrument questions.
First a general profile of the utilities is provided, followed by findings on PPM importance and
maturity, PPM methods, and finally the performance results of the portfolio management.

4.1 Utilities characteristics and PPM importance


1646 In all, 30 complete responses were received from the 124 questionnaires sent out (24 percent
response rate). The percentages of respondents by power sub-sectors are shown in Table I.
As can be seen, the response rate was similar among sub-sectors. From the respondents, 60
percent of the surveyed utilities are publicly owned and the remainder are private. There is a
Downloaded by Universidad Peruana de Ciencias Aplicadas Lima At 18:08 29 April 2019 (PT)

strong focus on financial measures in commercial environments, and the PPM research has
largely ignored “public good” project environments (Martinsuo and Killen, 2014). Thus, the
findings were analyzed from the perspective of both private and public utilities to identify
possible differences, since in public utilities R&D projects may involve a different set of
measures than private organizations (Plessis and Killen, 2013).
Most of the respondents are from areas directly involved in daily management of project
portfolio and answered the questionnaire based on their knowledge on PPM. This is aligned
with the findings of Martinsuo and Lehtonen (2007) who also pointed out that the
understanding of portfolio-level issues needs to be considered as part of project managers’
capabilities and not only as a top management concern.
The respondents were, first, asked about the importance of PPM for the business, broken
down by functions. Although portfolio management has been emphasized as one of the most
important facets of management leadership (Roussel et al., 1991), evidence suggested that
not every senior manager perceive the importance in the same way. Figure 2 shows how
important portfolio management is perceived to be by different levels of management,
considering five levels of assessment on a Likert scale.
The level of importance placed on PPM is mentioned in the literature as a success factor
that influences product portfolio outcomes (Killen, 2008). As expected, in both public and

Sub-sector Sent questionnaires Returned questionnaires Response rate (%)


Table I.
Survey responses Power distribution 50 13 26
according to electric Power generation 47 11 23
power sub-sectors Power transmission 27 06 22

...by operations/production managers?

...by marketing/sales managers?

...by technology managers?

...by senior managers?

...by corporate executives?


Figure 2.
0 1 2 3 4 5
Importance of
portfolio management Not too Critically
important important
by functional area
Respondents’ perception about PPM importance (mean)
private utilities the highest importance was attributed by technology managers followed by R&D portfolio
operation and production managers, since R&D projects in electric utilities, in general, have management
a strong focus on technological aspects. In addition, those professionals are the ones more practices
involved with PPM. It has been stated in the literature that the ability to influence projects
plays a crucial role in the successful management of projects (Beringer et al., 2013).
The results demonstrate that the highest importance was attributed and perceived by those
who are involved and have influence in the projects. 1647
In contrast, the importance of PPM to sales and marketing managers was reported to be
lower (in fact the lowest) in both public and private utilities. Due to a federal law, only
consumers with a load of more than 10,000 kW can choose the power utility they buy
electricity for consumption. Other consumers must purchase electricity from the utilities in
Downloaded by Universidad Peruana de Ciencias Aplicadas Lima At 18:08 29 April 2019 (PT)

the region where they are located. Since there is almost no competition among the utilities,
marketing and sales do not need to put strong efforts on new solutions and technologies.
This may explain the low level of importance placed on PPM by those departments. The low
importance (Figure 2) may also represent a shortcoming in the widespread acceptance and
implementation of portfolio management by marketing and sales managers, and could
affect the performance, especially in the case of commercial environments as these
managers are important partners in new solutions development (Cooper et al., 1998).
Moreover, when comparing public and private utilities, although the mean values are very
similar, in private utilities a high proportion (nearly 60 percent) of senior managers
perceived the importance of PPM as “very important” and “critically important,” when
compared to public utilities where only 39 percent reported such a level of importance. This
is relevant, since senior managers intensity of engagement may influence project portfolio
success (Beringer et al., 2013). In fact, the importance that management places on portfolio
management is correlated with the results in terms of the explicitness and formality of the
portfolio management process as well as to management perception and satisfaction with
the outcomes (Cooper et al., 1998).
The power utilities were also asked to rate possible reasons why portfolio management is
important. Financial aspects are the most common reasons cited for portfolio importance;
portfolio management is reported to be vital for organizations to maximize return and
maximize R&D productivity, and to achieve financial goals (Cooper et al., 2001b). Although
financial aspects were often mentioned as relevant, the most important reason why portfolio
management is vital for both private and public power utilities is the link to the business
strategy. This is crucial, because the closer R&D strategy is aligned with corporate strategy,
the more successful the effort will be, corroborating a similar study by Daim et al. (2013).
These results were also expected, since non-commercial aspects of strategic value must be
considered in the case of power utilities that involve other aspects related to ANEEL audit.
Moreover, power utilities cited their risk averse nature as one of the important reasons why
PPM is vital. Indeed, risk management is considered relevant in the context under analysis
due to the ANEEL audit. This aspect is corroborated by Martinsuo (2013), Sanchez et al. (2008)
and Teller et al. (2014), who emphasized the need to understand risks associated with the
project portfolio and its context, as well as to take the risks into account in PPM practices.
Risks are related to possible future events that may impact the successful completion of the
intended goals and there are many risks associated with the ANEEL audit.

4.2 PPM capabilities maturity and methods used


Cooper et al. (2001b) and Killen (2008) demonstrated a positive correlation between PPM
activities, contextual factors and improved outcomes. The present survey considered
some enquiries related to the level of maturity of the PPM capability in the power utilities.
The responses show mid-range levels of maturity (scale from 1 to 5), for both public and
private utilities.
BIJ Both public and private utilities presented low mean scores in terms of dealing with
25,6 projects as a portfolio (the mean was 3.2 for both public and private utilities) and in having
well-defined rules and procedures for portfolio management (the mean was 3.3 for both
public and private utilities). However, Cooper et al. (1998) argued that having a consistently
applied, explicit portfolio management process impacts strongly on performance. Cooper
et al. (2004) and Daim et al. (2013) also showed that top performers have well-defined
1648 portfolio procedures and apply their portfolio methods to all projects. From the overall
results, PPM maturity was not perceived high in both public and private power utilities.
The survey also aimed to identify methods used in portfolio management. One-third of
public utilities claimed to have a formal PPM process, whereas more than a half (58 percent)
of the private utilities reported that they have established PPM procedures. Having a
Downloaded by Universidad Peruana de Ciencias Aplicadas Lima At 18:08 29 April 2019 (PT)

structured and formalized PPM process in place help to foster the communication of
responsibilities, and are key drivers of high PPM performance (Spieth and Lerch, 2014).
Other studies demonstrate that top performers are the companies that have a well-defined,
structured portfolio management process (Barczak et al., 2009; Cooper et al., 1998).
Data were also collected on which PPM selection methods were used, how these methods
are applied to the PPM process and which methods are dominant. Table II shows the
percentage of respondents that use each method.
Project selection approaches can be distinguished between financial and non-financial
models (Costantino et al., 2015). Financial methods were identified as the most popular for
projects selection among public and private utilities. Financial methods were also identified as
the most used approach in other electric utilities in the study performed by Daim et al. (2013).
Indeed, financial methods are the most commonly used methods for PPM (Behrens, 2016;
Cooper et al., 2001b). Actually, any portfolio management process must start with an assessment
of the potential worth of each of the projects under consideration (Szwejczewski et al., 2006).
Other methods such as scoring methods are also widely applied by the utilities.
Other empirical research shows similar results (e.g. Daim et al., 2013); no single methods are
appropriate for all firms, and that companies generally use more than one method
(Spieth and Lerch, 2014).
Moreover, when asked about which method dominates the decision-making process
(when more than one method is used), both public and private utilities stated that scoring
methods are the most used. Indeed, scoring method has distinct advantages over other
methods of project selection (Henriksen and Traynor, 1999), and one of the benefits of
scoring approaches is that they require managers to consider the full range of factors that
are relevant to a project’s success (Szwejczewski et al., 2006). This is relevant in the context
under analysis because ANEEL audit involves different aspects that need to be taken into
account in the project selection process such as the applicability of the project, social and
environmental impacts, and financial performance.

Utilities
Question on the use of the PPM method Public (%) Private (%)

Do you use a financial method for project selection? 77.7 66.7


Do you use a checklist method? 22.2 16.7
Do you use a scoring model method? 61.1 41.7
Do you use the business strategy as a basis for allocation of money for different
types of new product projects? 27.7 33.3
Do you plot projects on a bubble diagram or portfolio map and look for
Table II. projects in certain zones or quadrants of the bubble diagram? 5.6 8.3
PPM methods Note: n ¼ 30
Concerning the use of business strategies as the basis for allocating money across different R&D portfolio
types of projects, almost one-third of utilities stated that they use this approach for project management
selection. As pointed out by Daim et al. (2013), lower levels of application of mapping practices
approaches or bubble diagrams were found in both public and private utilities, although
Killen and Kjaer (2012) pointed out that best practice organizations use graphical and visual
information displays such as portfolio maps to facilitate group decision making. Other
studies (Barczak et al., 2009; Lerch and Spieth, 2013) also showed that, in general, portfolio 1649
maps are the methods that are applied by fewer firms.
The utilities were also asked about which factors were included in companies’ processes
for assessment and comparison of projects to determine the final mix of projects in the
portfolio (Figure 3).
Downloaded by Universidad Peruana de Ciencias Aplicadas Lima At 18:08 29 April 2019 (PT)

The financial return (pay-off ) was not the most often used assessment criterion, although
it is relevant (as pay-off is one of the assessment criteria applied by the regulatory agency
ANEEL). In general, the most important factors are: “technology,” “strategy fit” and “risk.”
Those were expected, especially due to the nature of this kind of R&D projects ( focus on
technology) as well as the risks associated with the ANEEL audit. This is also aligned with
the findings of Daim et al. (2013), which found that the most influential factors for evaluation
are the strategic fit and technological capability. Moreover, regarding the risk, this factor
seems to be more relevant in the context of public utilities; perhaps this is because they are
subject to an audit by ANEEL in addition to a Federal account control agency (required by
all public companies). Therefore, public utilities tend to be more cautious in relation to the
risks than private utilities.
In addition, it was noticed that most of the utilities use multiple criteria. This is a positive
aspect, since more criteria seem to be connected to better performers, according to Cooper
et al. (1998). Nevertheless, only two utilities (one public and one private) use other factors
(e.g. “originality”), which are related to the ANEEL assessment. The adoption of
standardized criteria is a feature of best practice PPM (Barczak et al., 2009; Spieth and Lerch,
2014). In addition, ANEEL criteria should be considered in project selection and assessment
in the context under analysis.
Respondents were also asked about the greatest challenges faced in PPM. A quarter of
utilities reported that the main challenge faced is “the originality of the projects.” This
response may be predictable since this is the main evaluation criterion of ANEEL. Indeed,
the responses indicated that the PPM process should address a way to reduce the risks of
having a project rejected in case it does not satisfy the ANEEL audit.

Technology

Strategic Fit/Core Competence

Risk/Probability of Success

Pay-off
Private
Factors

Public
Timing

Protectability

Synergy between projects


Figure 3.
Commercialization
Factors included in
companies process of
0 20 40 60 80 100
portfolio assessment
% of utilities
BIJ 4.3 Portfolio assessment and project selection methods
25,6 Management perception and satisfaction is a central construct in the PPM context as it
directly impacts the PPM outcome and performance (Cooper et al., 1999; Spieth and Lerch,
2014). Regarding the satisfaction of the respondents with the current portfolio assessment
and project selection methods, in general, the utilities are not completely satisfied with their
portfolio approaches. Figure 4 shows the percentage of respondents who ticked Levels 4
1650 (“agree”) plus those who ticked 5 (“strongly agree”) in the Likert scale with regard to
portfolio management methods.
The most positive responses are related to how easy it is to use the portfolio management
method, and the understanding of the method by senior managers. As already pointed out
by Daim et al. (2013), an important attribute of a successful portfolio process is that it is well
Downloaded by Universidad Peruana de Ciencias Aplicadas Lima At 18:08 29 April 2019 (PT)

understood throughout the organization. On the other hand, in general, the respondents
neither judge the current methods as excellent nor recommend them for use by other power
utilities. Although 60 percent of the respondents stated that the methods are realistic in
capturing the key facets of the problem, only five respondents (about 16 percent) strongly
agreed with the way the methods capture the key aspects of the problem ( for instance,
relatively to the ANEEL audit). However, as already pointed out by Cooper et al. (1998), the
best practice companies are those that have more realistic portfolio methods. Therefore,
current PPM methods adopted by the power utilities need improvements in some areas.

4.4 Performance of portfolio management methods


The survey also aimed to check how well portfolio management performs, since
organizations implement PPM in order to achieve improved outcomes.
Figure 5 shows the project portfolio performance results on six metrics, illustrating the
percentage of respondents who reported being “satisfied” and “very satisfied” (respectively,
4 and 5 point of a five-point Likert scale). The survey addressed three primary goals for a
PPM capability (Cooper et al., 1997): maximize the value of the portfolio, select the right
balance of projects and link the portfolio to the business strategy. Another goal pointed out
by Cooper et al. (2002) involves the selection of the right number of projects. PPM
performance is often measured by how well these goals are met (Killen, 2008).
The responses show mid-range levels of performance in all metrics for both public and
private utilities. Thus, portfolio management appears to be working in a moderately
satisfactory fashion in the utilities. Both public and private utilities scored lower in terms of

We would recommend our method for use by


other businesses like ours

Overall we would rate our method as excellent

Our method is perceived by management to be


efficient
Our method is realistic capturing the key facets
of the problem
Our method is very “user friendly” and easy to
use

Our method is understood by management

Our method fits our management style of


decision - making
Our method is truly used to make go/kill
decisions on projects
Figure 4.
Satisfaction with 0 20 40 60 80 100
portfolio management Percentage of respondents that “agree” (4) plus
“strongly agree” (5)
The breakdown of spending (resources) in our
portfolio of projects truly reflects our
R&D portfolio
business strategy management
We have the right number of new product projects practices
for our resources—people, time and money—available
Our portfolio of new product projects has an excellent
balance in terms of long vs short term, high vs
low risk, a cross markets and technologies and so on
Our projects are done on time—in a timely and time
1651
efficient fashion

Our portfolio of new product projects contains only high


value ones to business–profitable, high retum projects
Downloaded by Universidad Peruana de Ciencias Aplicadas Lima At 18:08 29 April 2019 (PT)

with solid commercial prospects


The projects in our portfolio are aligned with our
business objectives and our business strategy
Figure 5.
0 20 40 60 80 100 Portfolio performance
Percentage of respondents that “agree” (4) plus results on six metrics
“strongly agree” (5)

project portfolio balance. However, a right balance within a portfolio enables firms to realize
profit and growth objectives without high risks (Mikkola, 2001). Moreover, although all
utilities present mid-range levels of performance, few companies allocated high scores on
the metrics. In the case of portfolio value, for instance, only about 13 percent of the utilities
stated that their portfolios contain only high-value projects (score 5 on the Likert scale). The
value is measured only in financial terms, and as argued by Martinsuo and Killen (2014), in
the case of power utilities, it is necessary to complement financial dimensions of value with
other, non-commercial aspects. The moderate-mean performance scores suggest that the
majority of utilities are performing in a substandard fashion and improvements are needed.

4.5 Summary and further discussion


The exploratory survey in the context under analysis confirmed that PPM is more than a
toolset for rational decision-making processes, as already pointed out by Martinsuo (2013).
The findings reinforce that the successful PPM depends on the organizations structural and
strategic alignment with the needs of PPM, as supported by Kaiser et al. (2015). The findings
also confirm the importance of the ANEEL audit in the process and suggest that PPM in
power utilities needs to be structured taking into account the context. Project selection and
evaluation methods should consider ANEEL criteria, reinforcing the need to assess other
dimensions of strategic value, as suggested by Martinsuo and Killen (2014).
The responses show that PPM processes do not differ very much among private and
public utilities. PPM is generally applied ( formally or informally), some companies treat
projects together as one portfolio, the rules and procedures are relatively clear and there is a
procedure defined, although these aspects were not reported by all utilities and in general
the responses showed mid-range levels when comparing to the literature (e.g. Cooper et al.,
1998, 2001b). Financial methods are used, and other considerations are also applied for
projects prioritization. The most important considerations for the project selection are the
schedule and budget aspects which are relevant for complying with ANEEL requirements.
Furthermore, the reasons for considering PPM as important include the need to manage the
risks associated with the ANEEL audit. This is consistent with the findings of Kaiser et al.
(2015) that the project selection criteria and measures derive from the organizations
strategic goals.
When asking about the challenges faced, the open-ended responses strongly highlighted
the importance of the ANEEL audit. The greatest challenges faced by respondents were
related to the originality of the R&D projects, which is the most significant evaluation
BIJ criterion applied by ANEEL. Although some aspects of the PPM process assist the portfolio
25,6 managers to assess matters related to the ANEEL audit, their PPM processes are not
designed specifically taking into account the ANEEL requirements.
Based on the high importance of the ANEEL audit that emerged in this survey, the
adoption of a customized PPM process, i.e. a PPM designed to purposefully meet the needs
of the ANEEL audit taking into account other dimensions of strategic value, would bring
1652 many benefits to the power utilities. It would be valuable to develop a PPM framework that
considers the risk of the projects in each step of the process, taking into account the issues
relevant to the ANEEL audit. In a framework like this, the goals of PPM would go beyond
the scope of the PPM mentioned earlier. In addition to the traditional goals of maximizing
the value of the portfolio, obtaining the proper balance of projects and connecting the
Downloaded by Universidad Peruana de Ciencias Aplicadas Lima At 18:08 29 April 2019 (PT)

portfolio to the company strategy, the PPM process for the Brazilian R&D program in power
utilities would be designed to ensure alignment with the regulation and to improve the
compliance of projects in the portfolio to ANEEL demands. This is a goal for future
research. The conclusion and main contributions of this work are summarized next.

5. Conclusions
This paper reports the findings of an exploratory study conducted in the context of
Brazilian power utility R&D programs and provided an initial understanding of how PPM is
applied. The findings confirm the importance of the audit of the Brazilian regulatory agency
(ANEEL) and show how PPM processes should be aligned with the context.
The contributions of this paper to the body of knowledge are twofold. First, this study
addresses PPM in utilities, including also public organizations, which is still little explored in
the literature and may find valuable to the empirical body of knowledge, although it did not
identify considerable differences between public and private utilities. The findings also
emphasized the importance of strategic value and the need for PPM to be customized for the
specific context of electric power utilities. Moreover, there are still few studies that investigate
PPM in developing countries and in the Brazilian context, and this paper provides
contributions to the literature on this subject as well. Indeed, most of the empirical investigation
of PPM practices has been conducted with companies located in developed economies.
Second, recent studies have demonstrated some relevant aspects such as the importance of
the context, other dimensions of strategic value, the alignment between organizations strategic
goals and the project selection criteria and measures. This study provided empirical findings to
support PPM concepts on strategic alignment in a relevant context by demonstrating PPM
practices that manage strategic issues in a unique environment. Regarding implications for
corporate practice, this study contributes to the management of R&D projects and portfolio in
power utilities by providing examples that may offer guidance. The contributions from this
study may also offer insights that are valuable for R&D portfolio management in other kind of
public and private companies. Further studies should investigate the relationship between
PPM capabilities and outcomes in both public and private utilities to better understand the
impact of different PPM practices on performance.

Acknowledgments
The authors thank the Brazilian research agencies Coordination for the Improvement of
Higher Level Education Personnel—CAPES (Grant No. 88881.1191612016-01) and National
Council for Scientific and Technological Development (CNPq) for the financial support of this
research project. The authors appreciate the utilities that made this study possible. However,
any analysis is the responsibility of the authors and, thus, it does not reflect the position of the
companies. Finally, the authors acknowledge the reviewers for their positive comments as
well as valuable suggestions and recommendations to improve the manuscript.
References R&D portfolio
Amiri, M.P. (2010), “Project selection for oil-fields development by using the AHP and fuzzy TOPSIS management
methods”, Expert Systems with Applications, Vol. 37 No. 9, pp. 6218-6224. practices
Archer, N.P. and Ghasemzadeh, F. (1999), “An integrated framework for project portfolio selection”,
International Journal of Project Management, Vol. 17 No. 4, pp. 207-216.
Baker, N.R. (1974), “R&D project selection models: an assessment”, IEEE Transactions on Engineering
Management, Vol. 21 No. 4, pp. 165-171. 1653
Barczak, G., Griffin, A. and Kahn, K.B. (2009), “Perspective: trends and drivers of success in NPD
practices: results of the 2003 PDMA best practices study”, Journal of Product Innovation
Management, Vol. 26 No. 1, pp. 3-23.
Downloaded by Universidad Peruana de Ciencias Aplicadas Lima At 18:08 29 April 2019 (PT)

Behrens, J. (2016), “A lack of insight: an experimental analysis of R&D managers’ decision making in
innovation portfolio management”, Creativity and Innovation Management, Vol. 25 No. 2,
pp. 239-250.
Beringer, C., Jonas, D. and Kock, A. (2013), “Behavior of internal stakeholders in project portfolio
management and its impact on success”, International Journal of Project Management, Vol. 31
No. 6, pp. 830-846.
Blichfeldt, B.S. and Eskerod, P. (2008), “Project portfolio management—there’s more to it than what
management enacts”, International Journal of Project Management, Vol. 26 No. 4, pp. 357-365.
Bozeman, B. and Rogers, J. (2001), “Strategic management of government-sponsored R&D portfolios”,
Environment and Planning C: Government and Policy, Vol. 19 No. 3, pp. 413-442.
Cauchick Miguel, P.A. (2008), “Portfolio management and new product development implementation: a
case study in a manufacturing firm”, International Journal of Quality & Reliability Management,
Vol. 25 No. 1, pp. 10-23.
Cooper, R.G. (1981), “An empirically derived new product project selection model”, IEEE Transactions
on Engineering Management, Vol. 28 No. 3, pp. 54-61.
Cooper, R.G., Edgett, S.J. and Kleinschmidt, E.J. (1997), “Portfolio management in new product
development: lessons from the leaders—I”, Research-Technology Management, Vol. 40 No. 5,
pp. 16-28.
Cooper, R.G., Edgett, S.J. and Kleinschmidt, E.J. (1998), “Best practices for managing R&D portfolios”,
Research-Technology Management, Vol. 41 No. 4, pp. 20-33.
Cooper, R.G., Edgett, S.J. and Kleinschmidt, E.J. (1999), “New product portfolio management: practices
and performance”, Journal of Product Innovation Management, Vol. 16 No. 4, pp. 333-351.
Cooper, R.G., Edgett, S.J. and Kleinschmidt, E.J. (2001a), “Portfolio management for new product
development: results of an industry practices study”, R&D Management, Vol. 31 No. 4,
pp. 361-380.
Cooper, R.G., Edgett, S.J. and Kleinschmidt, E.J. (2001b), Portfolio Management for New Products,
Basic Books, Cambridge.
Cooper, R.G., Edgett, S.J. and Kleinschmidt, E.J. (2002), “Optimizing the stage-gate process: what best-
practice companies do—II”, Research-Technology Management, Vol. 45 No. 6, pp. 43-49.
Cooper, R.G., Edgett, S.J. and Kleinschmidt, E.J. (2004), “Benchmarking best NPD practices—I”,
Research-Technology Management, Vol. 47 No. 1, pp. 31-43.
Costantino, F., Di Gravio, G. and Nonino, F. (2015), “Project selection in project portfolio management:
an artificial neural network model based on critical success factors”, International Journal of
Project Management, Vol. 33 No. 8, pp. 1744-1754.
Daim, T.U., Oliver, T. and Iskin, I. (2013), “Research and development (R&D) portfolio management in
the electric utility sector: does it change for the service sector?”, Benchmarking: An International
Journal, Vol. 20 No. 2, pp. 186-211.
Danila, N. (1989), “Strategic evaluation and selection of R&D projects”, R&D Management, Vol. 19
No. 1, pp. 47-62.
BIJ Dye, L.D. and Pennypacker, J.S. (1999), Project Portfolio Management: Selecting and Prioritizing
25,6 Projects for Competitive Advantage, PM Solutions, West Chester, PA.
Forza, C. (2002), “Survey research in operations management: a process-based perspective”,
International Journal of Operations & Production Management, Vol. 22 No. 2, pp. 152-194.
Gillham, B. (2000), Developing a Questionnaire, Continuum, New York, NY.
Hall, D.L. and Nauda, A. (1990), “An interactive approach for selecting R&D projects”, IEEE
1654 Transactions on Engineering Management, Vol. 37 No. 2, pp. 126-133.
Hansen, K.F., Weiss, M.A. and Kwak, S. (1999), “Allocating R&D resources: a quantitative aid to
management insight”, Research Technology Management, Vol. 42 No. 4, pp. 44-50.
Hendriks, M.H.A., Voeten, B. and Kroep, L. (1999), “Human resource allocation in a multi-project R&D
Downloaded by Universidad Peruana de Ciencias Aplicadas Lima At 18:08 29 April 2019 (PT)

environment: resource capacity allocation and project portfolio planning in practice”,


International Journal of Project Management, Vol. 17 No. 3, pp. 181-188.
Henriksen, A.D. and Traynor, A.J. (1999), “A practical R&D project-selection scoring tool”, IEEE
Transactions on Engineering Management, Vol. 46 No. 2, pp. 158-170.
Kaiser, M.G., El Arbi, F. and Ahlemann, F. (2015), “Successful project portfolio management beyond
project selection techniques: understanding the role of structural alignment”, International
Journal of Project Management, Vol. 33 No. 1, pp. 126-139.
Killen, C.P. (2008), “Project portfolio management for product innovation in service and manufacturing
industries”, doctorate Thesis, Macquire University, Sydney.
Killen, C.P. and Hunt, R.A. (2010), “Dynamic capability through project portfolio management in
service and manufacturing industries”, International Journal of Managing Projects in Business,
Vol. 3 No. 1, pp. 157-169.
Killen, C.P. and Kjaer, C. (2012), “Understanding project interdependencies: the role of visual representation,
culture and process”, International Journal of Project Management, Vol. 30 No. 5, pp. 554-566.
Killen, C.P. and Plessis, M. (2012), “Valuing non-commercial projects for portfolio decision making”, paper
presented at Australian Institute of Project Management National Conference, Melbourne,
7-10 October, available at: https://opus.lib.uts.edu.au/handle/10453/23319 (accessed August 8, 2016).
Lerch, M. and Spieth, P. (2013), “Innovation project portfolio management: a qualitative analysis”,
IEEE Transactions on Engineering Management, Vol. 60 No. 1, pp. 18-29.
Markowitz, H. (1952), “Portfolio selection”, The Journal of Finance, Vol. 7 No. 1, pp. 77-91.
Martinsuo, M. (2013), “Project portfolio management in practice and in context”, International Journal
of Project Management, Vol. 31 No. 6, pp. 794-803.
Martinsuo, M. and Dietrich, P. (2002), “Public sector requirements towards project portfolio management”,
paper presented at PMI Research Conference, 14-17 July, available at: www.pmi.org/learning/library/
public-sector-requirements-project-portfolio-management-1946 (accessed August 8, 2016).
Martinsuo, M. and Killen, C.P. (2014), “Value management in project portfolios: identifying and
assessing strategic value”, Project Management Journal, Vol. 45 No. 5, pp. 56-70.
Martinsuo, M. and Lehtonen, P. (2007), “Role of single-project management in achieving
portfolio management efficiency”, International Journal of Project Management, Vol. 25 No. 1,
pp. 56-65.
Martinsuo, M., Korhonen, T. and Laine, T. (2014), “Identifying, framing and managing uncertainties in
project portfolios”, International Journal of Project Management, Vol. 32 No. 5, pp. 732-746.
Menke, M.M. (2013), “Making R&D portfolio management more effective”, Research-Technology
Management, Vol. 56 No. 5, pp. 34-44.
Mikkola, J.H. (2001), “Portfolio management of R&D projects: implications for innovation
management”, Technovation, Vol. 21 No. 7, pp. 423-435.
Nielsen, J.A. and Pedersen, K. (2014), “IT portfolio decision-making in local governments: rationality,
politics, intuition and coincidences”, Government Information Quarterly, Vol. 31 No. 3, pp. 411-420.
Plessis, M. and Killen, C.P. (2013), “Valuing water industry R&D: a framework for valuing water R&D portfolio
research and development investments in financial and non-financial terms”, Water, Vol. 40 management
No. 6, pp. 63-67.
Prashar, A. (2016), “Six sigma adoption in public utilities: a case study”, Total Quality Management &
practices
Business Excellence, Vol. 27 Nos 5-6, pp. 479-506.
Presidency of the Republic (2016), “Investments in R&D and energy efficiency by power utilities”, Law
No. 13,280, March 3, available at: www.planalto.gov.br/ccivil_03/_ato2015-2018/2016/lei/l13280.
htm (accessed September 10, 2016).
1655
Roussel, P., Saad, K. and Erickson, T. (1991), Third Generation R&D: Managing the Link to Corporate
Strategy, Harvard Business School Press and Arthur D. Little, Boston, MA.
Downloaded by Universidad Peruana de Ciencias Aplicadas Lima At 18:08 29 April 2019 (PT)

Sanchez, H., Robert, B. and Pellerin, R. (2008), “A project portfolio risk-opportunity identification
framework”, Project Management Journal, Vol. 39 No. 3, pp. 97-109.
Solak, S., Clarke, J.-P.B., Johnson, E.L. and Barnes, E.R. (2010), “Optimization of R&D project portfolios
under endogenous uncertainty”, European Journal of Operational Research, Vol. 207 No. 1,
pp. 420-433.
Spieth, P. and Lerch, M. (2014), “Augmenting innovation project portfolio management performance:
the mediating effect of management perception and satisfaction”, R&D Management, Vol. 44
No. 5, pp. 498-515.
Szwejczewski, M., Mitchell, R. and Lemke, F. (2006), “A study of R&D portfolio management among
UK organisations”, International Journal of Management and Decision Making, Vol. 7 No. 6,
pp. 604-616.
Teller, J. and Kock, A. (2013), “An empirical investigation on how portfolio risk management influences
project portfolio success”, International Journal of Project Management, Vol. 31 No. 6, pp. 817-829.
Teller, J., Kock, A. and Gemünden, H.G. (2014), “Risk management in project portfolios is more than
managing project risks: a contingency perspective on risk management”, Project Management
Journal, Vol. 45 No. 4, pp. 67-80.
Teller, J., Unger, B.N., Kock, A. and Gemünden, H.G. (2012), “Formalization of project portfolio
management: the moderating role of project portfolio complexity”, International Journal of
Project Management, Vol. 30 No. 5, pp. 596-607.

Appendix

Sections Description

1 Questions about general profile of respondents


2 General questions about companies portfolio management process
3 Questions related to how portfolio management is done and projects selected
4 Questions related to the nature of the portfolio management methods used
5 Questions about the level of satisfaction of the portfolio assessment and project selection methods Table AI.
6 Questions related to the performance results of the portfolio management methods, by covering Questionnaire
performance measures and success factors for the new product and service development program structure

Corresponding author
Paulo A. Cauchick-Miguel can be contacted at: paulo.cauchick@ufsc.br

For instructions on how to order reprints of this article, please visit our website:
www.emeraldgrouppublishing.com/licensing/reprints.htm
Or contact us for further details: permissions@emeraldinsight.com