Escolar Documentos
Profissional Documentos
Cultura Documentos
. When L
1
and L
2
metrics are used the models
derived are non-linear. In case of the L
. When L
1
and L
2
metrics are used the models
derived are non-linear. In case of the L
= = (2)
The Chebychev norm is a distance function where the absolute value of the largest
coordinates difference between two points absolutely dominates. The L
1
metric can
be calculated by the following equation,
1
1
|| ||: min (1 )
n
k
k
d h
=
=
(3)
The Manhattan norm hat represents the shortest distance in unit steps along
each axis between two points. The case of L
2
is calculated from,
2
2
1
|| || : min (1 )
n
k
k
d h
=
=
(4)
The Euclidean norm between two points is the length of the straight line between the
two points and it is by far the most commonly used norm.
Therefore, if this metric is used in Global DEA, only the largest factors difference is
taken into account (thus leading to the most balanced solution between achievements
of different factors).
For the case of L
, the above model Eq. (1) although not linear it can be handled
through bisection search - Despotis (2002) [0] using the following equivalent form,
|| || min d z
= (5)
s.t.
1 1
0, 1,...,
s m
r rj i ij
r i
u y v x j n
= =
=
1
1
1 ( ) 0, 1,...,
s
r rj
r
m
i ij
i
u y
z j n
v x
=
=
+ =
, ,
r i
u v r i
0 z
For the case of L1, the model becomes
1
1
|| || min , j=1,...,n
s
j
r
d d
=
=
(6)
Proceedings of the 10
th
International Conference on DEA Brazil2012 12
s.t.
1
1
1, j=1,...,n
s
r rj
r
j m
i ij
i
u y
d
v x
=
=
+ =
, ,
r r
u v r i
0, j=1,...,n
j
d
and for L2,
2
2
1
|| || min , j=1,...,n
s
j
r
d d
=
=
(7)
s.t.
1
1
1, j=1,...,n
s
r rj
r
j m
i ij
i
u y
d
v x
=
=
+ =
, ,
r r
u v r i
0, j=1,...,n
j
d
The above two models are non-linear.
GA DEA
Since Eq. (1) is not linear, it can be approached with Genetic Algorithms. Multi-
objective Evolutionary Algorithms (EAs) are GAs customized to solve multi-objective
problems by using specialized fitness functions. EAs, such as Non-dominated Sorted
Genetic Algorithm NSGA-II can be modified to find a set of multiple non-dominated
solutions in a single run.
Multiobjective Evolutionary Algorithms (EAs) are GAs customized to solve multi-
objective problems by using specialized fitness functions. In order to achieve
evolutionary multi-objective optimization, three different aspects must be considered:
Fitness Assignment, Diversity Mechanism and Elitism. Fitness Assignment is actually
the fitness function chosen. Diversity Mechanism is the way next population is
generated. Elitism is whether the best dominating solutions found so far survive to the
next generation. Therefore a controlled genetic algorithm, which is a variant of Non-
dominated Sorted Genetic Algorithm (NSGA-II), is proposed in this research. For
fitness function Pareto Ranking approach is utilized. The population is ranked
according to a dominating rule. In order to maintain the Diversity, crowding distance
is employed, while Elitism exists partially. An elitist GA always favours individuals
with better fitness value (rank). A controlled elitist GA also favours individuals that
can help increase the diversity of the population even if they have a lower fitness
Proceedings of the 10
th
International Conference on DEA Brazil2012 13
value. It is important to maintain the diversity of population for convergence to an
optimal Pareto front. Diversity is maintained by controlling the elite members of the
population as the algorithm progresses.
The main advantage of evolutionary algorithms, when applied to solve multi-objective
optimization problems, is the fact that they typically optimize sets of solutions,
allowing computation of an approximation of the entire Pareto front in a single
algorithm run. The main disadvantage of evolutionary algorithms is the much lower
speed. Through multi-objective solution global weights are acquired, rendering some
DMUs on the Pareto front efficiency frontier. This is because a DMU is (Pareto)
efficient if there is no other DMU, or a non negative convex linear combination of m
inputs and s outputs of some DMUs, that improves one DMU score without worsening
at least one of the other DMU score for all objectives simultaneously. Finally in order
to choose the best solution we apply the LP metric. In order to select one from the
Pareto front Eq. (3~5) are used. Below are the parameters used to execute GA DEA.
Table 1. Parameters used in Genetic Algorithm setup
Population size 75
Maximum number of generations 150
Generation gap (GGAP) 0.9
Mutation rate 0.5 under mutationadaptfeasible
Pareto front 0.5
Crossover strategy Double point crossing
Stopping criteria Either the best solution does not improve
for 20 generations or the maximum
number of generations has been reached.
Genetic Algorithm approach examines each solution in a specific area. After that, a
pareto front is formed with the dominating solutions. The selection of the best
solution is a decision of the decision maker. Therefore, by applying the metrics, the
decision maker can select the best solution minimizing the distance from ideal value 1.
Results and discussions
To examine this approach a simulation was conducted. In each case, we initially
generate data from a data generation process and then conduct 200 trials. The number
of DMUs for experimental purposes was 20, a relatively small number.
The inputs and the outputs of each DMU were produced from a productive function
( ) x
. Let n be the number of DMUs and m the number of inputs and s the number of
outputs for each DMU. Supposing a Data Generation Process that generates the inputs
ij
x x = . Therefore in order to produce the outputs
ij
y y = , the following equation was
used,
( )
1
m
x i
i
y x e
=
= =
(8)
Proceedings of the 10
th
International Conference on DEA Brazil2012 14
where
i
is generated independently from independent uniform random distribution
on the interval [0.6, 0.8]. Variable e is a small error multiplied with the production
function to emulate the inefficient DMUs.
For the simulation, mean differences were calculated for each metric. Genetic
Algorithm approach is compared with bisection linear programming (bLP) algorithm
(Eq. 5) using L metric (Eq. 2). Similarly it was compared with nonlinear L1 (Eq. 6)
and nonlinear L2 (Eq. 7) using the L1 (Eq. 3), and L2 (Eq. 4) metrics respectively.
Taking different cases into account, combinations of 2-4 inputs and 2-4 outputs are
tested. In Table 2, the mean differences are displayed.
Table 2. Mean differences of minimum distances using L
, L
1
and L
2
metric
bLP
d
- GA
d
nonLP
d1
- GA
d1
nonLP
d2
- GA
d2
IxO Mean Range Mean Range Mean Range
2x2 0.0571 [0.0066 , 0.2176] 0.0496 [0.0259 , 0.2491] 0.0137 [0.0096 , 0.0338]
2x3 0.0556 [0.0096 , 0.3149] 0.0567 [0.0024 , 0.2226] 0.0272 [0.0018 , 0.0478]
2x4 0.0437 [0.0169 , 0.1275] 0.0190 [0.0031 , 0.1190] 0.0318 [0.0064 , 0.0732]
3x2 0.0478 [0.0034 , 0.2140] 0.0439 [0.0167 , 0.1331] 0.0314 [0.0018 , 0.1882]
3x3 0.0515 [0.0089 , 0.1705] 0.0110 [0.0064 , 0.1516] 0.0380 [0.0038 , 0.1622]
3x4 0.0522 [0.0140 , 0.2165] 0.0950 [0.0528 , 0.2076] 0.0430 [0.0294 , 0.1700]
4x2 0.0434 [0.0062 , 0.2104] 0.0110 [0.0064 , 0.2516] 0.0625 [0.0093 , 0.2538]
4x3 0.0447 [0.0085 , 0.1693] 0.0560 [0.0019 , 0.0772] 0.0427 [0.0252 , 0.2005]
4x4 0.0431 [0.0118 , 0.1775] 0.0496 [0.0222 , 0.0537] 0.0137 [0.0083 , 0.1862]
Table 2 shows that the mean values are close to zero. The minimum distances using
different metrics are obtain with a single run of the genetic algorithm. It is also
important to note that the range of the values is also significant low. A further
inspection shows that the overall efficiency scores from GA and bLP do not vary
much.
Conclusions
In this paper, a genetic algorithm is presented to estimate the efficiency scores with
common weights. No prior research is been made towards using genetic algorithms in
order to solve multi objective DEA. Different metrics are used as part of the algorithm
in order to minimize distance of all DMUs from efficient value 1. The general fitness
framework utilizes the NSGA-II approach taking into consideration the LP metric.
A simulation is conducted in order to review the capability of the algorithm. The
results indicated that the genetic algorithm is capable of minimizing the distance from
optimum value 1. In other words, through optimization of the genetic algorithm there
is a set of common weights that minimizes the total distance. In scenarios with various
number of inputs and outputs, the data has been analyzed and metrics are computed.
It is proved that using Genetic Algorithms is a viable solution to estimate efficiency
scores with common weights, and is in fact less complex than nonlinear equivalent. In
addition, the efficient units are slightly increased since the genetic algorithm uses
pareto front optimization.
Proceedings of the 10
th
International Conference on DEA Brazil2012 15
References
Dyson R.G., Thanassoulis E. (1988). Reducing weight flexibility in data envelopment
analysis, Journal f the Operational Research Society (39): 563576.
Angulo-Meza, L., Estellita Lins, M.P. Review of methods for increasing discrimination in
data envelopment analysis Annals of Operations Research 116(1-4): 225-242.
Deb K., Pratap A., Agarwal S., Meyarivan T. (2002). A fast and elitist multi-objective
genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation (6): 182
197.
Kao, C. , Hung, H.-T. (2005) Data envelopment analysis with common weights: The
compromise solution approach Journal of the Operational Research Society 56(10):
1196-1203.
Liu, F. - H. F. , Hsuan Peng, H. (2008) Ranking of units on the DEA frontier with
common weights Computers and Operations Research 35(5): 1624-1637.
Wang, Y. M. , Chin, K. S. (2010) A neutral DEA model for cross-efficiency evaluation
and its extension Expert Systems with Applications 37(5): 3666-3675.
Kao, C. (2010) Malmquist productivity index based on common-weights DEA: The
case of Taiwan forests after reorganization Omega 38(6): 484-491
Wu, J. , Liang, L. , Yang, F. (2009) Determination of the weights for the ultimate cross
efficiency using Shapley value in cooperative game Expert Systems with Applications
36(1): 872-876
Despotis D.K. (2002). Improving the Discriminating Power of DEA- Focus on Globally
Efficient Units. Journal of the Operational Research Society (53): 314-323.
Acknowledgements
This study is funded and supported by the Institute of National Funds of Greece, since
one of the authors is under financial scholarship.
Proceedings of the 10
th
International Conference on DEA Brazil2012 16
2. A new approach to assess
performance of the Brazilian
national immunization program
(NIP)
Castro Lobo MS
Federal University of Rio de Janeiro, University Hospital HUCFF/UFRJ, University City/Fundo
Island, SEAV 5 andar, sala 5 A 26, R. Professor Rodolfo Rocco, 255, 21941-913 Rio de Janeiro,
RJ, Brasil, ms.lobo@superig.com.br (corresponding author)
Menegolla IA
Pan-American Health Organization PAHO, Setor de Embaixadas Norte, Lote 19, CEP 70800-400
Federal District, Brazil Caixa Postal 08-729, 70312-970 Brasilia, DF, Brasil,
ivonemenegolla@yahoo.com.br
Estellita Lins MP
Alberto Luiz Coimbra Post-Graduation and Engineering Research Institute, Operational Research,
COPPE/UFRJ, Technology Center, Block F, Room 103, University City/ Fundo Island, Rio de
Janeiro, Brazil, estellita@pep.ufrj.br
Abstract
The study develops an alternative measure of efficiency to assess the Brazilian
National Immunization Program, using Data Envelopment Analysis (DEA), output
oriented, Variable Returns to Scale (VRS) model, in order to congregate differing
indicators in a unique index and to consider the differences among the federal units
when comparing the twenty six Brazilian states that have diverse socioeconomic and
urbanization scenarios. The NIP program can be considered highly efficient in Brazil.
The mean efficiency score for the 26 states was 93.7 % (5.4 % SD). 16 states were
considered efficient. To reach the frontier of best practices, each state and region
could have an individual goal for vaccine homogeneity. DEA technique evaluates
homogeneity indicators for various vaccines in the same model making it possible to
construct an efficiency index for the first year of life immunization cycle.
Keywords: Data Envelopment Analysis, Health Services Assessment, Public Health
Policy, National Program of Immunization
Introduction
Immunization is the process whereby a person is made immune or resistant to an
infectious disease, typically by the administration of a vaccine. Vaccines stimulate the
bodys own immune system to protect the person against subsequent infection or
disease. Immunization is n a proven tool for controlling and eliminating life-
threatening infectious diseases and is estimated to avert between 2 and 3 million
deaths each year. It is one of the most cost-effective health investments, with proven
Proceedings of the 10
th
International Conference on DEA Brazil2012 17
strategies that make it accessible to even the most hard-to-reach and vulnerable
populations.
The Brazilian National Immunization Program (NIP) as a health policy nowadays
practiced dates from 1973. The NIP is public, universal and managed by state or
provinces' health authorities. The major challenges for the program are logistics for a
country that have continental dimensions and the necessity of qualified human and
material resources to cope with that. NIP is organized by life cycles (children,
teenagers, adults and old people) and vaccines are applied as routine or by campaign
means. For children under one year-old, the vaccines are to prevent: severe
tuberculosis (BCG), poliomyelitis (polio), B hepatitis (HepB), diphteria, tetanus,
whooping cough and haemophilus influenzae B (tetravalent), rotaviruses,
pneumococcus, meningoencephalitis, and measles, mumps and rubeola (trivalent
MMR), yellow fever (in endemic States).
According to WHO, the vaccination programs efficiency is assessed by three main
indicators, which are used for each dose, and each kind of vaccine, giving ways to
long spreadsheets and datasets, turning the analysis sometimes exhaustive and difficult
to summarize. They are:
a) Coverage, a key measure of immunization systems performance, is calculated
by the actual number of applications divided by expected number of
applications, according to the demographic structure of the state. The index is
expected to be above 90%, ideally 100%;
b) Homogeneity, that is, the proportion of municipalities inside the state with
coverage above 95%. The importance of this indicator is based in its capacity
for herd immunity, which means that, even if you are not vaccinated, if you
live in a place where all around you are immune, you may be considered
protected as well;
c) Abandon Rate, that is, for vaccines that must be repeated, as tetravalent and
poliomyelitis, the difference between the number of applications for last and
the first given doses, divided by the number of first doses. These are proxies to
the fact that a person had access to all doses that guaranteed full protection.
The expected value should be below 5%.
As there is no sense in trading off the usual goals for coverage and abandon rate, this
paper intends to develop an alternative measure of efficiency to assess the Brazilian
Immunization Program in 2010, using Data Envelopment Analysis (DEA) in order to
congregate the differing homogeneity indicators as a unique index and to consider the
different resource facilities observed when comparing the twenty six Brazilian states
that together give this country a continental dimension, with diverse socioeconomic
and urbanization scenarios.
Proceedings of the 10
th
International Conference on DEA Brazil2012 18
Methods
Many performance and benchmark reviews approach the issue of health care
assessment from the efficiency perspective. Chillingerian and Sherman, Hollingsworth
and ONeill et al. health care efficiency studies collectively provided comprehensive
overview of the theme, pointing out the general advantages, concerns and limitations
of applying these methods in multi-product organizations, practices in public policies
or programs.
The most common used technique, Data Envelopment Analysis based on linear
programming, shows which health organizations are efficient, gives the magnitude of
inefficiency and indicates the means of improving efficiency by giving targets for each
of the inputs and/or outputs individually. Details of the technique are presented
elsewhere.
The DMUs, in our case are the Brazilian States, or the local units responsible for the
NIP organization and logistics. The ones over the frontier, therefore, efficient, have an
efficiency measure equals to 1.00 or 100%, while the DMUs located under the frontier
are inefficient (values between 0 and 1.00, or 100%). The production model used in
this work considers variable returns to scale (VRS) and is oriented to the increase of
outputs to the projection in the frontier (maximization). The VRS model allows an
inefficient unit to be compared only with others efficient units of similar size or
operate in similar scale and is the choice to cope with the diversity of the States' sizes
and social scenarios. The orientation choice (output) admits the maximum success of
the results (higher homogeneity), given a fixed amount of resources.
Considering the fact that the traditional indicators used to assess NIP are already
fractions, which could jeopardize the linear properties of the frontier, the authors ran
a model assuming only the absolute values that generated the homogeneity indicators.
The inputs were the number of births in 2009 and the number of municipalities of
each State; the outputs were the number of municipalities with coverage over 95% for
the studied vaccines: BCG, HepB (3rd dose), polio (3rd dose), tetravalent (3rd dose)
and MMR (first). In this way, the DEA score index would comprise homogeneity for
many vaccines in the same model. Once calculated the number of municipalities that
needed to reach 95% coverage, the projection was used as a numerator to reconstruct
the homogeneity index to be pursued as a new goal.
All demographic and vaccination data were provided by the Brazilian Ministry of
Health.
Results and Discussion:
As observed in Figure I, there are clearly two distinct subgroups of Brazilian Regions,
that congregate, at one side: Regions North and Northeast, less developed, with lower
mean Human Development Index - HDI (0.73) and higher proportions of rural and
disperse populations (mean 25%); at the other side: Regions South, Southeast and
Center-West, with higher mean HDI (0.82), less rural population (mean 13%), and
where the bigger cities are located (with higher populations, higher number of
newborns). In a clockwise overview of Figure I, the first subgroup goes from
Rondnia to Bahia and the second subgroup goes from Minas Gerais to Gois. So
Proceedings of the 10
th
International Conference on DEA Brazil2012 19
Paulo is an outlier as the State comprises almost 22% of all the Brazilian population.
This demographic and socioeconomic difference has a sanitary impact, with lower
values for the first subgroup, especially concerning the indicators for homogeneity as
the access to remote and disperse populations can only be guaranteed by mobile
teams.
Table I shows the DEA scores, the new goals established for each homogeneity output
after the model was ran, and the benchmarks for the inefficient States. The mean
efficiency score for the 26 states was 93.7% (5.4 % SD), the minimum score was 53.8%
(Rondnia).
Given the different proportions of rural populations, each state could have an
individual goal for homogeneity to each vaccine to catch the frontier. In North and
Northeast Regions, the mean BCG homogeneity should raise from 40.1 % to 50.9 %;
HepB homogeneity, from 52.7% to 62.4 %; polio homogeneity, from 56,7% to 65.4 %;
tetravalent homogeneity, from 56.1% to 65.3 %; MMR homogeneity, from 55.4% to 64.6
%. In South, Southeast and Center-West Regions, the mean BCG homogeneity should
raise from 60.4% to 63.0%; HepB homogeneity, from 66.4% to 69.2 %; polio
homogeneity, from 72.4% to 75.8 %; tetravalent homogeneity, from 72.1% to 75.9 %;
MMR homogeneity, from 64.6% to 66.8 %. These mobile goals give a more realistic
scenario, that is, the DEA model brings an evidence-based figure of what goal should
be made plausible and attainable to the health manager.
For each inefficient State, the benchmarks were defined based on the projection in the
frontier of best practices. Sixteen States were efficient (number of references in
parenthesis): Acre (0), Roraima (6), Par (0), Amap (0), Tocantins (4), Maranho (1),
Pernambuco (0), Sergipe (1), Minas Gerais (3), Esprito Santo (4), So Paulo (1),
Paran (2), Santa Catarina (3), Rio Grande do Sul (0), Mato Grosso do Sul (3), Gois
(5). As seen in administrative practice, the respective benchmarks should be explored
even beyond the studied variables - as examples to orientate changes for better
results.
In the benchmarking process, care must be taken when comparing States with
different demographic and socioeconomic structures as the inefficient units are
projected into different parts (facets) of the best practice frontier. The risk of ranking
efficiency scores and misinterpretation must be highlighted because, when you show
these results to a health care authority, this is the usual first observation of the
manager, which can damage the reliability and face validity of the method.
In summary, from the multi-input multi-output model perspective, the DEA score in
a unique index - presents a picture of homogeneity performance of each State that
congregates many vaccine schemes offered in the first year of life; in some way,
substituting lots of datasheets that can even bring contradictory results. Indicators for
vaccine coverage and abandon rate should maintain the goals preconized by the
World Health Organization (WHO). On the other hand, the concise DEA measure can
be scrutinized to offer much more information, as: a) the new goals for each indicator,
so that the DMU can project and reach the best practice frontier (an useful tool for
health manager); b) the reference groups according to the facet of the frontier were
the inefficient unit should be projected (the benchmarks are the vertices of theses
Proceedings of the 10
th
International Conference on DEA Brazil2012 20
facets); c) the weights given to each variable, so that the model tend to prioritize the
ones that behave in a more efficient way comparing to the other units.
Figure I: Comparing Brazilian States according to Demographic Characteristics
Table II DEA Model Results: Scores, Immunization Goals and Benchmarks for
the NIP in Brazilian States
Proceedings of the 10
th
International Conference on DEA Brazil2012 21
Conclusions
DEA model is a potential tool to assess the Brazilian NIP that should be explored. As a
multi-input multi-output index, it allows the comparison of states that have diverse
resources and demographic structure and also establishes differing homogeneity goals
to each one, based on their real capabilities. Also, the DEA technique evaluates
homogeneity for various vaccines in the same model, giving an index that considers
the linear combination of all of them. In this way, it is possible to construct an
efficiency index for the first year of life immunization cycle, as the vaccines are
given at different months according to the immunization calendar.
The NIP program can be considered highly efficient as a whole in Brazil, with a mean
DEA efficiency homogeneity index of 93.7 %; mean coverage above 90% for all the
studied vaccines, mean tetravalent abandon around 5, 0 %. The goals for homogeneity
proposed by the model will displace all units towards the best practice frontier and, as
the efficient units could also ameliorate results; new frontiers would be constructed,
giving ways to technological change and a systemic development of the public policy.
References
Atkinson W., Wolfe C., Hamborski J. (2011) Epidemiology and Prevention of Vaccine-
Preventable Diseases. 12th edition. The Public Health Foundation; 2011.
Barreto M.L., Teixeira M.G., Bastos F.I., Ximenes R.A.A., Barata R.A. (2011) Successes
and failures in the control of infectious diseases in Brazil: social and environmental
context, policies, interventions, and research needs. The Lancet; 377: 1877-1889.
Chilingerian J.A., Sherman D. (2004) Health Care Applications - From Hospitals to
Physicians; From Productive Efficiency to Quality Frontiers. In: Cooper WW; Seiford
LM; Zhu J. Handbook on data envelopment analysis. Boston: Kluwer Academic
Publishers.
Hollingsworth B. (2003) Non-Parametric and Parametric Applications Measuring
Efficiency in Health Care Health Care Management Science;6:203-218.
Hollingsworth B. (2008) The Measurement of Efficiency and Productivity of Health
Care Delivery, Health Economics; 17 (10):1107-1128.
ONeill L, Rauner M, Heidenberger K, Kraus M. (2008) A cross-national comparison
and taxonomy of DEA-based hospital efficiency studies. Socio-Economic Planning
Sciences 2008; 42(3):158-189.
Cooper WW, Seiford LM, Tone K. (2007) Data Envelopment Analysis-A
Comprehensive Text with Models, Applications, References and DEA Solver Software.
2nd. Ed. Massachusetts: Springer
Ozcan YA. Health Care Benchmarking and Performance Evaluation: An Assessment
using Data Envelopment Analysis (DEA). International Series in Operations Research
and Management Science. Springer; 2008.
Lins MPE, Lobo MSC, Fiszman R, Silva ACM, Ribeiro VJP. O Uso da Anlise Envoltria
de Dados DEA - para Avaliao de Hospitais Universitrios Brasileiros. Revista
Cincia e Sade Coletiva 2007; 12(4): 985-998.
Proceedings of the 10
th
International Conference on DEA Brazil2012 22
3. A new approach to cross
efficiency evaluation based on
MILP model and measurement
of energy efficiency
Mehmet Guray UNSAL
Gazi University, Science Faculty, Statistics Department, Ankara, TURKEY mgunsal@gazi.edu.tr
(corresponding author)
Hasan BAL
Gazi University, Science Faculty, Statistics Department, Ankara, TURKEY hasanbal@gazi.edu.tr
H. Hasan ORKCU
Gazi University, Science Faculty, Statistics Department, Ankara, TURKEY hhorkcu@gazi.edu.tr
Abstract
DEA has become a very popular method of performance measure, but it still suffers
from some shortcomings. One of these shortcomings is the issue of having multiple
optimal solutions to weights for efficient DMUs. The cross efficiency evaluation as an
extension of DEA is proposed to avoid this problem. Lam (2010) is also proposed a
mixed-integer linear programming formulation based on linear discriminant analysis
and super efficiency method (MILP model) to avoid having multiple optimal solutions
to weights. In this study, we modified MILP model to determine more suitable weight
sets and also evaluate the energy efficiency of OECD countries as an application of
the proposed model.
Keywords: Data envelopment analysis, discriminant analysis, cross efficiency, MILP
model.
Introduction
DEA was first developed by Charnes et al. [3] that seems to be the most popular
method for measuring the efficiency of homogenous decision making units. It become
very popular method which is used in operations research and management science.
The improvements made to the DEA technique have resulted in several new problems
[2]. For example, the issue of unrealistic weights distribution, the weak discrimination
power, and having multiple optimal solutions to weights for efficient DMUs.
Having multiple optimal solutions to weights affects to a great extent the consistency
of operations related to weights cross efficiency method is the most frequently studied
topic in DEA literature. Sexton et al. [13] developed the cross efficiency method to rate
the DMUs. Their technique made use of the cross evaluation scores computed as
related to all DMUs and hence identified the best DMUs [1].
Proceedings of the 10
th
International Conference on DEA Brazil2012 23
Despite the extensive use of the cross efficiency method, it has some limitations
arising from the classical DEA. Doyle and Green [5] stated that the non-uniqueness,
i.e. having multiple solutions to optimal weights in the DEA, decreases the usefulness
of the cross efficiency method. Sexton et al. [13] and Doyle and Green [5]
recommended the use of a secondary objective (model) for the cross efficiency
evaluation related to the non-uniqueness of optimal weights in DEA. They proposed
the aggressive and benevolent models for achieving the secondary objective.
See also the following papers with applications of cross efficiency evaluation: Sexton
et al. [16] to nursing homes, Oral et al. [10] to R&D projects, Doyle and Green [4] to
higher education, Green et al. [5] to preference voting, Lu and Lo [9] to economic
environmental performance, Liang et al. [8], Ramon et al. [12], Lam [7], rkc and Bal
[11], Jahanshahloo et al. [6].
For having multiple optimal solutions problem in DEA, Lam [7] proposed a mixed-
integer linear programming formulation based on linear discriminant analysis and the
super efficiency method. In this paper, we modify this model and introduce a new
model to choose suitable weight sets to be used in cross efficiency evaluation.
Lam [7] used a h constant to separate efficient and inefficient DMUs. In the Lam [7]
model, h constant is determined from the super efficiency model. As h is considered
as a variable in our proposed model, it is not necessary to use the super efficiency
model to determine h. Lam [7] used three data scenario to illustrate his model. We also
have used these data scenario for the comparison methods. With this new
modification model, one can compare the efficiency scores and obtain a better picture
of cross efficiency stability with respect to multiple DEA weights. The results obtained
from three different data scenario in the related literature show that proposed model is
compatible with the most used cross efficiency models and make valid contributions
to cross efficiency evaluation.
A New Approach to Cross Efficiency Evaluation Based on the Lam
Model
The cross efficiency method was developed as a DEA extension tool to be utilized for
identifying the best performing DMUs, and for ranking DMUs using cross efficiency
scores that are linked to all DMUs [13,15]. The basic idea of the cross efficiency
method that alleviates the weak discrimination of the classical DEA model can be
explained in two stages: In the first stage, the classical DEA analysis is performed, and
the optimal weights of inputs and outputs are calculated for each DMU. However, the
optimal weights computed by classical DEA have multiple solutions, especially for the
efficient DMUs, and these solutions provide unrealistic weights, i.e., weights with
extreme or zero values. In the second stage, these drawbacks are reduced, and a
suitable set of weights preserving the efficiency values obtained by DEA is selected for
each DMU.
In the first stage, the optimal weights of inputs and outputs are calculated for each
DMU using the classical DEA formulation. Given the results of the first stage, the
weights used by the DMU can be utilized for calculating the peer rated efficiency for
Proceedings of the 10
th
International Conference on DEA Brazil2012 24
each of the other DMUs. The peer evaluation score,
, p j
, indicates the efficiency score
for the DMU
j
using the weights obtained by the DMU
p
[15].
,
1
,
,
1
s
r p rj
r
p j m
i p ij
i
u y
v x
=
=
=
. (4)
Lam [7] proposed a mixed-integer linear programming (MILP) formulation based on
linear discriminant analysis and the super efficiency method. The MILP formulation is
summarized as follows:
In the first step, the CCR model given in equation (3) is used to determine the
efficiency ratio for each DMU. Then, based on the efficiency ratios, the DMUs are
classified as either efficient ( E ) or inefficient ( E ).In the second step, an MILP model
is run for each efficient DMU in E. The objective of the MILP model is to separate the
efficient and inefficient DMUs while keeping the efficiency ratio of the DMU under
evaluation to be the highest. The intuition of keeping the efficiency ratio of the DMU
under evaluation as the highest is that the obtained weight set will then reflect the
relative strengths of the efficient DMU under consideration over the other DMUs. All
the obtained weight sets are used to compute cross efficiency ratios for all DMUs. The
efficient DMU under evaluation is expressed thus,
e
DMU . The MILP model proposed
by Lam [7] can be stated as follows:
1
n
j
j
Min z
=
. . s t
1 1
0, ,
s m
r rj i ij j
r i
u y v x M z j E
= =
+
1 1
, ,
s m
r rj i ij j
r i
u y v x M z j E
= =
1
1,
m
i ie
i
v x
=
=
(5)
1
,
s
r re
r
u y h
=
1 1
0 ; 1, , , ,
s m
r rj i ij
r i
u y h v x j n j e
= =
| |
=
|
\ .
.
{ } 0,1 , 1, , ,
j
z j n = .
, 0, 1, , , 1, ,
r i
u v r s i m = = . .
where, is a very small positive number, M is an extremely large positive number,
E is the efficient set which contains all the efficient DMUs, while E contains all the
inefficient DMUs. The value of h is predetermined. In the MILP model proposed by
Proceedings of the 10
th
International Conference on DEA Brazil2012 25
Lam [7], constant h is determined by the super efficiency model and in fact, the value
of h is based on the idea of the cross efficiency calculation. In the Lam [7] model, the
constant h is determined from the constraint is given in (6).
1 1
0 ; 1, , , , (6)
s m
r rj i ij
r i
u y h v x j n j e
= =
| |
=
|
\ .
.
If this constraint is organized,
1
1
; 1, , , ,
s
r rj
r
m
i ij
i
u y
h j n j e
v x
=
=
=
.
(7)
is obtained. Lam [7] proposed to determine h with the super efficiency method due to
the difficulty of fractional programming problems. Due to h>1, it seems appropriate to
determine h by the super efficiency method.
The constraint given with (8) was selected instead of this constraint in this study.
1 1
; 1, , , ,
s m
r rj i ij
r i
u y v x h j n j e
= =
=
. (8)
If the constraint given with (8) is organized, (
1
0
m
i ij
i
v x
=
),
1 1
1 1 1
; 1, , ,
m s
i ij r rj
i r
m m m
i ij i ij i ij
i i i
v x u y
h
j n j e
v x v x v x
= =
= = =
=
. (9)
is obtained. If the constraint given with (9) is organized,
1
1 1
1 ; 1, , , ,
s
r rj
r
m m
i ij i ij
i i
u y
h
j n j e
v x v x
=
= =
+ =
. (10)
is obtained.
Here, the value of h is selected as a variable, being h>1. When the constraint given
with (10) is examined, it can be seen that the efficiency ratio (
1 1
s m
r rj i ij
r i
u y v x
= =
)
remains lower than a value higher than 1. In the MILP model, the efficiency ratio is of
a value lower than the value of h (h>1). That is to say, the efficiency ratio is similar to
the MILP model in the model that we propose also. While the model that we propose
determines the threshold value of h itself for the efficiency ratio for the model as well,
the value of h in the MILP model is predetermined with the super efficiency model. As
h is considered as a variable in our proposed model, it is not necessary to use the
super efficiency model to determine h. Besides, our proposed model is not an integer
linear programming. The model suggested is given in the equation (11).
Proceedings of the 10
th
International Conference on DEA Brazil2012 26
1
n
j
j
Min d
=
. . s t
1 1
0, ,
s m
r rj i ij j
r i
u y v x d j E
= =
+
1 1
, ,
s m
r rj i ij j
r i
u y v x d j E
= =
1
1, (11)
m
i ie
i
v x
=
=
1
,
s
r re
r
u y h
=
1 1
, 1, , , ,
s m
r rj i ij
r i
u y v x h j n j e
= =
=
.
1 h
0, 1, ,
j
d j n = .
, 0, 1, , , 1, ,
r i
u v r s i m = = . .
where, is a very small positive number, E is the efficient set which contains all
efficient DMUs, while E contains all inefficient DMUs.
Application Study
According to an application study on energy data of OECD countries, we consider
gross domestic product and nonfossil fuel consumption as outputs variables. CO
2
Emission and fossil fuel consumption are the inputs similarly
Ramanathans study [14] in Table 1. We obtained the data set from International
Energy Agency web page http://www.eia.doe.gov. The data belongs to 2008.
Table 1. Outputs and input variables for measuring of
OECD Countries energy consumption and CO
2
emission efficiency
Efficiency Measurement
Approach
Inputs Outputs
Energy Consumption and
CO
2
Emission
(Ramanathan,2005)
CO2 emissions, fossil fuel
energy consumption (FOSS)
Gross domestic product
(GDP), non-fossil fuel
energy consumption
(NFOSS)
Proceedings of the 10
th
International Conference on DEA Brazil2012 27
Table 2. Spermans Rank Correlations According to Rankings of DMUs
CCR SUPER CROSS MILP PROPOSED
CCR 1 0.998 0.924 0.973 0.938
SUPER 0.998 1 0.925 0.974 0.946
CROSS 0.924 0.925 1 0.910 0.857
MILP 0.973 0.974 0.910 1 0.942
PROPOSED 0.938 0.946 0.857 0.942 1
Table 3. Efficiency ratios of models for OECD Countries
Country
CCR
Eff.
Ratio
R
a
n
k
Super
Eff.
Ratio
R
a
n
k
Cross
Eff.
Ratio
R
a
n
k
Lam
model
Eff.
R
a
n
k
Prop
osed
Mode
R
a
n
k
AUSTRALIA 0,1073 24 0,1073 24 0,3639 8 0.1556 25 0.0836 25
AUSTRIA 0,3023 12 0,3023 12 0,1929 14 0.3813 13 0.3041 13
BELGIUM 0,3505 11 0,3505 11 0,2523 12 0.5326 10 0.3622 9
CANADA 0,5986 7 0,5986 7 0,4041 6 0.8569 4 0.4582 6
CZECH REPUBLIC 0,1870 20 0,1870 20 0,1272 20 0.2626 19 0.1884 18
DENMARK 0,1613 22 0,1613 22 0,1004 24 0.1945 23 0.1626 21
FINLAND 0,6207 6 0,6207 6 0,3751 7 0.7085 7 0.6475 3
FRANCE 1 3 2,3988 2 0,8757 1 1.9039 1 1.1272 1
GERMANY 0,5910 8 0,5910 8 0,3567 9 0.7322 5 0.3724 7
GREECE 0,0658 28 0,0658 28 0,0466 27 0.0979 27 0.0676 28
HUNGARY 0,2091 18 0,2091 18 0,1237 22 0.2365 21 0.1991 17
IRELAND 0,0719 27 0,0719 27 0,0444 28 0.0854 28 0.0731 26
ITALY 0,1788 21 0,1788 21 0,1248 21 0.2660 18 0.1441 22
JAPAN 1 3 1,0875 4 0,5257 3 0.7089 6 0.3442 10
KOREA 1 3 2,1397 3 0,5106 5 0.5992 9 0.3182 11
LUXEMBOURG 0,0595 29 0,0595 29 0,0253 29 0.0521 29 0.0511 29
MEXICO 0,2202 17 0,2202 17 0,1495 18 0.3133 16 0.1682 20
NETHERLANDS 0,0898 25 0,0898 25 0,0745 25 0.1618 24 0.0937 24
NEW ZEALAND 0,3022 13 0,3022 13 0,1827 15 0.3451 14 0.3150 12
NORWAY 0,4685 10 0,4685 10 0,2814 11 0.5304 11 0.4991 5
POLAND 0,0734 26 0,0734 26 0,0571 26 0.1234 26 0.0698 27
PORTUGAL 0,2038 19 0,2038 19 0,1283 19 0.2507 20 0.2054 16
SLOVAK REPUBLIC 0,2531 15 0,2531 15 0,1528 17 0.2883 17 0.2649 14
SPAIN 0,2726 14 0,2726 14 0,2112 13 0.4570 12 0.2580 15
SWEDEN 1 3 2,0432 1 0,6001 2 1.1419 2 1.1082 2
SWITZERLAND 0,5568 9 0,5568 9 0,3353 10 0.6320 8 0.5872 4
TURKEY 0,1221 23 0,1221 23 0,1017 23 0.2208 22 0.1281 23
UNITED KINGDOM 0,2356 16 0,2356 16 0,1530 16 0.3212 15 0.1690 19
USA 1 3 4,0592 5 0,5141 4 0.9146 3 0.3651 8
Proceedings of the 10
th
International Conference on DEA Brazil2012 28
Table 4. Outputs and inputs weights obtained from Lams MILP
and proposed model (for efficient DMUs)
The proposed model produces far fewer zero weights than Lams MILP. As pointed
out by Cooper et al [16], it is better to use weight sets which have more balanced
virtual weights than extreme virtual weights in measuring the performance of the
DMUs in the DEA. The results of application study are listed in Table 2-4. All of the
models have significant and high correlation coefficients between each other. This
solution is lead us these models have same usefulness in ranking of DMUs.
Furthermore, the proposed model has an ability to find of value of h by itself. Another
significant advantage of our modified MILP model is that it can effectively reduce the
number of zero weights for inputs and outputs as seen in Table 3.
Conclusions
Lam [7] proposed a mixed-integer linear programming (MILP) formulation based on
linear discriminant analysis and super efficiency method. In this paper, we modify this
model to choose suitable weight sets to be used in cross efficiency evaluation. The
weight sets obtained from the proposed model are suitable for cross-evaluation
because they reflect the different strengths of the efficient DMUs. The proposed model
produces far fewer zero weights than Lams MILP. The results are obtained from
energy consumption and CO
2
emmission efficiency of OECD Countries.
References
[1] T.R. Andersen, K.B. Hollingsworth, L.B. Inman, (2002) The Fixed Weighting Nature
of a Cross Evaluation Model, J. Prod. Anal., 18 (1) 249255.
[2] H. Bal, H.H. rkc, S. elebiolu, (2010) Improving the Discrimination Power and
Weight Dispersion in the Data Envelopment Analysis, Comput. Oper. Res., 37 (1) 99
107.
[3] A. Charnes, W.W. Cooper, E. Rhodes, (1978) Measuring the Efficiency of Decision
Making Units, Eur. J. Oper. Res., 2 429444.
[4] J.R. Doyle, R. Green, (1994) Efficiency and Cross Efficiency in Data Envelopment
Analysis: Derivatives, Meanings and Uses, J. Oper. Res. Soc., 45 (5) 567578.
[5] R. Green, J.R. Doyle, W. Cook, (1996) Preference voting and project ranking using
DEA and cross-evaluation, Eur. J. Oper. Res., 90 (3) 461472.
MODELS
COUNTRIES u1 u2 v1 v2 h u1 u2 v1 v2 h
FRANCE 0 0,0481 85,3197 0,0153 2,35 0 0,0262 19,8738 0,0185 1,2820
JAPAN 0 0,0445 797,6734 0,0033 2,35 0 0,0126 1,5746 0,0120 1
KOREA 0 0,0387 427,6736 0,0023 2,35 0,0001 0,0259 38,4121 0,0113 46,0388
SWEDEN 0 0,0348 0 0,0304 2,35 0 0,0289 6,9926 0,0262 1,9457
USA 0 0,3683 5434,7830 0 2,35 0 0,0675 704,6617 0,0102 1
LAM MODEL PROPOSED MODEL
Proceedings of the 10
th
International Conference on DEA Brazil2012 29
[6] G.R. Jahanshahloo, L.F. Hosseinzadeh, Y. Jafari, (2011) R. Maddahi, Selecting
symmetric weights as a secondary goal in DEA cross-efficiency evaluation, Appl. Math.
Model., 35, 544549.
[7] K.F. Lam, (2010) In the determination weight sets to compute cross-efficiency ratios
in DEA, J. Oper. Res. Soc., 61 134143.
[8] L. Liang, J. Wu, W.D. Cook, J. Zhu, (2008) Alternative secondary goals in DEA
cross-efficiency evaluation, Int. J. Prod. Eco., 113 (2) 10251030.
[9] W.M. Lu, S.F. Lo, (2007) A closer look at the economic-environmental disparities
for regional development in China, Eur. J. Oper. Res., 183 (2) 882894.
[10] M. Oral, O. Kettani, P. Lang, (1991) A methodology for collective evaluation and
selection of industrial R&D projects, Manage. Sci., 37 (7) 871885.
[11] H.H. rkcu, H. Bal, (2011) Goal programming approaches for data envelopment
analysis cross efficiency evaluation, Appl. Math. Comput., 218 346-356.
[12] N. Ramn, J.L. Ruiz, I. Sirvent, (2010) On the choice of weights profiles in cross-
efficiency evaluations, Eur. J. Oper. Res., 207 (3) 15641572.
[13] T.R. Sexton, R.H. Silkman, A.J. Hogan, (1986) Data Envelopment Analysis:
Critique and Extension. In: Silkman R.H. (Ed.), Measuring Efficiency: An Assessment of
Data Envelopment Analysis, 32. Jossey-Bass, San Francisco 73-105.
[14] R. Ramanathan, (2005) An analysis of energy consumption and carbon dioxide
emissions in countries of the Middle East and North Africa, Energy, 30 28312842.
[15] T.R. Andersen, K.B. Hollingsworth, L.B. Inman, (2002) The Fixed Weighting
Nature of a Cross Evaluation Model, J. Prod. Anal., 18 (1) 249255.
[16] W.W. Cooper, J.L. Ruiz, I. Sirvent, (2007) Choosing Weights From Alternative
Optimal Solutions of Dual Multiplier Dual Models in DEA, Eur. J. Oper. Res., 180 443
458.
Proceedings of the 10
th
International Conference on DEA Brazil2012 30
4. An effectiveness analysis of
different techniques for
development of IT software
projects
Marco Mendes
School of Economics - FACE - Management Department UFMG, marco-mendes@ufmg.br
Ana Lcia Miranda Lopes
School of Economics - FACE - Management Department UFMG, analopes.ufmg@gmail.com
Rajiv Banker
Fox School of Business and Management Temple University, banker@temple.edu
Abstract
Software development best practices claim to reduce IT projects risks and deliver
better and efficient software products. Yet widely accepted on industry, these
techniques are the subject of intense debate in academia. Objective: This paper
presents a quantitative analysis of the impact of a set of software development
techniques on the technical efficiency of software projects. A study is conducted on
105 software development projects for efficiency analysis of effort, time elapsed,
productivity and defect density. The following software practices are evaluated:
capability maturity models like CMMI, requirements elicitation techniques, design and
architecture techniques, test techniques, project management techniques, business
process management use and case tools adoption. Method: Benchmarking is
performed using efficient frontier analysis with DEA BCC input oriented. Efficiency
scores among software project groups are compared using DEA based hypothesis
tests. Results: No single software engineering technique could explain an increase in
overall performance in IT projects. Yet, we find some evidences that IT firms are more
efficient than other firms types when delivering software projects. Conclusion: Results
emphasize the importance of recognizing that optimal management techniques
depend on the characteristics of the software development project, organization type
and its sociotechnical environment.
Keywords: Software Economics, DEA BCC, Information Systems, CMMI, Software
Architecture
Introduction
Information technology (IT) has been used for over 50 years as a plausible instrument
for increasing efficiency in firms business processes. Several software engineering best
practices such as capability maturity models, project management techniques,
requirement, design and test techniques are proposed instruments to reduce software
Proceedings of the 10
th
International Conference on DEA Brazil2012 31
development risks and to deliver better products. Although the value of these
practices is widely accepted in industry, formal analysis of its benefits has been the
subject of great debate in academia.
In this context, this paper promotes a discussion of the influence of software
development practices on technical efficiency of software projects. The term technical
efficiency is used according to Farrell [1] and it means the product results
maximization from a predetermined set of inputs. For example, if a set of software
project of the same size and effort reports different defect densities the projects with a
lower defect density are classified as more efficient. The key question posed in this
study is: the use of recognized software engineering practices could yield superior
project performance? Economic efficiency has great significance and encourages
managers in directorial search for techniques that can yield software products in less
time and effort or with fewer defects.
The paper is organized as follows. Section 2 presents related work in IT projects
technical efficiency. Section 3 presents the DEA based methodology and DEA based
hypothesis test for comparison of groups of IT projects. Section 4 presents the results
and discussion of the experiment, followed by conclusion in section 5.
Related Work
A longitudinal study conducted since 1985, compiled the report Chaos Manifesto [2],
indicates that the majority of IT projects present efficiency problems regarding the
quality as perceived by its users, project deadlines and costs. In a meta-analysis of
investments in IT services Brynjolfsson [3] studied the IT productivity paradox, arguing
that the use of IT often does not generate the expected returns on investment. The
same author presents in the later work efficiency factors when information technology
is linked to the innovation of business processes [4]. Since the 70s, software costs
exceed hardware costs in IT in a range of up to 5:1 [5] [6]. Currently this ratio exceeds
10:1 and continues to grow. With respect to the unit of analysis for software projects,
several investigations have been conducted to determine factors that guide economic
efficiency. Factors related to product size, teams, technologies and construction
process are more accepted in the literature. These aspects are discussed in the classic
book The Mythical Man-Month [7] and its elements already indicate diseconomies of
scale. Early econometric models such as SLIM and COCOMO II [6] are seminal
references in determining these factors and also presents evidence of diseconomies for
large projects.
Yet classical analyses suggest decreasing returns to scale for software projects, other
authors present divergent arguments [8]. Banker and Kemerer [9] analysed several IT
project databases and noticed that some of them exhibit economies of scale. The same
authors in subsequent work [10] present similar evidence in the context of software
maintenance and other projects database. Over the past years, investigations by other
authors show evidence of varying scales in projects [11].
Academic software economics literature encompasses many quantitative methods of
analysis. Pioneering work in the 70s and 80s were based simply on multiple linear
regression methods. A systematic review of measurement and prediction of
productivity of software projects analyses the main quantitative methods in use in 38
Proceedings of the 10
th
International Conference on DEA Brazil2012 32
selected papers [12]. The author indicates that methods like DEA (Data Envelopment
Analysis), Bayesian networks and discrete event simulation are promising.
This paper investigates the effects of some software engineering practices on the
performance of software projects. We analyse the following practices:
Capability maturity models, like SEI CMMI. These maturity models claim to
deliver better results.
Software development life cycle techniques, like requirements, design,
architecture and test techniques.
Business Process Management techniques. Business Process Management is
supposed to deliver better IT and business alignment.
Case tools adoption. Case tools are used to automate modelling and reduce
design errors. Again, their use is expected to improve project performance.
Project planning and monitoring techniques. Use of managing techniques and
standards, like PMBOK, is supposed to help project performance.
Methods
This paper investigates the hypothesis that software development best practices leads
to better project outcomes.
To investigate this hypothesis, it is necessary to understand the relationships of
efficiency in software projects and nature of returns to scale. Equation (1) is usually
used to establish the relative efficiency of software development projects from the
correlation between the functional size (lines of code implemented, function points,
tables, databases) and their effort in hours.
=
(1)
< 1 o
= 1
> 1
Methods such as COCOMO II use this equation as basis. In COCOMO II, the a term is
a 14 factor composite. Although COCOMO II does not use modern software
development terminology, some of its attributes are linked to aspects of software
design attributes such as Application of software engineering methods, Software
Complexity or Required Software Reliability of the Product.
If we assume that terms a and b are known to a certain project database in a firm, it is
expected that similar size projects have similar efforts. In the presence of different
efforts we can compare the efficiency of these projects. Projects that present lower
efforts when compared to another project of similar size are the one that have better
economic efficiencies. However, examination of project results through a single
outcome (univariate analysis) such as effort in hours can be deceptive. Quality
measures, effort and time elapsed should be analysed together.
We understand that the efficiency of a project should be analysed from a set of
technical and managerial results (multivariate analysis). In the design of the
experiment we used four variables to analyse software projects efficiency:
Proceedings of the 10
th
International Conference on DEA Brazil2012 33
Effort, defined as the total number of hours spent on the project.
Time Elapsed, defined as the number of months spent on the project.
Software functional size defined as function points as defined by IFPUG.
Defect density, defined by the ratio of the number of defects by the functional
size of the software. Defect density is a popular quality measure on IT projects.
See equation (2)
(2)
When we compare two projects, ceteris paribus, reduced effort, reduced time elapsed,
reduced defects and greater functional size suggests better efficiency.
The efficiency function in our experiment is not known, i.e., we make no assumption
about the nature of this function (non-parametric function). To compare diverse
software projects in a problem of multiple inputs and outputs we used DEA BCC
(Data Envelopment Analysis) method.
The experiment consisted in the comparative analysis of a set of software projects.
The database used was ISBSG Release Date v11, composed of 5052 software projects.
The data of the projects were selected from the following criteria:
Evaluation of data quality of a project is sound and has been formally evaluated
by experts from ISBSG. (Date Quality Rating = A and B).
Software project defects were reported.
Functional size counting technique respects Function Point technique as defined by
IFPUG (IFPUG Count = Approach) and function points project size in the range
(200, 2000).
Development of new products (Development Type = New development).
Maintenance and reconstruction projects were not evaluated in this work.
One hundred and five (105) projects met these criteria and were used for efficiency
comparison.
Our DEA model has two inputs (Effort and Elapsed Time) and two outputs
(Functional size and Defects). Since software defects is an undesirable output, we
modelled it using the reciprocal multiplicative approach [13]. In this approach, the
undesirable output is modeled as being desirable: f(uik)=1/uik, where uik is one of
the elements of the matrix U of the undesirable outputs i of the decision making unit
k.
In our work, we are interested in the comparison of a group of IT projects which used
a specific technique and a group of IT projects which didnt use this specific
technique. We use DEA-based hypothesis tests for comparing two groups of decision
making units, according with [14]. We use the modified T-Test procedure for
comparing the equality of means of two populations random variables.
Proceedings of the 10
th
International Conference on DEA Brazil2012 34
DEA results are considered robust if the number of DMUs compared is threefold
higher than the sum of inputs and outputs. Since we have 2 inputs and 2 outputs, our
database sample suffices.
Results and discussions
Table 01 shows, for each software project technique, the technical average efficiency
scores (mean and standard deviation values).
Software Project Technique
Technical
Efficiency Scores
Mean
Value
Standard
Deviation
CMMI 43.47 13.44
Business Process Management/BPM 89.19 18.83
Business Project Modeling 63.91 27.46
Project Management Techniques 46.76 18.99
Software Requirement Techniques 45.79 18.33
Software Design Techniques 47.04 16.96
Software Test Techniques 42.26 13.18
Table 02 shows the technical efficiency scores breakdown for firm type.
Firm type
Technical
Efficiency Scores
Mean
Value
Standard
Deviation
IT company 79.26 11.58
Banking company 71.42 20.20
Telecom company 47.18 3.29
Proceedings of the 10
th
International Conference on DEA Brazil2012 35
We first compared groups of software projects to check if a specific technique could
indicate superior efficiency technical performance.
A t test failed to reveal a statistically reliable difference between the mean number of
CMMI projects ( = 43.47, s = 13.44) and non-CMMI projects, t(3) = -3.379, p < .05,
= .05.
A t test succeed to reveal a statistically reliable difference between the mean number
of BPM projects ( = 89.19, s = 18.83) and non-BPM projects, t (3) = 3.15, p < .05, =
.05.
A t test failed to reveal a statistically reliable difference between the mean number of
projects with process modeling techniques ( =63.91 s = 27.46) and projects without,
t (16) = 0.66, p < .05, = .05.
A t test failed to reveal a statistically reliable difference between the mean number of
projects with formal project management techniques ( =46.76 s = 18.99) and projects
without, t (35) = -4.03, p < .05, = .05.
A t test failed to reveal a statistically reliable difference between the mean number of
projects with formal planning techniques ( =45.79 s = 18.33) and projects without,
t (33) = -4.37 p < .05, = .05.
A t test failed to reveal a statistically reliable difference between the mean number of
projects with formal software design techniques ( =47.04 s = 16.96) and projects
without, t (32) = -4.23 p < .05, = .05.
A t test failed to reveal a statistically reliable difference between the mean number of
projects with formal test techniques ( =42.26 s = 13.18) and projects without,
t (17) = -5.55 p < .05, = .05.
No single software engineering technique could deliver better results. BPM projects,
however, presented some evidences of superior performance.
In the analysis of firm types, Telecom and Banking companies failed to pass on the t-
test. IT companies, however, passed on the t-test. Superior technical performance of
IT firms projects can be partially explained by the economic drivers of this type of
organization. IT projects are in the core value chain of these organizations. IT projects
outcomes determine the success or failures of this type of organization and therefore
are continuously monitored and improved by managers and engineering teams.
Conclusions
Quantitative analysis of projects software can be better studied with multivariate
analysis methods such as DEA. Results emphasize the importance of recognizing that
optimal management techniques depend on the characteristics of the software
development project, organization type and its sociotechnical environment. IT
companies presented better results and non-IT companies, which can show how the
socio-technical can influence the technical performance of projects.
Proceedings of the 10
th
International Conference on DEA Brazil2012 36
References
M. J. Farrell, The Measurement of Productivity Efficiency. Journal of Royal Statistical
Society, 1957.
Standish Group, CHAOS Manifesto 2011, 2011.
E. Brynjolfsson, The productivity paradox of information technology,
CommunIcations of ACM, vol. 36, no. 12, pp. 6677, 1993.
W. T. Lin and B. B. M. Shao, The business value of information technology and
inputs substitution: the productivity paradox revisited, Decis. Support Syst., vol. 42,
no. 2, pp. 493507, Nov. 2006.
L. H. Putnam, A General Empirical Solution to the Macro Software Sizing and
Estimation Problem, IEEE Transactions on Software Engineering, pp. pp. 345361, Jul.
1978.
B. W. Boehm, Software Engineering Economics, 1st ed. Upper Saddle River, NJ, USA:
Prentice Hall PTR, 1981.
F. P. Brooks,Jr., The Mythical Man-Month: Essays on Softw, 1st ed. Boston, MA, USA:
Addison-Wesley Longman Publishing Co., Inc., 1978.
Kitchenham BA, The question of scale economies in software-why cannot researchers
agree?, Information and Software Technology, 2002. .
R. Banker and C. F. Kemerer, Scale economies in new software development, IEEE
Trans Software Eng, vol. 15, no. 10, pp. 11991205, 1989.
R. D. Banker, H. Chang, and C. F. Kemerer, Evidence on economies of scale in
software development, Inf Software Technol, vol. 36, no. 5, pp. 275282, 1994.
C. Comstock, Z. Jiang, and J. Davies, Economies and diseconomies of scale in
software development, Journal of Software Maintenance and Evolution: Research and
Practice, vol. 23, no. 8, pp. 533548, 2011.
K. Petersen, Measuring and predicting software productivity: A systematic map and
review, Inf. Softw. Technol., vol. 53, no. 4, pp. 317343, Apr. 2011.
[13] E. Gomes and M. Lins, Modelling undesirable outputs with zero sum gains
data envelopment analysis models, Journal of the Operational Research Society, pp.
616623, 2008.
[14] R. D. Banker, N. Natarajan, N, and Z. Zeng, DEA-based hypothesis tests for
comparing two groups of decision making units., European Journal of Operational
Research, 2010.
Acknowledgements
We authors would like to thank ISBSG for making available the ISBSG R11 data.
Proceedings of the 10
th
International Conference on DEA Brazil2012 37
5. An IDEA model to evaluate
the overall performance of
Buyer-Supplier
Zahra Yousefi
Department Mathematics, Islamic Azad University, Science and Research Branch, Tehran, Iran,
sarir.yousefi@yahoo.com
Mohsen Rostamy-Malkhalifeh
Department Mathematics, Islamic Azad University, Science and Research Branch, Tehran, Iran
mohsen_rostamy @yahoo.com
Somayeh Mamizadeh
Department Mathematics, Islamic Azad University, Central Tehran Branch, Tehran, Iran
somayeh_mamizadeh@yahoo.com
Abstract
After proceeding with international management, enterprises have to face the
challenge of Buyer-Supplier mainly because of the paid change in the business
environment and severe competition I market and customers diverse demand. In some
enterprise we may come across imprecise data such as interval, ordinal and fuzzy
data. And also for some Buyer-Suppliers, there are indexes that its decrease and or
increase are not possible. These indexes calls non-controllable or somewhat
controllable. We consider evaluating of Buyer-Supplier so that parts of inputs are non-
controllable variable. We focus on the case when imprecise input-output data are
represented by intervals.
Keywords: Imprecise Data Envelopment Analysis, Performance evaluation, Buyer-
Supplier
Introduction
The organization is customer-oriented organization, which aims to anticipate customer
demand in the issues that are most valuable to them. Customer service organization
are always the cornerstone of thinking and planning organizations. In the world
today that quality is driven customer-oriented, customer focus is the foundation of all
commercial activities. The definitive impact of the supply chain on enterprises
performance has been reported from many industries. A supply chain consists of
organizations working in the entire supply chain. The supply chain performance
evaluation problem is one of the most comprehensive strategic decision problems that
need to be considered for long-term efficient operation of the whole supply chain [2].
Our approach is evaluating of the overall performance in Buyer-Supplier relationships
through the measurement of intensity and effectiveness in a supply chain: we use
evaluation tools such as Imprecise Data Envelopment Analysis (IDEA). In the
Proceedings of the 10
th
International Conference on DEA Brazil2012 38
conventional DEA model, input and output are assumed to be exact. In recent years,
various applications of DEA, there are many input and output values is unlimited.
Imprecise data can be probabilistic, interval, ordinal, qualitative, or fuzzy. Therefore,
some papers were presented on the theoretical development of this technique with
interval data, of which we can name Inuiguchi (2011) [1]. Thus, we presented an IDEA
model with intervals are shown and we consider evaluating of Buyer-Supplier so that
parts of inputs are non-controllable variable. In addition, by compared with other
Buyer-Suppliers, the evaluated Buyer-Supplier can be identified as efficient or
inefficient. Application of classical DEA regards the supply chain as a black box and
considers only the inputs from the beginning of the upstream members and final
outputs at the very end of downstream members in the performance evaluation. Thus,
those intermediate product are ignored. Therefore, the performance scores obtainable
from the original approaches will overrate real performance of supply chain. In this
paper we examine the supply chain to the black box.
Methods
Assume that there are n supply chains (SCs) each of them producing S outputs by
consuming M inputs so that D represented controllable variable. The correspondences
between Q
1
and (Q
11
,Q
12
,Q
13
,Q
14
) and between Q
2
and (Q
21
,Q
22
,Q
23
,Q
24
) are shown in
Table 1.
Figure1: A supply chain
Therefore, Buyer-Supplier supply chain performance assessment model using the
following model:
Buyer-Supplier
X Y
Proceedings of the 10
th
International Conference on DEA Brazil2012 39
Q
1
Q
11
Q
12
Q
13
Q
14
L R R L
N R L L R
L L L R R
R R R L L
Q
1
Q
21
Q
22
Q
23
Q
24
R L L R
N L R R L
L L L R R
R R R L L
Table 1: The correspondence between Q
i
and (Q
i1
, Q
i2
, Q
i3
, Q
i4
), i=1,2
1
11 12
11 12
13 14
1
s.t.
if Q
X i D
X i D
Y r 1,...,
if Q
X , X i D
X , X
Q Q
i iq
Q Q
i iq
Q Q
r rq
L L R R
i iq i iq
L L R
i iq i
Min
LR
x
x
y s
LR
x x
x x
=
=
2
1 1
21 22
1 1
21 22
1 1
23 24
i D
Y , Y r 1,...,
if Q /
X z z i D
X z z i D
Y z z
R
iq
L L R R
r rq r rq
Q Q
i i iq i
Q Q
i i iq i
Q Q
r r rq r
y y s
L R
x
x
y
+ +
2
1 1 2 2
1 1 2 2
2 2 1 1
r 1,...,
if Q /
X z z , X z z i D
X z z , X z z i D
Y z z , Y z z
L L R R
i i iq i i i iq i
L L R R
i i iq i i i iq i
L L R R
r r rq r r r rq r
s
L R
x x
x x
y y
+ + + +
=
=
{ }
1 2 1 2
r 1,...,
0 , e 1 , 0
z , z , z , z 0,1.
T
q
i i r r
s
+ +
=
= =
Proceedings of the 10
th
International Conference on DEA Brazil2012 40
Results and discussions
The proposed approach, the Buyer-Supplier performance is evaluated both
qualitatively and quantitatively.
Conclusions
Today, enterprises have found that they can buy them increasingly effective in
increasing the efficiency and effectiveness, and therefore buying practices have
changed and try to choose an suitable manner so that they can meet its strategic
objectives and purchasing. To accomplish of this subject we have to seek suitable
suppliers and strategic and related with them to attain competitive advantages. To
achieve this goal, the implementation of the supply chain is necessary.
References
Inuiguchi M., Mozioshita F., (2011) Qualitative and quantitative data envelopment
analysis with interval data, Annals of operation research, DOI 10.1007/s10479-011-
0988-y.
Yang F., Wu D., Liang L., Bi G.,& Wu D.D., (2009) Supply chain DEA: Possibility set
and performance evaluation model, Annals of operation research, DOI
10.1007/s10479-008-0511-2.
Proceedings of the 10
th
International Conference on DEA Brazil2012 41
6. Assessing Performance of
Organized Pharmacy Retail
Stores using Data
Envelopment Analysis
G N Patel
gn.paptel@bimtech.ac.in
Smiti Pande
smriti.pande@bimtech.ac.in
Abstract
The purpose of the study is to evaluate the performance of a chain of organized
pharmacy retail stores using a new enhanced technique, DEA. The study focuses on
evaluation of technical and cost efficiency. Technical efficiency does not take into
account the substitution possibilities between inputs and therefore we applied cost
and allocative efficiency models. To illustrate the DEA models discussed above we are
focusing upon a real-world problem of revival of loss making retail stores of an
organization which deals in pharmacy retailing. This study uses secondary data
collected over a time period of three years from a pharmacy retail chain located in
National Capital Region. This organization has expanded itself into a chain of 46
pharmacy retail stores over a time span of three years. The study is deemed to be
helpful to the retail managers in providing a framework for performance evaluation
and enabling the pharmacy retail stores in gaining a competitive edge over the
increasing competition faced by the emerging organized pharmacy retail market in
India.
Keywords: Pharmacy, Retailing, Performance Evaluation, Data Envelopment Analysis
Introduction
In the current Indian pharmacy retailing scenario, the sector is mainly dominated by
unorganised players, which comprises of, the neighbourhood chemist stores owned
by small families. The rising affordability, increased consciousness and willingness to
spend has not only given rise to the health and pharmacy retailing business of the
society but has also demanded for a big transition. Therefore, the challenge faced by
the organised sector is to position itself in such a way that they can be easily
differentiated from the small and unorganised retailers of pharmacy sector. Currently
there is not much differentiation in the offerings of organised players but at the same
time, organised players are a big threat for dominating unorganised sector (Pande and
Patel, 2011). Technology and cost are the two wheels that drive a business and
therefore play a crucial role in running an efficient business. A considerable amount of
attention received by the retail efficiency research is understandable but the scarcity of
Proceedings of the 10
th
International Conference on DEA Brazil2012 42
research in the area of cost efficiency evaluation is a matter of apprehension. This is
because companies are always eager to enquire, how effectively and efficiently their
resources that incur huge costs are being utilized (Pande and Patel, 2011). As per the
authors knowledge, the contribution of this paper to retailing research is based on the
application of allocation models in order to show how DEA can be used to identify
not only the technical efficiency but also the allocative efficiency where, just
reallocation of inputs can improve the overall performance.
Theoretical Framework
According to Farrell (1957), cost efficiency gets decomposed into technical and
allocative efficiency. The cost minimization problem and its decomposition can be
diagrammatically represented as-
Figure 1: Cost efficiency and its decomposition
Now, to evaluate the performance of A we can use Farrell measure of technical
efficiency * (Refer Cooper et. al., 2007 for details) which is represented in the ratio
form, in the following manner-
1
) , (
) , (
0
A O d
B O d
(1.1)
The optimal point C is obtained from the following LP (1.2) (Refer Ray, 2004 for
details)
j j
n
j
j
n
j
r j rj
n
j
i j ij
m
i
i i
U L y y x x t s x w
= = = =
0 , , , . . , min
1 1
0
1 1
Proceedings of the 10
th
International Conference on DEA Brazil2012 43
where, m i w
i
.... 2 , 1 , = is the unit price for
m
x x x .... ,
2 1
respectively. Based on the
optimal solution (
* *
, x ) derived from the above linear programming the cost
efficiency of a DMU can be calculated. Allocative efficiency () is represented by
determining the relative distance of B and D to obtain the following ratio-
1
) , (
) , (
0
B O d
D O d
(1.3)
Furthermore, another measure that is referred as cost efficiency () is given as-
1
) , (
) , (
0
0
*
=
wx
wx
A O d
D O d
(1.4)
This is a measure of extent to which the originally observed values at A have fallen
short of achieving the minimum cost. To put this in a way that relates all three
efficiency concepts to each other, we have - * =
(1.5)
Data and Variables
Data Envelopment Analysis requires identification of input and output variables. For
the purpose of this study, input and output variables are identified based on the
objectives of the company and also the literature reviewed.
Table 1: Inputs and Outputs
Inputs Authors
Rent paid Joo et al. 2009
Store Size Moreno, 2008; Barros and Alves, 2003; Donthu and Yoo,
1998
Wages Moreno 2010; Joo et al. 2009
Inventory Cost Donthu and Yoo, 1998; Barros and Alves, 2003
Marketing expenses Donthu and Yoo, 1998
Maintenance expenses Moreno 2006
Other day-to-day expenses Joo et al. 2009; Barros and Alves 2003
Outputs Authors
Sales Joo et al. 2009; Barros 2006; Moreno 2006; Seller-Rubino &
Mas-Ruiz 2006; Barros and Alves 2003; Donthu and Yoo
1998
Footfalls
Here, the relative unit costs of store for each DMU were also recorded in the
following manner: Per unit cost of store size = Total rent / Store size. The other input
variables such as maintenance, marketing and other day-to-day expenses are clubbed
Proceedings of the 10
th
International Conference on DEA Brazil2012 44
under a single heading named operating expenses. The relative unit cost for
operating expenses, inventory cost and wages to the employees remains equals to 1
for each DMU.
Result and Discussion
Firstly, we applied the BCC model (Cooper et. al. 2007). This model includes two
outputs: sales and footfalls and four inputs: Wages of the employees, store size,
inventory cost and operating expenses. Secondly, we applied the linear programming
equation (1.2) to evaluate the cost efficiency of each DMU. This model records the
relative unit costs of store size for each DMU, apart from the inputs and outputs
considered in the previous model. Third, the study also evaluates the allocative
efficiency of each DMU by using the equation (1.5). The scores of three different
efficiencies obtained for each DMU for three different years are as shown in Table 2
(refer Appendix A). From the results obtained it can be observed that, the DMUs that
turn out to be the most excellent or best performers are DMU 1, 3, 6, 19, 30 and 43
with all their efficiency scores equal to 1 since their inception. Apart from these best
performers there are some DMUs with their technical efficiency equal to 1, but, these
DMUs do not have the best cost based measures. For example: DMU 2, 5 in the year
2009. There are some that are neither technically efficient nor posses a good cost
based measure, like: DMU 4, 7 in 2009. These set of DMUs also includes the ones that
have worst allocative efficiency scores for example DMU 13, 14 and 15 in all the three
years. All such cases identifies the need of reallocation of resources. Lastly, there are
few that are neither technically efficient nor have efficient cost based measures but
have attained high allocative efficiency, for example: DMU 4 and 29 in 2010.
\This case is a part of a thesis wherein in order to clearly examine the determinants of
efficiency, we applied Tobit regression model regressing the BCC efficiency scores as
dependent variable. As per DEA literature Coelli (1998), Tobit regression model is
suitable when the dependent variable is censored. The Tobit regression model is
represented as-
i i
i i i i i i i
Location
Age size Store enses Operating Footfalls Sales
+ +
+ + + + + =
) (
) ( ) ( ) exp ( ) ( ) (
6
5 4 3 2 1
Where,
i
is the efficiency score for the retail store i computed from the BCC model
with categorization of age and location as non-discretionary variables.
i i i i
size Store enses Operating Footfalls Sales , exp , , are sales, footfalls, operating
expenses and store size of the
th
i retail store. The non-discretionary variables
i i
Location and Age are part of the thesis work and are not considered in this cost
efficiency section.
The
2
test statistics (=161.7) with six degrees of freedom associated with p value
(=2.5844x10
-32
) shows that the model is a good fit for the data. Also we find that the
value of constant 2 (e
-2.49479751
= 0.082513157) from the Tobit model is much less that the
standard deviation of
i
(=0.1676) which again shows that the models appears to fit
Proceedings of the 10
th
International Conference on DEA Brazil2012 45
the data well. From the results obtained using Tobit model we find that, footfalls,
sales, operating expenses, store size are significant contributors to the efficiency of
retail store. For an increase in footfalls and sales the efficiency of the retail store will
increase by 0.00164001 and 0.00105268 respectively. However, the sign of operating
expenses and store size is negative as expected. This indicates that the efficiency of a
retail store will fall by (-0.00465915) and (-0.00094193) for an increase in operating
expenses and store size respectively. The age and location of a store were not found
to be significant contributors to the efficiency.
Managerial Implications and Conclusions
There is a likelihood of huge reductions in the inputs and also, reallocation of inputs
as discussed above in various cases. Second, we observe that the older the store, the
higher is the cost efficiency. For majority of the stores, the analysis shows huge
reduction and reallocation possibilities to attain the optimal input mix. Third, we can
state that allocative models better describes the performance of chain of pharmacy
retail stores because it brings number of inputs, costs and right mix of inputs
altogether. If all the three considerations are well taken care of by retail managers, it
may lead to enhancement of the overall performance.
References:
Banker, R. D., Charnes, A. and Cooper, W.W. (1984) Some Models for Estimating
Technical and Scale Inefficiencies in Data Envelopment Analysis, Management
Science, No. 30, pp.1078-1092.
Barros, C.P. and Alves, C.A. (2003) Hypermarket retail store efficiency in Portugal,
International Journal of Retail & Distribution Management, Vol. 31, No. 11, pp.549-560.
Barros, C.P. (2006) Efficiency measurement among hypermarkets and supermarkets
and the identification of the efficiency drivers, International Journal of Retail &
Distribution Management, Vol. 34, No. 2, pp. 135-154.
Charnes, A., Cooper, W.W. and Rhodes, E. (1978), "Measuring the Efficiency of
Decision Making Units, European Journal of Operational Research, Vol. 2, 1978,
pp.429-444.
Coelli, T.J., Rao, P. And Battese, G.E. (1998), An Introduction to Efficiency and
Productivity Analysis, Kluwer Academic Press, Dordrecht
Cooper, W.W., Seiford, L.M. and Tone, Kaoru (2007) Data Envelopment Analysis: A
comprehensive text with models, applications, references and DEA-Solver Software,
Springer Science + Business Media, LLC.
Donthu, Naveen and Yoo, Boonghee (1998) Retail Productivity Assessment Using
Data Envelopment Analysis, Journal of Retailing, Vol. 74(1), pp. 89-105
Farrell, M.J. (1957) The Measurement of Production Efficiency, Journal of the Royal
Proceedings of the 10
th
International Conference on DEA Brazil2012 46
Statistical Society A, 120, pp.253-281
Joo, Seong-Jong, Stoeberl, P.A. and Fitzer, Kristin (2009) Measuring and
benchmarking the performance of coffee stores for retail operations, Benchmarking:
An International Journal, Vol. 16, No. 6, pp. 741-753
Moreno, Justo de Jorge (2006) Regional regulation analysis of performance in Spanish
retailing, International Journal of Retail & Distribution Management, Vol. 34, No. 10,
pp. 773-793
Moreno, Justo de Jorge (2008) Efficiency and regulation in Spanish hypermarket retail
trade: A cross-section approach, International Journal of Retail & Distribution
Management, Vol. 36, No. 1, pp.71-88
Moreno, Justo de Jorge (2010) Productivity growth of European retailers: a
benchmarking approach, Journal of Economic Studies, Vol. 37, No. 3, pp.288-313
Pande, Smriti and Patel, G.N. (2011) Assessment of Performance using DEA: A Case
on Chain of Pharmacy Retail Stores located in NCR, in Sardana, G.D. and
Thatchenkery, Tojo (Eds.), Building Competencies for Sustainability in Organizational
Excellence, Macmillan publishers India Ltd., pp. 319-336
Ray, Subhash C. (2004) Data Envelopment Analysis Theory and Techniques for
Economics and Operations Research, Cambridge University Press, New York, USA
Sellers-Rubio, R. and Mas-Ruiz, F. (2006) Economic efficiency in supermarkets:
evidences in Spain, International Journal of Retail & Distribution Management, Vol.
34, No. 2, pp. 155-171
Sellers-Rubio, R. and Mas-Ruiz, F. (2007) An empirical analysis of productivity growth
in retail services: evidence from Spain, International Journal of Service Industry
Management, Vol. 18, No. 1, pp. 52-69
Proceedings of the 10
th
International Conference on DEA Brazil2012 47
Appendix A Table 2: Efficiencies obtained
2009 2010 2011
S.No.
BCC Eff. Cost Eff.
Allocative
Eff.
BCC Eff. Cost Eff.
Allocative
Eff.
BCC Eff. Cost Eff.
Allocative
Eff.
1 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
2 1.0000 0.8323 0.8323 1.0000 0.7856 0.7856 1.0000 0.7847 0.7847
3 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
4 0.9146 0.7349 0.8036 0.7284 0.6197 0.8508 0.7022 0.6251 0.8902
5 1.0000 0.6619 0.6619 0.9290 0.5567 0.5993 0.8824 0.5562 0.6304
6 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
7 0.9706 0.9027 0.9300 0.9412 0.7713 0.8195 0.9107 0.7522 0.8260
8 0.9587 0.7332 0.7648 0.9963 0.6796 0.6822 1.0000 0.6691 0.6691
9 0.9428 0.6686 0.7092 0.9247 0.6043 0.6536 0.9170 0.5883 0.6415
10 0.9657 0.8743 0.9054 0.9175 0.8566 0.9336 0.9210 0.7378 0.8010
11 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 0.9042 0.9042
12 0.8695 0.6373 0.7329 0.8815 0.6475 0.7346 0.8533 0.5364 0.6286
13 0.8349 0.4594 0.5502 0.7951 0.4026 0.5064 0.7579 0.4041 0.5331
14 0.8854 0.5706 0.6445 0.8498 0.5659 0.6660 0.8667 0.3905 0.4506
15 0.9348 0.4764 0.5097 0.8644 0.4268 0.4938 0.8088 0.3557 0.4399
16 1.0000 0.8645 0.8645 1.0000 0.7863 0.7863 1.0000 0.7603 0.7603
17 0.9012 0.7526 0.8352 0.8909 0.6790 0.7621 0.8962 0.6798 0.7585
18 1.0000 0.5625 0.5625 1.0000 0.5749 0.5749 1.0000 0.4587 0.4587
19 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
20 0.7347 0.3187 0.4337 0.7345 0.3134 0.4267
21 1.0000 0.8099 0.8099 1.0000 0.8340 0.8340
22 1.0000 0.6796 0.6796 1.0000 0.5752 0.5752
23 0.8429 0.5150 0.6110 0.8582 0.5297 0.6172
24 0.8199 0.6512 0.7943 0.7461 0.4975 0.6668
25 0.8046 0.6451 0.8018 0.7906 0.6415 0.8115
26 1.0000 0.9421 0.9420 0.9249 0.7646 0.8267
27 0.7690 0.4849 0.6306 0.7592 0.4847 0.6384
28 1.0000 0.6197 0.6197 0.9740 0.4822 0.4951
29 0.7775 0.7552 0.9713 0.7607 0.7438 0.9777
30 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
31 1.0000 0.5752 0.5752 1.0000 0.5655 0.5655
32 1.0000 0.8634 0.8634 1.0000 0.8817 0.8817
33 1.0000 0.4477 0.4477 1.0000 0.4301 0.4301
34 1.0000 0.6035 0.6035 1.0000 0.6068 0.6068
35 0.8492 0.7987 0.9406 0.8156 0.8115 0.9949
36 0.7357 0.5160 0.7013 0.5891 0.5102 0.8662
37 0.9418 0.2850 0.3026
38 0.8774 0.7284 0.8301
39 0.9169 0.8111 0.8846
40 1.0000 0.5684 0.5684
41 1.0000 0.6334 0.6334
42 0.9908 0.8003 0.8077
43 1.0000 1.0000 1.0000
44 0.9906 0.6208 0.6267
45 0.8861 0.7307 0.8247
46 0.8948 0.8287 0.9262
Proceedings of the 10
th
International Conference on DEA Brazil2012 48
7. Behavioral effects of DEA on
performance assessment
Heinz Ahn
Technische Universitt Braunschweig, Germany, hw.ahn@tu-bs.de
Nadia Vazquez Novoa
Technische Universitt Braunschweig, Germany, n.vazquez-novoa@tu-bs.de
Abstract
This paper examines the Data Envelopment Analysis (DEA) from a cognitive
perspective. Experimental evidence regarding the role of DEA efficiency scores as an
overall non-financial performance measure in a context with multiple alternatives and
attributes is outlined. The study not only confirms that the efficiency score acts as a
strong performance marker when deciding on which decision making units (DMUs)
should be awarded for their non-financial performance, but also shows that the score
may induce a halo effect, significantly influencing a posterior financial assessment.
These results have practical consequences for planning, reporting, and controlling
processes that incorporate DEA efficiency scores.
Keywords: DEA, performance assessment, performance markers, halo effect,
experimental study.
Introduction
DEA is a promising approach to performance evaluation that allows the simultaneous
analysis of financial and non-financial performance indicators. Nevertheless, this
instrument presents some shortcomings. Imminent pitfalls have been extensively
discussed from a prescriptive point of view. Dyson et al. (2001), e.g., systematically
outline conceptual problems of DEA and describe possibilities to cope with them. In
contrast, although the specification of cognitive processes is important to theory
development (Peters, 1993, p. 391), we have found no contributions discussing DEA
applications from the descriptive point of view. This is astonishing, since a DEA-based
evaluation also includes subjective components, implying that it is susceptible to
behavioral influences and limitations. In fact, a (DEA-based) performance assessment
aiming at identifying the best performing DMUs can be seen as a multiple attribute
choice with multiple alternatives, and as such, it can be analyzed from the perspective
of psychological theories.
In this study, the non-financial and the financial performance assessment tasks are
considered as two choice problems. In the first one, DEA scores serve to measure the
overall non-financial performance of a set of DMUs. Based on heuristics and bias
literature, we hypothesize that the relatively high accessibility of DEA scores will make
them act as performance markers, thus (over-)simplifying the choice problem by
Proceedings of the 10
th
International Conference on DEA Brazil2012 49
focusing the analysis on a single cue (Shah and Oppenheimer, 2008; Cardinaels and
van Veen-Dirks, 2010).
Literature on contingent decision making indicates that information used for a choice
task has an effect on posterior choices by influencing the attributes used and their
weights (Dholakia et al., 2005). Additionally, research on decision making has also
shown that (irrelevant) previous information is commonly used when determining a
searched value, thus producing an anchoring bias. A special case of this bias is the
halo effect (Huber, et al., 1987). It refers to the influence that the information about
one (often irrelevant) trait, i.e., a characteristic of a person or an object, has on the
perception of another trait. In this context, the non-financial performance assessment
is expected to act as a halo for the posterior financial performance assessment,
improving (worsening) the financial performance assessment of those DMUs with a
perceived relative high (low) non-financial performance (Tversky and Kahneman,
1974; Huber et al., 1987). We therefore hypothesize that the presence of high DEA
scores will indirectly affect the posterior financial performance assessment through the
non-financial performance assessment.
In the following, we depict the methodological aspects of the experiment conducted
as well as its results. A discussion of the findings concludes the paper.
Method
Bachelor students (N = 72) taking introductory Management Control and Business
Accounting courses at a German university were asked to answer the questionnaire
during a lecture. Students were randomly assigned to one of two conditions (DEA
score, N = 37; no DEA scores, N = 35). Seven cases were eliminated from the analysis
since no valid answers were provided for the dependent variables defined in the
general research model.
The experiment comprised two phases. The first had the aim of evaluating the role of
DEA scores as performance markers on the non-financial performance assessment.
The second phase was designed to investigate possible effects of DEA scores on a
posterior financial performance assessment.
Participants received a table with non-financial data corresponding to 10 different
DMUs and were asked to decide to which three DMUs they would assign a bonus for
non-financial performance (Bnf). The non-financial data included in the report were
based on a real case of a European pharmacy chain and included the following
performance criteria: average number of employees, sales area in m, sales
transactions, and number of customer advices. For those participants in the treatment
with DEA scores (), this measure was also included. The treatment containing DEA
scores permitted to identify three DMUs as efficient ( = 100%), one as almost efficient
( = 98%) and all others as inefficient, with = 54% being the lowest efficiency score.
A brief description regarding the DEA methodology was included in the
corresponding vignette.
In the second phase, participants were provided with a table containing three different
financial indicators for the 10 DMUs: profit, cash flow in thousand euro, and return on
capital. The financial performance indicators were designed to avoid not only a clear
Proceedings of the 10
th
International Conference on DEA Brazil2012 50
domination of one DMU over its peers but also a significant correlation among DEA
scores and the financial measures. Before executing any kind of analytical
examination, participants were required to decide whether to assign a bonus for
financial performance (B
f
) to A. This question had the purpose of concentrating the
attention of respondents on the performance of DMU A. In case an anchoring effect
would occur, augmenting the accessibility of A should reinforce this effect.
The financial performance assessment was measured by means of two main variables.
The first question required participants to decide to which three subsidiaries they
would assign a B
f
. The answer was considered to have nominal scale since
respondents were not asked to elaborate a ranking. The second relevant question
required participants to estimate the financial performance of A in an interval between
0% and 100%. Since this estimation was made only for DMU A, the general research
model was evaluated exclusively for this case (i.e., a DMU with a high non-financial
performance that does not clearly dominate other DMUs and with a relatively good
financial performance).
Previous research has shown that a sequential presentation of alternatives affects the
way they are evaluated. Consequently, the data corresponding to each DMU was
randomized to avoid undesired biases related to the presentation order.
Results
We used a path analysis to prove the general research model for DMU A. The
correlation matrix for the four variables included in the causal model together with the
main statistics for each variable are presented in Table 1.
As expected, the presence of DEA scores is significantly correlated with the
assignment of B
nf
to A, and the assignment of a B
f
to A is significantly correlated with
the perceived score for financial performance for this DMU. It is interesting that the
assignment of a B
nf
to A is also significantly correlated both with the posterior
assignment of a B
f
to this DMU and its perceived financial performance score. This
provides initial support for our hypothesis.
When developing the hypothesis it was predicted that DEA scores would serve as
performance markers and therefore would have an influence on the decision of Bnf
assignment. The path analysis for the case of DMU A (Fig.1) indicates that the
presence of a DEA score of 100% has a positive influence on the Bnf assignment. The
direct effect is positive and significant (a = 0.329, z = 2.822, p = 0.005). The contrary
occurs for the case of DMU D, whose DEA score was only 98% (a = 0.419, z =
3.742, p = 0.000), therefore providing confirmatory evidence for our research
hypothesis.
Following the criterion of considering an indirect effect to be significant if all the paths
involved in its calculation are significant (Kline, 2011), it can be concluded that DEA
scores have an effect on the posterior financial performance assessment. The presence
of DEA scores has an indirect effect on the assignment of Bf to A (ab) as well as on
the perceived financial performance score (abd). Similar results are obtained for
DMU D (ab significant, with b = 0.265, z = 2.292, p = 0.022) Table 2 presents the
coefficients for each of the paths included in the causal model for DMU A.
Proceedings of the 10
th
International Conference on DEA Brazil2012 51
Table 1: Correlation matrix for DMU A and main statistics.
DEA B
nf
to A B
f
to A Score A
DEA 1
B
nf
to A 0.324** 1
B
f
to A 0.109 0.244* 1
Score A 0.169 0.303* 0.401** 1
Mean 0.492 0.430 0.540 70.26
SD 0.504 0.499 0.502 19.38
* Significant at 0.05 level (two-tailed).
** Significant at 0.01 level (two-tailed).
The proposed model for DMU A fits with the data. All but one path coefficients were
significant and several measures of goodness of fit gave support to the model
(CFI = 1.000, (2,65) = 0.411, p = 0.814, and SRMS = 0.034). However, other measures
of fit indicate that the model should still be improved (RMSEA = 0.000 with
confidential interval (0.000 0.150), TLI = 1.270). To augment the model fit, different
modifications could be conducted adding parameters that were not considered in this
experiment. Incorporating other variables to the model could help to improve the
proportion of explained variance for each endogenous variable (RBnf = 0.108, RBf =
0.057, RScore = 0.211).
b = 0.239*
c = 0.206
Assignment of
B
f
to A
Perceived
financial
efficiency score
of A
Assignment of
B
nf
to A
Presence of
DEA scores
a = 0.329**
d = 0.365***
Performance
marker
Halo
effect
Halo
effect
* Significant at 0.05 level (one-tailed).
** Significant at 0.01 level (one-tailed).
*** Significant at 0.001 level (one-tailed).
FI G. 1: PATH COEFFI CI ENTS FOR DMU A (N=65, SOLI D LI NES REPRESENT
S G CA A S 0 21)
Proceedings of the 10
th
International Conference on DEA Brazil2012 52
Conclusions
The present study offers an analysis of the DEA approach focusing on its decisional
consequences. To the best of our knowledge, it constitutes the first attempt to deal
with the DEA approach from a behavioral operations perspective. The results of the
experiment indicate that the presence of DEA scores influences not only the non-
financial performance assessment of the DMUs, but also the posterior financial one.
As expected, DEA scores calculated to measure the overall non-financial performance
of a set of DMUs were found to act as performance markers. Consistent with previous
research on lexicographic heuristics, decision-makers seem to concentrate their
attention on the DEA scores as a manner to reduce the cognitive load and to save
time.
A managerial limitation associated to DEA scores operating as performance markers
arises from a methodological drawback of the basic DEA models: the weights of the
performance criteria are endogenously determined, allowing zero-value weights. This
may induce managers to concentrate their efforts only on some criteria when
attempting to raise the efficiency score of their DMU. A bonus assignment solely based
on the DEA scores may encourage this behavior, leading to a biased resource
allocation. At the same time, it may decrease the motivation of managers running
DMUs that are achieving all relevant performance criteria on a decent level. These
undesirable effects emphasize the importance of the ongoing research on how to cope
with the zero-value weights problem.
As the second main result of our study, it could be shown that the inclusion of DEA
scores to measure the overall non-financial performance may affect the posterior
financial assessment. For the case of DMUs with a general good performance level,
this effect is mediated by the decision of awarding the respective DMU for its non-
financial performance. A DEA score of 100% contributed to increment the proportion
of Bnf awarded to the DMU which in turn affected its financial assessment: more Bf
were assigned to this DMU and its estimated financial performance score was higher.
For cases of good performers with a DEA score lower than 100% such as D
(D = 98%), the presence of the DEA score led to a lower Bnf assignment that
negatively influenced the posterior financial assessment.
The unintended consequences of the presence of DEA scores can be attributed to a
halo effect. A DMU being awarded for its non-financial performance (positive trait)
will be perceived as attaining better financial results. Thereby, the B
nf
assigned to the
DMU serves as a self-generated anchor that acts as a starting point when assessing
the financial performance.
These findings strongly suggest that including DEA scores in the performance report
for facilitating decision-making may result in biased decisions. Since the DEA
approach is gaining relevance for everyday management control, it results essential to
investigate how to deal with this difficulty. To this respect, theories of mental
protection provide some insights (Wilson et al., 2002). On the one side, instruments
that prevent contaminating stimulus to enter managers minds could be developed. On
the other side, providing adequate training regarding cognitive biases could at least
help to diminish the undesirable effects.
Proceedings of the 10
th
International Conference on DEA Brazil2012 53
The halo effect is also relevant from a decision influencing perspective. Incrementing
the recognition for financial performance to DMUs of superior non-financial
achievement level and reducing such recognition to other DMUs may be perceived as
unjust. As a consequence, problems related to organizational commitment, job
satisfaction, performance, trust, etc. may arise.
The results were obtained from a sample of 65 bachelor students of a German
university. Even if the practical choice of resorting to students as surrogates for
managers has been supported by several authors (Moore, 2005), the adequacy of
undergraduate students as substitute for managers has been questioned. One of the
main critics suggests that undergraduate students may differ from experts on a number
of psychological and behavioral dimensions. However, this argument could be
relativized, since it can be assumed that not many managers have a more
comprehensive understanding of the DEA approach. Nevertheless, the current work
should be replicated using different samples in order to achieve a higher validity of
the results.
Further research analyzing DEA from a behavioral perspective is necessary if this
instrument is to be satisfactorily applied in the business context. Besides the cognitive
effects associated to the interpretation of the results, the influence of the DEA
approach on fairness perception should also be considered. A thorough investigation
of the behavioral aspects will enrich the analysis and provide insights into necessary
modeling improvements.
References
Cardinaels E., P.M.G. van Veen-Dirks (2010) Financial versus non-financial
information: the impact of information organization and presentation in a balanced
scorecard, Accounting, Organizations and Society 35 (6): 565578.
Dholakia U.M., M. Gopinath, R.P. Bagozzi (2005) The role of desires in sequential
impulsive choices, Organizational Behavior and Human Decision Processes 98 (2):
179194.
Dyson R.G., R.S. Allen, A.S. Camanho, V.V. Podinovski, C.S. Sarrico, E.A. Shale (2001)
Pitfalls and protocols in DEA, European Journal of Operational Research 132 (2): 245
259.
Huber V.L., M.A. Neale, G.B. Northcraft (1987) Judgment by heuristics: effects of ratee
and rater characteristics and performance standards on performance-related
judgments, Organizational Behavior and Human Decision Processes 40 (2): 149169.
Kline R.B. (2011) Principles and practice of structural equation modeling, 3rd edition,
The Guilford Press: New York.
Moore D.A. (2005) Commentary: conflicts of interest in Accounting. In: D.A. Moore,
D.M. Cain, G. Loewenstein, M.H. Bazerman (Eds.), Conflicts of interest. Challenges and
solutions in Business, Law, Medicine, and Public Policy. Cambridge University Press:
Cambridge et al., 7073.
Proceedings of the 10
th
International Conference on DEA Brazil2012 54
Peters J.M. (1993) Decision making, cognitive science and Accounting: an overview of
the intersection, Accounting, Organizations and Society 18 (5): 383405.
Shah A.K., D.M. Oppenheimer (2008) Heuristics made easy: an effort-reduction
framework, Psychological Bulletin 134 (2): 207222.
Tversky A., D. Kahneman (1974) Judgment under uncertainty: heuristics and biases,
Science, New Series 185 (4157): 11241131.
Wilson T.D., D.B. Centerbar, N. Brekke (2002) Mental contamination and the
debiasing problem. In T. Gilovich, D. Griffin, D. Kahneman (Eds.), Heuristics and
biases. The psychology of intuitive judgment, Cambridge University Press: Cambridge et
al., 185200.
Table 2: Effects of the presence of DEA scores on the performance assessment
of DMU A.
Path
Standardized
coefficient
Coefficient
(original units)
Standard
error
z-stat. Sign.
Direct effects
DEA B
nf
a 0.329** 0.325 0.115 2.822 0.005
B
nf
B
f
b 0.239* 0.241 0.119 2.024 0.043
B
nf
Score c 0.206 8.259 4.679 1.765 0.077
B
f
Score d 0.365*** 14.527 4.272 3.400 0.001
Indirect effects
DEA B
f
a
b
0.079* 0.078
DEA Score
a
c
0.068 2.684
a
d
0.029* 1.138
* Significant at 0.05 level (one-tailed).
** Significant at 0.01 level (one-tailed).
*** Significant at 0.001 level (one-tailed).
Proceedings of the 10
th
International Conference on DEA Brazil2012 55
8. Data Envelopment Analysis of
the effieincy frontier for the
results achived by Formula 1
drivers and teams
Prof. Dr. Aparecido Jorge Jubran
UNINOVE, jubran@uninove.br
Profa. Msc. Laura Martinson Provasi Jubran
UNINOVE, laura.jubran@uninove.br
Jos Rubens Moura Martins
UNINOVE, jrubens.martins@uninove.edu.br
Jane Leite Silva
UNINOVE, janelleitesilva@hotmail.com
Abstract
The current Formula 1 driver and team ranking system does not allow for an impartial
and unbiased comparison between results, since the criteria used are oftentimes
inconsistent. In order to make such comparison, the mathematical optimization tool
DEA - Data Envelopment Analysis was used. As a result of this research, an plan was
elaborated to review the efficiency frontier for the results achieved by Formula 1
drivers and teams, and a new driver and team ranking list was created.
Key words: Data Envelopment Analysis, Formula 1, Ranking.
Introduction
The objective of this paper was to design a plan to analyze the efficiency frontier for
the results achieved by Formula One car drivers and teams. This topic is particularly
relevant to the sports community, since the existing comparative performance
measurement methods applied along decades of competitions have always lacked
indicators that apply to all sports categories.
The existing car racer and racing team ranking system does not allow for an impartial
and unbiased comparison, since the criteria used are oftentimes inconsistent.
This paper also aims to help shed a light on the reasons why Formula One car racers
and agents constantly criticize the changes in the ranking criteria set out by FIA -
Fdration Internationale de l'Automobile. By constantly introducing technological
innovations, Formula 1 is today the most audacious and technologically sophisticated
motorsport category in the world.
One of the problems that affect this auto racing category is its points scoring system,
which generates distortions when analyzing and ranking the best drivers.
Proceedings of the 10
th
International Conference on DEA Brazil2012 56
The year 2010 marks the beginning of a new scoring scheme up to 10th place in each
Grand Prix, as illustrated in Fig. 1.
Scoring system
Temp./Pos. 1 2 3 4 5 6 7 8 9 10
1950-1955 8 6 4 3 2
1956-1990 9 6 4 3 2 1
1991-2002 10 6 4 3 2 1
2003-2009 10 8 6 5 4 3 2 1
2010 25 18 15 12 10 8 6 4 2 1
Fig.1:Scoring system from 1950- 2010.
These issues bring a negative impact upon sponsors and manufacturers, as well as on
drivers and teams, which are greatly affected. The points scoring system adopted by
FIA has changed frequently along the years, making it impossible to objectively
compare and analyze the results of each race and each championship along the years.
Due to the criteria currently in use, the rankings prepared and released to the public
are inaccurate. The rankings measure performance by driver, nationality, constructor,
engine, and tire manufacturer.
An example of these distortions can be seen in Table 1 (in APPENDICES), showing
the top 10 in the 2010 championship, where the 3 and 4 positions would be reversed
only if the current scoring criteria valid in 2009 continued in 2010.
Methods
The mathematical optimization tool DEA - Data Envelopment Analysis was used to
prepare the reports. Data Envelopment is an analytical tool designed to identify the
best practices for use of resources, which, in our study, comprise those resources
available for Formula 1 teams.
According Emrouznejad (2005), Figure 2 shows a number of units P1, P2 ... P6, where
each unit consumes a resource, but produce different amounts of outputs y1 and y2.
Thus, for a given amount of input feature units that provide larger quantities of
outputs will be considered efficient.
Proceedings of the 10
th
International Conference on DEA Brazil2012 57
Fig. 2: Efficiency frontier.
Source: Emrouznejad (2005).
Through this research, an analysis plan was elaborated to review the efficiency
frontier for the results achieved by Formula 1 drivers and teams. It also allowed
putting together a new ranking covering all Formula 1 drivers and teams, as well as
other stakeholders, including engine, tires, and service suppliers.
The mathematical optimization tool DEA - Data Envelopment Analysis was used to
prepare the reports.
A bibliographical research was carried out to collect data on the results of all F1 races
ever beginning with the first race in 1950.
After the results of all F1 races were known, a study was conducted to analyze the
inputs and outputs for a review using the DEA method, with the following
components being assessed:
Decision Maker Units - DMUs: Teams and Drivers.
INPUT: Grid position at qualifying session.
OUTPUT: Position at the end of the race.
Results and discussions
After extensive testing, the following factors were chosen for assessment of F1 results.
Inputs: The driver's position at the start of the qualifying session, i.e., weight 1 for all
drivers, since it was assumed all drivers start the session under the same conditions.
Outputs: The driver's position at the end of the race, considering the arrival order
according to FIAs rules. The output weight for each driver's position was calculated
based on the number of cars participating in each race.
Proceedings of the 10
th
International Conference on DEA Brazil2012 58
Therefore, in a race where 20 cars cross the start line, the car crossing the finish line
first is given weight 20; with weight 19 given to the second place; 18 to the third
place; and so on, up to the last car completing the race. This way, all competing cars
are ranked.
After efficiency was calculated using the DEA tool, each competitor was attributed an
efficiency frontier value. In this case, the driver crossing the finish line first was
attributed an efficiency value 1 or 100%.
The other efficiency values were attributed according to the number of competing
cars, which means the efficiency frontier value increases when the number of
competing cars increases.
The Figure 3 (in APPENDICES), shows the results calculated by the DEA model for the
first Grand Prix (England - 1950), where it is reported the name, the position in the
hallway and Rank obtained. Thus the sum of the Scores of all Grand Prix shows the
champion, according DEA efficiency.
Conclusions
FIAs criteria determine that the ranking be based on points awarded to drivers
according to their position at the end of the race, with up to a number of positions
scoring points, and the remaining drivers scoring no points, regardless of the number
of drivers. Sometimes the scoring criteria, which is set out annually, are changed at the
end of a season. By using this new scoring method, i.e., the DEA efficiency frontier,
these comparisons become a reality, since the assessment is based only on the order
of arrival, which is translated into an efficiency ranking.
References
FARREL, M. J. (1957) The measurement of productive efficiency. Journal of the Royal
Statistic Society, London, v. 120, n. 3, p.253-290.
FIA. http://www.fia.com/en-GB/Pages/HomePage.aspx. Acessado em 30/jan/2011.
Access 30/jan/2011.
EMROUZNEJAD, Ali. (1995-2000) Coventry: Warwick Business School, Data
Envelopment Analysis Home Page. Disponvel em:
<http://www.deazone.com/index.htm>. Access 25 mai 2005.
FORMULA 1. http://www.formula1.com/default.html. Access 30/jan/2011.
JUBRAN, L.M.P. (2005) Aplicao da Anlise por Envoltria de Dados: um estudo da
eficincia das companhias seguradoras. 2005. 143 p. Dissertao (Mestrado)
Departamento de Engenharia Eltrica, Escola Politcnica da Universidade de So
Paulo. So Paulo.
PRADO, D. L. Pontos na F1: Deciso polmica da FIA no a primeira na histria.
Disponvel em < http://www.dzai.com.br/>, 2009. Access 30/jan/2011.
PUCCINI, A. L.; PIZZOLATO, N. D. (1989) Programao linear. 2.ed. Rio de Janeiro:
LTC.
Proceedings of the 10
th
International Conference on DEA Brazil2012 59
Appendices
Table 1: Distortions between scoring criteria 2009 and 2010.
Fig. 3: Results calculated by the DEA model
Proceedings of the 10
th
International Conference on DEA Brazil2012 60
9. Data Envelopment Analysis
Type Linear and Goal
Programming Models For
Measuring Energy Efficiency
Performance of OECD Countries
Hasan BAL
Gazi University, Science Faculty Department of Statistics 06500 Ankara TURKEY,
hasanbal@gazi.edu.tr (corresponding author)
Mehmet Guray UNSAL
Gazi University, Science Faculty Department of Statistics 06500 Ankara TURKEY,
mgunsal@gazi.edu.tr
Abstract
Models planned in this work are as follows: the goal programming DEA ranking
model recognized as an alternative to multiple criteria DEA and the model which
closes each weighted output (input) component to weighted output (input) sum, and
hence providing a contribution to efficiency account of each output (input)
component proportional to the output(input) values. The proposed models are
applied to measure the energy efficiency performances of OECD countries and the
results obtained are presented.
Keywords: Linear and Goal Programming, Energy Efficiency, Data Envelopment
Analysis
Introduction
DEA was first developed by Charnes et al. [1] that seems to be the most popular
method for measuring the efficiency of homogenous decision meaking units. Bal et al.
[2] suggest goal programming approaches to improve the discrimination power of
DEA. Sexton et al. [3] and Doyle and Green [4] suggest cross efficiency evalution as
an extension of DEA aimed at avoiding some of the mentioned difficulties. Their
technique made use of the cross evaluation scores computed as related to all DMUs
and hence identified the best DMUs [5].
The problem of having multiple optimal solutions to weights for efficient DMUs affect
to a great extent the consistency of operations related to weight cross efficiency
method is most frequently studied topic in DEA literature. Sexton et al. [3] and Doyle
and Green [4] recommended the use of a secondary objective (model) for the cross
efficiency evaluation related to the non-uniqueness of optimal weights in DEA. They
proposed the aggressive and benevolent models for achieving the secondary
objective. Andersen et. al. [5] proved the fixed weighting nature of the cross efficiency
Proceedings of the 10
th
International Conference on DEA Brazil2012 61
evalution in the case of single input and multiple outputs. Orkcu and Bal [6] proposed
goal programming models that could be used in the second stage of the cross
evalution.
DEA, Super Efficiency, Cross Efficiency Evaluation and Multiple
Criteria DEA Models
Data Envelopment Analysis (DEA) is a mathematical programming approach that
utilizes multiple inputs and outputs to measure the relative efficiencies within a group
of decision making units (DMUs). In DEA, it is assumed that there are n DMUs to be
evaluated in terms of m inputs and s outputs. Let
ij
x ( 1, . . . , i m = ) and
rj
y ( 1, . . . , r s = ) represent the input and output values of DMU
j
( 1, . . . , j n = ),
respectively. Subsequently, the efficiency of DMU
p
can be calculated as
1
1
s
r rp
r
p m
i ip
i
u y
v x
=
=
=
(1)
where,
i
v ( 1, . . . , i m = ) and
r
u ( 1, . . . , r s = ) are the input and output weights
assigned to
th
i input and
th
r output,respectively.
1
max
s
p r rp
r
u y
=
=
s.t.
1
1
m
i ip
i
v x
=
=
(2)
1 1
0
s m
r rj i ij
r i
u y v x
= =
, 1, 2, . . . , j n =
0
r
u , 1, . . . , r s =
0
i
v ,
i=1, ..., m
In the above-mentioned models, DMUs is considered to be efficient if and only if
*p=1; otherwise, it is referred to as non-efficient.
DEA can be used only for ranking inefficient DMUs and in order to abolish this
disadvantage various methods were developed [8]. The most commonly used method
developed for ranking efficient decision units is the super efficiency model proposed
by Andersen and Petersen [9].
In addition, the cross efficiency method was developed as a DEA extension tool to be
utilized for identifying the best performing DMUs, and for ranking DMUs using cross
efficiency scores that are linked to all DMUs [3,6]. In the first stage, the optimal
weights of inputs and outputs are calculated for each DMU using the classical DEA
formulation. Given the results of the first stage, the weights used by the DMU can be
utilized for calculating the peer rated efficiency for each of the other DMUs. The peer
Proceedings of the 10
th
International Conference on DEA Brazil2012 62
evaluation score, p,j, indicates the efficiency score for the DMU
j
using the weights
obtained by the DMU
p
[5].
,
1
,
,
1
s
r p rj
r
p j m
i p ij
i
u y
v x
=
=
=
. (3)
If we consider the multiple criteria DEA model, CCR model could be expressed
equivalently in the form given by Li and Reeves [10].
1
min or max
st
s
p p r rp
r
d u y
=
| |
=
|
\ .
1
1 1
1 (4)
0
1, 2, . . . ,
, , 0 , all , and values
p
m
i i
i
s m
r rj i ij j
r i
r i j
v x
u y v x d
j n
u v d r i j
=
= =
=
+ =
=
where
p
d
is the deviation variable for DMU
o
. This DMU is efficient if and only if
0
p
d = or 1
p
= . A multiple criteria data envelopment analysis model formulation with
the minmax and minsum criteria, which minimizes a deviation variable, rather than
maximizing the efficiency score, is shown as below (MCDEA):
1
1
1
1 1
min or max
min
min (5)
st
1
0
1, 2, . . . ,
o
o
s
o o r rj
r
n
j
j
m
i ij
i
s m
r rj i ij j
r i
j
d w u y
M
d
v x
u y v x d
j n
M d
=
=
=
= =
| |
=
|
\ .
=
+ =
=
0 , 1, 2, . . . ,
, , 0 , All , and
r i j
j n
u v d r i j
=
Goal Programming DEA and Component Models
MCDEA which is given in model (6) can also be easily adapted to the weighted goal
programming as follows:
(GPDEA-CCR)
Proceedings of the 10
th
International Conference on DEA Brazil2012 63
1 1 2 3
1 1
1
2 2
1
1 1
min , ,
1
1 (6)
0 , 1, 2, . . . ,
j j
j j
m
i ip
i
s
r rp
r
s m
r rj i ij j
r i
j
a n p p n d
v x n p
u y n p
u y v x d j n
M d
=
=
= =
= + +
`
)
+ =
+ =
+ = =
3 3
0 , 1, 2, . . . ,
j j
n p j n + = =
0, 1,2, . . . ,
0, 1,2, . . . ,
r
i
u r s
v i m
=
=
1 1 2 2
3 3
0, 1, 2, . . . ,
, , , 0
, 0 , 1, 2, . . . ,
j
j j
d j n
n p n p
n p j n
=
=
Where for the DMU under evaluation,
1
n and
1
p are the unwanted deviation variables
for the goal which constraints the weighted sum of outputs less than or equal to unity,
2
n is the wanted deviation for the goal which makes the weighted sum of outputs less
than or equal to unity.
3j
n ( 1, 2, . . . , j n = ) are the unwanted deviation variables for
the goal which realizes M as the maximum deviation, and
3
j
p are the wanted
deviation variables for the same goal. Whereof our aim, given equal weight to the
unwanted deviations, is to minimize the sum of unwanted deviations, is to minimize
thesum of unwanted deviations
1
n , p (
1
1
m
i io
i
v x
=
=
) and
2
p ,
3j
j
n
and
j
j
d
.
The proposed approach is aimed at approximating each weighed output (input)
component to weighted output (input) sum in order to contribute to the efficiency
account of each output componenet in proportion to the output(input) values, i.e., to
the extent of their greatness or smallness. It is also aimed at obtaining weights that are
more appropriate when compared to those obtained by classical DEA. In the second
stage of cross evalution, a model is presented (7) in which the classical DEA
efficiency scores for each unit are preserved and more appropriate optimal weight
values are selected for the units for which the optimal weights obtained by classical
DEA in the first stage possibly have multiple and inappropriate solutions.
Proceedings of the 10
th
International Conference on DEA Brazil2012 64
*
1
1
1 1
1
1
min
st
1 (7)
0 j 1, 2, . . . ,
r 1, 2, . . . , s
p
p
p p
s
r rp p
r
m
i i
i
s m
r rj i ij
r i
s
r rj r rp p
r
m
i ij i i p
i
w z
u y
v x
u y v x n
u y u y z
v x v x z
=
=
= =
=
=
=
=
=
=
=
1, 2, . . . , m
, , 0 , all , values
r i p
i
u v z r i
=
Here,
*
p
is the efficiency value for p. DMU obtained from the classical DEA.
Application
In this section, the methods, which are mentioned above, are used to compare the
performance of OECD countries in respect of CO2 emissions, energy consumption
[10]. According to Ramanathans study [10], the input variables are CO2 emissions per
capita (denoted as CO2 per cap hereafter), fossil fuel energy consumption (FOSS), and
the output variables are gross domestic product per capita (GDP per cap) and non-
fossil fuel energy consumption (NFOSS) to measure the energy consumption and
CO
2
emmissionefficiency.
Table 1. DEA-CCR Results of Application Study
Proceedings of the 10
th
International Conference on DEA Brazil2012 65
Table 2. GPDEA-CCR Results of Application Study
Table 3. Component Model Results of Application Study
Table 4. Coefficients of Variation For Weights
According to the obtained results in Table 1-4, it could be seen GPDEA and
Component models have fewer efficient DMUs than classical DEA model. According
to coefficients of variation for weights of inputs and outputs, the proposed models
generally decrease the variation of weights of variables.
Proceedings of the 10
th
International Conference on DEA Brazil2012 66
Table 5. Rankings of DMUs For Methods
Table 6. Spermans Rank Correlations According to Ranking of DMUs
Conclusions
It could be seen that GPDEA and Component models have fewer efficient
DMUs and the proposed models generally decrease the variation coefficient of
weights of variables. The models models also have significant and high
correlation between each other according to ranking scores of DMUs.
References
A. Charnes, W.W. Cooper, E. Rhodes, Measuring the Efficiency of Decision Making
Units, Eur. J. Oper. Res., 2 (1978) 429444.
H. Bal, H.H. rkc, S. elebiolu, Improving the Discrimination Power and Weight
Dispersion in the Data Envelopment Analysis, Comput. Oper. Res., 37 (1) (2010) 99
107.
T.R. Sexton, R.H. Silkman, A.J. Hogan, Data Envelopment Analysis: Critique and
Extension. In: Silkman R.H. (Ed.), Measuring Efficiency: An Assessment of Data
Envelopment Analysis, 32. Jossey-Bass, San Francisco (1986) 73-105.
J.R. Doyle, R. Green, Efficiency and Cross Efficiency in Data Envelopment Analysis:
Derivatives, Meanings and Uses, J. Oper. Res. Soc., 45 (5) (1994) 567578.
T.R. Andersen, K.B. Hollingsworth, L.B. Inman, The Fixed Weighting Nature of a Cross
Evaluation Model, J. Prod. Anal., 18 (1) (2002) 249255.
Proceedings of the 10
th
International Conference on DEA Brazil2012 67
H.H. rkcu, H. Bal, Goal programming approaches for data envelopment analysis
cross efficiency evaluation, Appl. Math. Comput., 218 (2011) 346-356.
R. Ramanathan, An analysis of energy consumption and carbon dioxide emissions in
countries of the Middle East and North Africa, Energy, 30 (2005) 28312842.
D.L., Retzlaff-Roberts, Relating discriminant analysis and data envelopment analysis to
one another, European Journal of Operational Research , (1996) 23: 311-322.
P., Andersen, N., Petersen, A procedure for ranking efficient units in Data
Envelopment Analysis, Management Science, 39(10), (1993) 1261-1264.
X. B., Li, G.R., Reeves, A Multiple Criteria Approach to Data Envelopment Analysis,
European Journal of Operational Research, 115: (1999) 507517.
Proceedings of the 10
th
International Conference on DEA Brazil2012 68
10. Decentralization and
productivity of the public
health service in Brazil
Alssio Tony Cavalcanti de Almeida
UFPB Federal University of Paraiba, Brazil, alessio@ccsa.ufpb.br
Carlos Eduardo Gasparini
*
UFPB Federal University of Paraiba, Brazil, ceg2@uol.com.br
Abstract
The scientific literature has pointed to the process of fiscal decentralization as a
potential inducer of efficiency and productivity in the public sector. However, some
authors have questioned whether the process in Brazil would actually generate a
waste of resources and raise problems in the quality of services provided. This paper
uses the Malmquist index and econometrics with panel data to empirically assess the
question of the relationship between fiscal decentralization and performance of the
public health service in Brazil, as well as to provide an overview of the dynamics of
regional productivity in the sector. The results allow us to observe that the
decentralization of health spending has a negative relationship on the productivity of
these services, but fiscal responsibility has a greater influence on the performance of
the local governments.
Keywords: Health, Decentralization of Expenditures, Productivity, Fiscal
Responsibility.
Introduction
Worldwide, health care is of increasing concern, both politically and socio-
economically. In Brazil, the health system experiences increasing pressure to improve
its performance, both in regard to controlling the cost of services and ensuring greater
access and better quality health care is available to the population. To analyze the
current stage of this system it is necessary to understand the process of
decentralization, particularly with the creation of the Sistema nico de Sade (Unified
Health System, SUS), provisions for which are contained within the 1988 Constitution,
where states and municipalities gained the biggest transfers of funds and
responsibilities over the provision of health services.
Oates (1977 and 2005), among others, notes that the fiscal decentralization process
generates a number of benefits to society, given that local governments can provide
goods and services more efficiently, which are more relevant to local preferences and
demands. On the other hand, critics like Prud'homme (1995) point to the number of
*
Recipient of financial aid from IPEA Brazil.
Proceedings of the 10
th
International Conference on DEA Brazil2012 69
risks involved. In an analysis of the Brazilian case, Campos (1998) points out that the
SUS is a system designed to decentralize, "within the government," the management of
the Brazilian public health service, i.e., transferring some powers to states and
municipalities. However, he goes on to highlight the level of unpreparedness of sub-
national governments, in both administrative incapacity, nepotism, lack of technical
resources, inexperience, high level of corruption, etc - to assume the important
responsibilities of the provision of health care.
Against this backdrop, this study aims to examine whether the increased
decentralization of public health services favored or not the productive performance
of the sector in Brazil. In Brazil, several studies attempted to measure the efficiency of
the provision of public goods and services, e.g.: Gasparini and Ramos (2004) and
Faria et al. (2008). However, this study, besides considering the growth of
productivity, advances on two aspects of this issue. First, it focuses explicitly on the
relationship between changes in productivity in the public health service and
decentralization. In addition, it offers a regionalized analysis of the dynamics of
productivity in the sector in Brazil, considering the indicators of technological change
and efficiency for the years 1996 to 2007.
This paper is organized into four parts including this introduction. The next section
presents a description of the underlying data and methodological procedures based
on the nonparametric estimation of the technological frontier and the Malmquist
index, together with the econometric approach. The third section contains the analysis
of results and, finally, the fourth section summarizes the principal conclusions of this
study.
Methods
To achieve the set goals, the empirical analysis of the study was divided into two
stages. The first builds a dynamic index of productivity growth for public health
services, using data from the municipalities aggregated at the state level, where the
indicator can be divided into changes in efficiency and technical innovation. This
index is intended to ascertain the best relations of efficiency and technical changes
obtained during the period between 1996 and 2007. To calculate this indicator, we
used the Malmquist index of productivity, with the help of non-parametric method
Data Envelopment Analysis (DEA) to estimate the necessary efficiency scores. This
approach was chosen as it handles simultaneously multiple inputs and outputs that
are typical of the health sector, and also not to impose functional form on the
production frontier
1
. Thus, for the development of Stage I we used the following
production function of public health services:
(
1
,
2
) = (
1
,
2
,
3
)
(1)
1
The concept of Malmquist productivity index was first introduced by Malmquist (1953) and later
refined by several works, including Caves et al. (1982), Fre et al. (1994) and Thrall (2000). This index
represents the growth of total factor productivity (TFP) of decision making units (DMU), which reflect
two components: efficiency change and technological change over time.
Proceedings of the 10
th
International Conference on DEA Brazil2012 70
here:
1
= Number of public hospital beds;
2
= Number of doctors;
3
= Number of
nurses;
1
= Number of hospitalizations;
2
= Number of consultations. These
variables were taken from the DATASUS site (Ministry of Health - Brazil).
The second stage of the study evaluates, by means of an econometric approach with
Panel Data, the relationship between the indicator of productivity growth in health (as
calculated in the previous stage) and variables related to the issue of Brazilian fiscal
federalism and other relevant socioeconomic factors. In this step, we seek to precisely
evaluate the effect of decentralization on the performance of the provision of public
health in Brazil. The model final to be estimated in Stage II is structured as follows:
=
0
+
1
+
2
+
3
+
6
+
7
4
=1
+
10
(2)
Where:
= (, )|
, 0
, , 0
=1
=1
(1),
where is the row vector with all elements equal to one and R
n
is the intensity
vector, and L and U are the lower and upper bounds of the sums of the intensity
respectively. The production possibility set (X,Y)
t
is characterized by frontiers that are
composed of (x,y) (X,Y)
t
such that is not possible to improve any element of the
input x or any element of the output y without worsening some other input or output.
This frontier is called the frontier technology at the period t. In the Malmquist index
analysis, the efficiencies of DMUs (x
o
,y
o
)
1
and (x
o
,y
o
)
2
are evaluated by the frontier
technologies 1 and 2 in several ways.
The Malmquist index (MI) is computed as the product of Catch-up and Frontier-Shift:
MI = (Catch-up) x (Frontier-Shift) (2)
Proceedings of the 10
th
International Conference on DEA Brazil2012 78
Catch-up=
2
((
)
2
)
1
((
)
1
)
(3) Frontier-Shift=
1
((
)
1
)
2
((
)
1
)
1
((
)
2
)
2
((
)
2
)
1/2
(4)
MI=
1
((
)
2
)
1
((
)
1
)
2
((
)
2
)
2
((
)
1
)
1/2
.. (5)
The obtained output with the DEA Malmquist algorithm was adjusted through
bootstrapping techniques following Kneip et al (2008).
Panzar and Roses market power model
Panzar and Rose (1987) have a reduced form approach using banking information or
bank level to discriminate between perfect competition, monopolistic competition and
monopoly. The Panzar and Rose methodology shows how the changes in input prices
are reflected in the balance of industry or revenues of a specific bank.
The methodology is applied to the banking sector through the following equation:
(
) = +
(
,
)
(6)
INTR
it
is ratio of total interest revenue to total assets of bank i at time t. P
f, it
and X
k, it
denote the input price of factor f and control variable k, respectively, of bank i at
time t.
The Panzar and Rose H-statistic can be calculated as: =
(7)
Where, H is the sum of the elasticities of the (scaled) total revenue of banks with
respect to their factor input prices.
Liberalization model
Following Brissimis et al (2008), the liberalization model is specified as:
=
0
+
1
+
2
+
3
+
4
+ +
=
n
i
n
j
j ij
m
i j
j ij
m
i
i amb
n
i
n
j
j ij
m
i j
j ij
m
i
i g
i
i
W ae Y ab ab D
W e Y b b D
Y
C
v
1 1 1 1 1
1 1 1 1 1
int
ln ln
2
1
ln ln
2
1
) ln(
ln
'
Economies of scope were deducted from the marginal cost of the overlapping outputs
between specialized and integrated institutions. We determined the marginal costs at
varying levels of output to account for differences in scale between institutions. The
marginal costs were derived as follows:
( )
| ( ) ( ) |
j
n
i
i ji
m
i
i ij j
j j j
j
Y
C
W e Y b b
Y
C
Y
C
Y
C
mc * ln ln
) ln(
ln
1 1
= =
+ + =
=
Data
The mental health care institutions in the Netherlands report to the Ministry of Health
in the Netherlands. The yearly data collected for this purpose were used in this
analysis over the years 2008-2010. We selected the institutions that dealt with mental
health care only, so departments of psychiatry as part of general hospitals for example
were not included. We selected those institutions that had valid and plausible values
for all variables. In total 201 observations (institutions per year) remained (59 in 2008,
73 in 2009 and 69 in 2010).
We included four measures of treatment as output variables: the number of counsels,
the number of days in residence, the number of part-time treatments and the number
of day activities (Table 1). Of these institutions, 32 were specialized in the sense that
they only performed counseling. The other institutions did counseling and at least one
of the other treatment activities. None of the institutions was specialized in one of the
other activities. We therefore used two groups to which we refer to as specialized and
integrated institutions. We used two inputs, personnel and material and capital. The
latter two were added and used as one variable. Costs of the inputs were available
and we used the number of full time jobs to calculate the price of personnel. The
prices of materiel and capital were based on an index constructed from the Consumer
Price Index and a Price Index for investments of fixed activa of the government
(Statistics Netherlands, www.cbs.nl).
Proceedings of the 10
th
International Conference on DEA Brazil2012 86
Results and discussions
We estimated the cost function under the assumption of a common technology for
both specialized and integrated institutions and under the assumption of different
technologies (Table2). The hypothesis of the same technology for both groups was
rejected by the log likelihood ratio test (p< 0.001).
The parameters of the models are very different. Moreover, under the assumption of a
common technology the signs of two of the outputs become negative. The parameter
of output 1 (counseling) had a value that was close to the value of the specialized
institutions under the assumption of different technologies, but then overestimated the
total costs of the integrated institutions. Therefore, the parameters of the other outputs
were given values that decrease the total costs of the integrated institutions. Under the
assumption of different technologies, the costs of the specialized institutions were
slightly more influenced by costs of materiel and capital than by personnel, compared
to the integrated institutions. No significant change was found over time. The model
fulfilled the criteria of monotonicity and concavity.
Since the model with the assumption of the same technology for both types of
institutions does not result in plausible estimates, we report economies of scale and
scope only for the model with different technologies. The integrated institutions
operate under diseconomies of scale on average (cost flexibility 1.05). This particularly
applies to the relatively small integrated institutions (Table 3). The larger institutions
operate under increasing economies of scale. The specialized institutions operate
under economies of scale on average (0.72). However, the cost flexibility rapidly
increases for these institutions and the larger ones operate under diseconomies of
scale. All the specialized institutions are smaller than 0.5 of the average size institution.
The economies of scope follow from the marginal costs of counseling in a specialized
or integrated institution with the same scale (Table 4). The marginal costs of
counseling are higher for the specialized institutions than for the integrated
institutions, except for the very small institutions. Moreover, costs increase with scale
for the specialized institutions, and decrease with scale for the integrated institutions.
The marginal costs of all other products also decrease with scale, reflecting the
economies of scale.
Conclusions
Specialized and integrated mental health care institutions in the Netherlands vary in
the way they operate, as shown by the different technology assumption. We also
found indications that increasing the scale of the institutions and that integrating
counseling in institutions with other types of treatment could increase the productivity
of the sector.
The assumption of the same technology did not give plausible estimates because of
the negative parameter values of two of the outputs. Instead of using dummy
variables, we also tried to replace the zero-values with the minima of the non-zero
values and estimate the model under the assumption of the same technology for all
institutions. The estimates were very similar then to the estimates of the model that
Proceedings of the 10
th
International Conference on DEA Brazil2012 87
allows for different technologies, as derived for the integrated institutions alone. This
approach also led to the rejection of the hypothesis of equal technologies.
The technology can differ between institutions for several reasons. These differences
can be caused by the kind and severity of the disorders that are treated, and for
example by the average age of the patients. We had data on the types of treatments as
expressed by the outputs we used, but did not have any information about the
patients or the disorders that were treated. A Norwegian study showed a substantial
impact of case mix on productivity growth estimates but did not differentiate between
types of institutions (Halsteinli et al., 2010). Given the data, we could only differentiate
between institutions that had counseling as a single output, and all other institutions.
No further differentiation was possible for example of institutions that only performed
one of the other outputs.
We used the marginal costs as a proxy for the economies of scope. Due to the lack of
observations we could not estimate a separate cost function without counseling.
Moreover, the costs cannot be estimated by a translog function in case of a zero
output. Direct estimation of the economies of scope was therefore not possible. We
compared the marginal costs of specialized and integrated institutions of the same
size. The differences in costs could be overestimated because the size of the
integrated institution that relates to counseling is smaller and the marginal costs of
counseling increase with a decreasing scale for these institutions.
We used 201 observations of mental health care institutions over a 3 year period to
compare the costs of the institutions relative to their production. Unfortunately, we
had to exclude another 346 observations because of missing or implausible values. A
substantial part of the excluded institutions had high costs relative to the number of
outputs. Most of the excluded institutions perform other activities which are not
counted in the outputs we used, like for example reintegration activities or parental
support. We did not have usable data to systematically exclude institutions for this
reason. It is therefore likely that some of the institutions with other activities are
included in the analysis. Because of the high costs relative to the output, this would
underestimate the cost efficiency. Due to a lack of data of the costs of mental health
care activities alone, we were not able to include 137 health care institutions with
more than just mental health care, like for example the general hospitals.
Summarizing, this analysis covers a substantial but selective part of the mental health
care sector.
Measuring the output of health care institutions is subject of debate (Hollingsworth &
Street, 2006). In this study we only used the number of treatments and not the number
of patients or the effectiveness of the treatment. We used quality of care
measurements from interviews to evaluate the relation between the efficiency score of
the institutions and quality. We found a negative relation with patient perception of
the effectiveness of the treatment and a non-significant correlation with all other
quality measures (results not shown).
The plausibility of the same technology assumption between firms or institutions has
been largely unexplored. The couple of studies we found came to the same
conclusion that different cost functions are required for specialized and diversified
firms. Differences in cost functions were for example found between general and
Proceedings of the 10
th
International Conference on DEA Brazil2012 88
specialty hospitals in Vietnam (Weaver & Deolalikar, 2004), between firms that
provided both freight and passenger railway services in the US and firms that offered
primarily freight services (Weninger, 2003) and between the water and sewerage firms
in England and the water only companies (Bottasso et al., 2011).
In conclusion, specialized and integrated mental health care institutions in the
Netherlands operate under different cost functions. The issue of different technologies
deserves more attention. Drawing the wrong conclusions from the possibly incorrect
same technology assumption can have major policy implications on scale and scope.
References
Baumol, J., Panzar, J.C., & Willig, R.D. (1988). Contestable markets and the theory of
industry structure Sydney: Marcourt Brace Jovanovich.
Bottasso, A., Conti, M., Piacenz, M., & Vannoni, D. (2011). The appropriateness of the
poolability assumption for multiproduct technologies: Evidence from the English water
and sewerage utilities. International Journal of Production Economics, 130(1), 112-117.
Christensen, L.R., Jorgenson, D.W., & Lau, L.J. (1973). Transcendental Logarithmic
Production Frontiers. The Review of Economics and Statistics, 55(1), 28-45.
Halsteinli, V., Kittelsen, S.A., & Magnussen, J. (2010). Productivity growth in outpatient
child and adolescent mental health services: the impact of case-mix adjustment. Soc
Sci Med, 70(3), 439-446.
Hollingsworth, B., & Street, A. (2006). The market for efficiency analysis of health care
organisations. Health Econ, 15(10), 1055-1059.
Weaver, M., & Deolalikar, A. (2004). Economies of scale and scope in Vietnamese
hospitals. Soc Sci Med, 59(1), 199-208.
Weninger, Q. (2003). Estimating multiproduct costs when some outputs are not
produced. Empirical Economics, 28(4), 753-765.
Proceedings of the 10
th
International Conference on DEA Brazil2012 89
Table 2 Summary data of included institutions
Average stddev min max
Output (x 1.000)
Counseling (contacts) 137.7 170.8 1.1 888.8
Residence (day) 112.8 125.9 0.0 503.5
Part-time treatment 11.8 16.3 0.0 71.2
Day-activity 22.5 46.3 0.0 258.9
Costs (x 1 million euro)
Personnel 34.1 37.6 0.2 153.9
Kapital 2.1 2.8 0.0 14.9
Material 12.3 12.8 0.1 54.2
Total 48.5 52.7 0.6 215.7
Total # FTE 609.6 646.7 10.8 2602.9
Price personnel (euro) 53556 9376 15514 87747
Proceedings of the 10
th
International Conference on DEA Brazil2012 90
Table 3 Parameter estimates
Shared technology Different technologies
Integrated institutions Specialized institutions
Variable Paramete
r
Estimat
e
Erro
r
t-
statistic
Estimat
e
Erro
r
t-
statistic
Estimat
e
Erro
r
t-
statistic
Constant A0 -0.59 0.09 -6.88 0.30 0.05 6.03 -1.47 0.16 -9.11
Counseling
(y1)
B1 1.35 0.04 31.17 0.25 0.06 4.31 1.40 0.06 22.30
Day in
residence
(y2)
B2 0.22 0.10 2.18 0.34 0.05 7.20
Part-time
treatment
(y3)
B3 -0.28 0.09 -3.27 0.30 0.06 5.38
Dayactivity
(y4)
B4 -0.17 0.05 -3.84 0.01 0.02 0.63
Personnel
(w1)
C1 0.71 0.01 65.71 0.70 0.01 61.28 0.55 0.04 15.38
Materiel &
capital (w2)
C2 0.29 0.01 27.39 0.30 0.01 26.10 0.45 0.04 12.46
Time A1 -0.03 0.03 -0.95 -0.02 0.02 -1.20 0.05 0.05 1.02
y1 x y1 B11 0.06 0.05 1.38 0.26 0.05 5.41 0.35 0.03 12.03
y1 x y2 B12 -0.34 0.07 -5.17 -0.15 0.03 -4.88
y1 x y3 B13 0.21 0.04 5.85 -0.17 0.04 -4.46
y1 x y4 B14 0.19 0.03 6.82 0.00 0.01 0.23
y2 x y2 B22 0.16 0.07 2.35 0.05 0.04 1.35
y2 x y3 B23 0.05 0.04 1.10 0.03 0.02 1.45
y2 x y4 B24 -0.03 0.01 -2.00 -0.01 0.01 -1.19
y3 x y3 B33 0.04 0.07 0.48 0.17 0.04 4.07
y3 x y4 B34 -0.15 0.03 -5.89 -0.02 0.01 -1.82
y4 x y4 B44 -0.01 0.01 -0.66 0.00 0.01 0.63
w1 x w1 C11 0.13 0.05 2.55 0.12 0.07 1.63 0.08 0.08 1.01
w2 x w2 C22 0.13 0.05 2.55 0.12 0.07 1.63 0.08 0.08 1.01
w1 x w2 C12 -0.13 0.05 -2.55 -0.12 0.07 -1.63 -0.08 0.08 -1.01
y1 x w1 E11 -0.01 0.01 -1.41 0.01 0.01 0.82 -0.08 0.02 -5.00
y2 x w1 E21 -0.02 0.01 -1.85 -0.01 0.01 -1.46
y3 x w1 E31 0.03 0.01 3.46 0.02 0.01 2.82
y4 x w1 E41 0.00 0.00 -0.73 -0.01 0.00 -1.84
Proceedings of the 10
th
International Conference on DEA Brazil2012 91
Shared technology Different technologies
y1 x w2 E12 0.01 0.01 1.41 -0.01 0.01 -0.82 0.08 0.02 5.00
y2 x w2 E22 0.02 0.01 1.85 0.01 0.01 1.46
y3 x w2 E32 -0.03 0.01 -3.46 -0.02 0.01 -2.82
y4 x w2 E42 0.00 0.00 0.73 0.01 0.00 1.84
R2 0.98 0.99
Loglikeliho
od
123 206
Table 4 Economies of scale
Total costs* Specialized Integrated
0.2 1.32 1.14
0.5 1.54 1.04
1.0 0.94
2.0 0.83
*Standardized costs: the costs of an average size institution are equal to 1
Table 5 Marginal costs by type and scale of the institution
Total Costs * Specialized Integrated
Counsel Counsel Days in residence Part-time treatment Dayactivities
0.2 113 142 183 779 81
0.5 172 134 172 854 60
1.0 117 149 873 32
2.0 92 114 827 3
*Standardized costs: the costs of an average size institution are equal to 1
Proceedings of the 10
th
International Conference on DEA Brazil2012 92
13. Efficiency analysis and long run
performance: a sequential
model for organizational
assessment
Frederico A. de Carvalho
*
Rua Dona Maria, 303 Javary -26900-000 Miguel Pereira RJ Brazil; UFRJ (the Federal University
at Rio de Janeiro); fdecarv@gmail.com
Marcelino Jos Jorge
Avenida Brasil, 4365 Manguinhos 21040-360 Rio de Janeiro RJ Brazil; IPEC / FIOCRUZ (the
National Institute for Infectology / Oswaldo Cruz Foundation); marcelino.jorge@ipec.fiocruz.br
Marina Filgueiras Jorge
Avenida Padre Natuzzi, 22 So Francisco - 24360-180 Niteri RJ Brazil; INPI (the National
Institute of Industrial Property); marinafj@inpi.gov.br
Abstract
This paper presents a sequential approach to organizational assessment from an
efficiency standpoint. The empirical illustration originates from data referring to the
period 2000-2007 and collected from a sample of 37 libraries affiliated to a federal
university in Rio de Janeiro; this sample covers some 90% of the population. In the
first and second steps efficiency scores computed from estimated DEA (Data
Envelopment Analysis) models are employed to rank DMUs (libraries) as well as to
provide pro-efficiency allocative corrections. The third step presents a long run
evaluation that is accomplished by Markovian analysis through computing the
corresponding equilibrium distribution between (efficiency) states. The Markovian
approach also provides some particular durations such as mean recurrence times
and first passage times that allow managerial interpretation. Since DMUs (libraries)
are here classified as efficient or inefficient according to computed annual scores,
the proposed model is systemic to the extent that only aggregate data are
considered.
Keywords:Organizational assessment. Efficiency analysis. Markovian analysis.
Academic libraries - Brazi.
Introduction
This paper presents an optimization approach to organizational evaluation from an
Efficiency Analysis standpoint. The approach combines in a simple way efficiency
scores computed from the estimation of selected Data Envelopment Analysis (DEA)
models and a long run evaluation provided by Markovian analysis. The proposed
*
Corresponding author
Proceedings of the 10
th
International Conference on DEA Brazil2012 93
model relies essentially on the application of the so-called Efficiency Principle to
assess organizational performance. Following the literature, organization may be
interpreted quite broadly as meaning both public and private entities, and even
nonprofit ones among the latter (VAKKURI, 2003). Hence, even though the empirical
illustration employs data from a sample of Decision Making Units (DMUs) pertaining
to a public organization, there is no loss of generality if and when other kind of
organizations are considered.
Methods
The Efficiency Principle simply states that, when studying the production process in
any organization, whenever a production unit uses the same resources but yields
greater quantities of output than another unit, it should be considered relatively more
efficient (i. e., relative to one another), no matter how formally the productivity
problem is analyzed. Analogously it should be considered relatively more efficient if
it uses fewer resources and yields the same output. From an analytical standpoint
these properties correspond to evaluating an organizational unit in terms of its
position vis vis an adequately defined and computed efficiency frontier. There is
an established body of knowledge - Data Envelopment Analysis (DEA), a class of
mathematical programming models with a now long tradition (Emrouznejad, Parker,
Tavares, 2008) of being applied to a broad range of situations involving the analysis of
production frontiers in a multi-unit, multi-input and multi-output framework in such a
way that usual parametric restrictions are absent.
In a seminal methodological paper Tulkens and Vanden Eeckaut (1995) describe and
explain the main issues relating to the role of time in nonparametric efficiency
analysis, especially in what concerns alternative ways to accommodate empirical
information into reference production sets that will be submitted to efficiency
computations. Of particular interest here (see Table I) is their classification of the
variety of forms whereby the time dimension present in panels may be treated when
investigating observed productive activity.
The paper by Wang and Huang (2007) introduces two models to examine long run
efficiency analysis. However they do not compute any long run solution in any of
those cases. In addition, although they have modelled and specified the probability of
one-step temporal transition from efficient (resp. inefficient) to inefficient (resp.
efficient) state, there seems to be no indication as to how those probabilities might be
used to compute long run structural distributions of the DMUs among the two states
(efficient or inefficient).
Using direct results from finite ergodic Markov chains (Kemeny, Snell, 1972), and
assuming one (estimated) aggregate transition matrix is available, it is possible to
compute the long run distribution of the system (the set of DMUs) between the two
states. This is the purpose of the third step in our model.
The model
Our proposed sequential model consists of three steps. The first two steps involving
the computation of efficiency scores and of operational plans in turn are typically
Proceedings of the 10
th
International Conference on DEA Brazil2012 94
performed through the application of Data Envelopment Analysis to empirical data on
DMU performance. The third step, the novel one, incorporates the structural long
run assessment of efficiency. The case is summarized in Table I according to the
Tulkens-Vanden Eeckhaut (1995) framework. Our example is supported by a
convenience sample of 37 library units, corresponding to more than 80% of total
population. Data were collected from the library systems Management Information
System and relate to three inputs (number of employees, physical area and number of
volumes) and four outputs (number of visits, of loans, of registers and of
consultations). The solution of the appropriate linear programming problem provides
numerical scores for each DMU that characterize them with respect to efficiency
status. For each inefficient DMU an operation plan is also provided that indicates
(re)allocative targets for the DMU to reach efficiency. Finally scores will also be
needed to compute the transitions between the two states along the time period for
the whole set of DMUs.
TABLE I. SUMMARY ON CASE STUDY
Case (DMUs)
Number of
DMUs
Number of
variables
Time Period
University libraries 37 7 2000 -2007
DEA condition
satisfied *
DEA model
Yes BCC-O
TVE classification**
Contemporaneous
Notes: *: number of DMUs not less than two (three) times the number of
variables.
**: classification of (sample) observed subsets by Tulkens-Vanden Eeckaut (1995)
As soon as a transition matrix is available, long run analysis is possible and will result
from the simple computation of a fixed point for the transition matrix (KEMENY,
SNELL, 1972). This fixed point is a vector containing the long run distribution of the
system. Since we do not follow the statistical approach applied by Wang and Huang
(2007), some form of combination must be chosen for the initial transition matrix. In
our model we use the following basic result (Kemeny, Snell, 1972): when the number
of time steps grows indefinitely one has
lim (1/n)(P + P2 + . . . + Pn) = [1 1 1 1] (2)
where n is the number of steps; Pn = (( pij (n) )) is the nth power matrix, whose
(i ; j) element represents the probability of transition from state i to state j after n
steps; [1 1 ;1] is a column-vector with all elements equal to 1; the apostrophe
means transpose and is a constant row-vector containing the long run equilibrium
distribution between states whose components are nonnegative and sum to 1 (a so-
called probability vector). The finite mean in the left hand side of (2) suggests a way
to estimate a single matrix from the seven available and then to compute the long run
corresponding to this averaging matrix.
Proceedings of the 10
th
International Conference on DEA Brazil2012 95
Results and discussions
In this section findings are presented according to the order of proposed steps
computed efficiency scores, operation plans and long run distribution.
First step: efficiency scores are computed and DMUs may be ranked accordingly
A sample profile for the 37 DMUs is given in Table II for the last year of the period of
study. The basic results for any DEA analysis namely, computed efficiency scores
appear in Table III.
Second step: operation plans indicate optimal changes for each library along time
Operation plans are always produced as a typical result from a DEA solution and in
this example they appear in a consolidated form in Table IV. In every individual
matrix (not exhibited here) showing the allocative change for each (inefficient) library
and each year, there are indications of resource decrease and output increase; this
information is summarized in that table and deserves managerial attention.
Third step: first passages, mean recurrence and long run change
In order to compute, for the whole system of libraries, the transition matrix between
the two states - efficient and inefficient - all transitions are now considered in
addition to those implied in Table III, where the trivial case of DMUs that were
efficient along the whole period is only quoted in a footnote.
Table 2. Sample profile for university libraries in 2007
Number
Employees
1 33 8,41 8,06 95,83%
Total area
( m2 )
37 6000 865,16 1400,03 161,82%
Volumes 872 277134 35228,9 53343,4 151,42%
Visits 108 137385 20974,7 33971 161,96%
Registrations 0 5603 1043,38 1115,4 106,90%
Loans 0 30191 5116,03 6578,68 128,59%
Consultations 0 66638 8091,62 12228,7 151,13%
Service mix
(number)
5 13 9,54 1,87 20%
Variables
Coefficient
of Variation
Min Max Mean
Standard
deviation
Given that we are working with contemporaneous reference sets (see Tulkens and
Vanden Eeckhaut, 1995, and Table I), data for 2000-2007 allow to obtain 7 transition
matrices, say P1, P2, , P6, P7 . In order to apply the finite sum approach, we
employ successive products of yearly transition matrices, instead of powers of the
same (initial or otherwise chosen) transition matrix. Therefore we can take the seven
factor average matrix A defined as
A = [P1 + P1P2 + P1P2P3 + ... + P1P2P3P6P7) / 7 (3)
as a good candidate to be used when solving the fixed point problem, since it
incorporates more information than each individual matrix, in addition to being a
good picture of the successive one-step, two-step until seven-step transitions, in the
Proceedings of the 10
th
International Conference on DEA Brazil2012 96
spirit of equation (2). Solving the fixed point equation A = we obtain the
(estimated) long run distribution of the system between states as follows
E ( percent efficient ) = 49,52%; NE ( percent inefficient ) = 50,48%.
Note that these figures slightly differ from the mean (48,65%, equal to the median) of
percents in the last line of Table III. In this sense it can be argued that long run
analysis seems to be of a different nature vis vis the arithmetic of numerical
individual scores. Remember that products of transition matrices bring into play all the
transitory visits to the two states along time.
The fixed point for the equation A = also provides directly the mean
recurrence time (Kemeny et al. 1959, p. 413) for the states of the system, that is, the
mean time required before the system returns to a given state having started in that
same state. The mean recurrence time is approximately equal to 2 years in both cases,
so that the period of two years seems to be critical in the sense of monitoring the
return of a state to itself. In the case of inefficiency it represents a sort of safe mean
time span for managers to try to change the operating conditions facing inefficient
units, Since the operation plans already point to optimal changes by unit, managers
may evaluate for which units those changes would be feasible within (the next) two
years. Note that on average an inefficient will return to inefficiency four months
before it may reach efficiency for the first time, if no managerial action is taken.
Proceedings of the 10
th
International Conference on DEA Brazil2012 97
Table III. Efficiency Scores* And Yearly Averages: 2000 2007
SCORES SCORES SCORES SCORES SCORES SCORES SCORES
2001 2002 2003 2004 2005 2006 2007
1
1 0,841 1 1 0,605 0,811 0,68 1
2
0,571 1 1 1 0,965 0,943 1 1
3
0,305 0,936 0,845 0,661 0,542 0,846 0,775 0,574
4
0,989 0,96 0,769 0,783 0,829 1 1 1
5
1 1 1 1 1 0,947 1 1
6
1 0,696 0,742 0,494 0,584 0,757 0,548 0,65
7
1 0,731 0,87 0,452 0,353 0,127 0,466 0,624
8
0,941 1 0,471 0,559 0,782 0,65 0,626 1
10
0,62 0,895 0,712 0,974 0,619 0,74 1 0,679
11
0,528 0,66 1 0,779 0,727 1 0,847 0,646
12
0,404 0,59 0,287 1 1 1 1 1
17
1 1 0,627 1 1 1 0,336 0,37
18
0,604 0,815 0,696 1 1 1 0,807 1
19
1 1 1 1 1 0,959 1 0,921
20
0,6 1 0,867 0,779 0,743 0,498 0,543 0,56
21
0,401 0,302 0,396 0,109 0,138 0,371 0,145 0,115
22
1 1 0,507 0,654 0,337 1 0,842 0,121
24
0,391 0,501 0,492 0,387 0,395 0,931 0,319 0,32
25
0,733 0,69 0,84 0,329 0,307 0,482 0,64 0,506
26
0,838 1 0,467 0,683 0,236 0,562 0,384 0,863
27
0,334 0,412 0,41 0,407 0,358 0,223 0,496 0,241
28
0,892 0,574 1 1 1 1 1 0,945
30
1 0,442 1 0,555 0,972 1 1 0,82
31
0,071 0,064 0,055 0,143 0,185 0,02 0,01 0,017
32
0,45 0,781 0,928 0,873 0,87 1 1 1
34
0,562 1 1 1 1 1 1 1
35
1 0,793 1 0,665 0,757 0,354 1 1
36
0,107 0,202 0,196 0,172 0,113 0,353 0,401 0,381
37
0,359 1 1 1 0,892 1 1 1
Mean
(n=37) 0,7486 0,8077 0,7886 0,7691 0,7381 0,7993 0,7801 0,7663
Percent
efficient 45,96% 48,65% 48,65% 48,65% 40,54% 51,35% 54,05% 51,35%
DMU
SCORES
2000
Note. * - All DMUs with scores equal to 1 for the whole period have been excluded
Conclusions
Upon assuming the Efficiency Principle as a guideline to organizational evaluation,
this paper presented a model for organizational assessment in the short and long runs
by combining two approaches the DEA approach to efficiency analysis and the
Markovian assumption that introduces a long run perspective. Results have shown that
the three step model uncovers quantitative aspects that may be of assistance to
managers committed to efficiency in the short and long runs. For long run assessment
we rely on Markov Chains to compute an aggregate measure of the distribution of the
productive system (the organization) between two states efficient or inefficient.
Proceedings of the 10
th
International Conference on DEA Brazil2012 98
Since DMUs (libraries) are here classified as efficient or inefficient according to
computed annual scores, the proposed model is systemic to the extent that only
aggregated data are considered. The other useful application of the Markovian
approach provided better knowledge concerning the time delay required for efficiency
to be attained for the first time when a prescribed operation plan happens to be
adopted, as well as about the time during which an undesired (inefficient) situation
will persist if that adoption is postponed.
Table IV. Average Operation Plans: 2000 - 2007
Inputs 2000 2001 2002 2003
Inputs 2004 2005 2006 2007
-1,44 -1,15 -0,76 -1,29
-0,93 -1,18 -0,61 -0,81
-60,75 - 71,04 * -29,85 -70,35
-48,94 -143,47 -88,05 -136,27
-3064,48 -3373,49 -1880,71 -4601
-6447,08 -651,77 -4720,75 -3153,65
Area (m2)
Volumes
(number)
Employees
(number)
Area (m2)
Volumes
(number)
Employees
(number)
Note * - this figure relates to a single library.
References
Emrouznejad A., B. Parker; G. Tavares (2008) Evaluation of research in efficiency and
productivity: a survey and analysis of the first 30 years of scholarly literature in DEA,
Socio-Economic Planning Sciences 42 (3): 151-157.
Kemeny J. G., J. L. Snell (1972) Mathematical Models in the Social Sciences.
Cambridge, Mass.: The MIT Press.
Tulkens H., P. Vanden Eeckaut (1995) Non-parametric efficiency, progress and regress
measures for panel data: methodological aspects, European Journal of Operational
Research 80 (3): 474-499.
Vakkuri J. (2003) Research Techniques and Their Use in Managing Non-profit
Organizations an illustration of DEA analysis in NPO environments, Financial
Accountability and Management 19 (3): 243-263.
Wang M. H., T. H. Huang (2007) A study on the persistence of Farrells efficiency
measure under a dynamic framework, European Journal of Operational Research 180
(3): 1302-1316.
Proceedings of the 10
th
International Conference on DEA Brazil2012 99
14. Efficiency in the industrial
sectors of Brazil in terms of
contributing to sustainable
development
Flvia de Castro Camioto
Department of Production Engineering, University of So Paulo (Brazil),
flaviacamioto@yahoo.com.br
*
Enzo Barberio Mariano
Department of Production Engineering, University of So Paulo (Brazil), enzo.mariano@gmail.com
Daisy Aparecida do Nascimento Rebelatto
Department of Production Engineering, University of So Paulo (Brazil), daisy@sc.usp.br
Abstract
The purpose of this article is to analyze the efficiency of the industrial sectors in Brazil
from 1996 to 2009, considering their contributions to the sustainable development of
Brazil. To this end, we used the Data Envelopment Analysis (DEA), which enabled,
from the additive model and the window analysis, to evaluate the ability of industries
to reduce environmental impacts and increase social and economic benefits. The
results of this study indicated that the Textile sector is the most efficient industrial
sector in Brazil in terms of contributing to sustainable development, followed by these
sectors: Foods and Beverages, Chemical, Mining, Nonmetallic, Paper and Pulp and
Metallurgical.
Keywords: Industrial Sectors, Data Envelopment Analysis, Sustainable Development.
Introduction
The data presented in the last Intergovernmental Panel on Climate Change (IPCC,
2007) indicate that global warming is largely due to human activity, especially human-
caused CO
2
emissions. Along these lines, fossil fuel burning has been shown to be
responsible for approximately 85% of all anthropogenic CO
2
emission produced
yearly.
Silva and Guerra (2009) explain that the use of fossil fuels has driven the world
economy since the Industrial Revolution, with energy as an essential component for
the social and the economic development of a nation and its supply is an essential
pre-requisite to human activities.
*
Address correspondence to Flvia de Castro Camioto, Department of Production Engineering,
University of So Paulo, Trabalhador So-Carlense, 400, So Carlos, SP, 13566-590 Brazil. Phone: +51 16
3373 9428, Fax: +55 16 3373 9425. E-mail: flaviacamioto@yahoo.com.br
Proceedings of the 10
th
International Conference on DEA Brazil2012 100
Thus, the environmental implications of the production and the use of energy
resources have been presented as a major challenge for developed and developing
countries, since the production, distribution, processing and consumption of energy
should be directed to ensure development, without increasing the negative effects to
society and the environment.
Despite the study by La Rovere and Simes (2008), which analyzed the availability of
renewable energy sources in Brazil, to conclude that Brazils energy matrix is relatively
clean, Brazils internal use of renewable energy is of 43.7% (BEN, 2010), the industrial
sector still has many of its activities dependent on fossil fuels. As a result, the industry
ends up impacting the environment by emitting extremely high concentrations of
greenhouse gases (GHG), increasing global warming, in addition to adding to the
extensive mining in the form of fuel oil and coal.
Considering that the industrial sector can significantly contribute to the challenge
against climate change, several studies on environmental and economic aspects in the
industry have been developed (Oggiono et al., 2011, Schneider et al. 2011; Tomasula
and Nutter, 2011; Wernet et al. 2011; Hamzah et al. 2010, etc.). It should be
mentioned, however, that most of these studies tend to focus on particular industrial
sectors, processes or products.
Despite the few works, such as that by Mao et al. (2011), which using statistical data
analyzed Chinas energy consumption and GHG emissions by industrial subsystem
and sector, and that by Luken and Castellanos-Silveria (2011), which compared the
changes in economic, environmental and social variables that occurred in the
manufacturing industry, in groups of developing countries, between 1990 and 2004,
there are still ample opportunities for studies covering various industrial sectors and
their contribution to promoting economic development with environmental respect
and social improvement.
Given this context, this articles main objective is to analyze the efficiency of the main
industrial sectors in Brazil, during 1996 to 2009, in view of their energy consumption
and their contributions to the sustainable development of the country. Therefore, this
work was developed to address the three basic pillars of sustainability, which are
economic, social and environmental development.
Methods
To reach the goal, a mathematical programming method called Data Envelopment
Analysis (DEA) was used. This method, based on the additive model and on the
window analysis, enabled analyzing the performance of the industrial sectors of Brazil
to reduce energy consumption and CO
2
emissions from fossil fuels (inputs), while
increasing the GDP by sectors, the persons employed and personnel expenses
(outputs).
For this research, the main Brazilian industrial sectors were selected, for which the
National Energy Balance (BEN) and the Brazilian Institute of Geography and Statistics
(IBGE) provided data, and due to lack of available information from IBGE some
sectors were grouped. Thus, for this work, the spatial delimitation of the Brazilian
industry includes: (a) Nonmetallic, which corresponds to the cement and ceramics
Proceedings of the 10
th
International Conference on DEA Brazil2012 101
sectors, (b) Mining, which corresponds to the mining and pelletizing, excluding oil,
natural gas and coal exploration, (c) Metallurgical, which corresponds to the sectors of
pig-iron and steel, iron alloys and non-ferrous (d) Chemical, (e) Foods and Beverages,
(f) Textiles, and (g) Paper and Pulp.
In addition to the energy consumption, the variables used in this analysis were: (1)
sectoral GDP, as a variable of economic growth; (b) personnel expenses in the form
of salaries, withdrawals and other remunerations, as a variable related to social
development; (c) persons employed in each sector, as a variable related to social
development; and (d) CO
2
emissions from fossil fuels, as a variable related to the
environmental development.
The data related to the variables "personnel expenses" and "persons employed" were
collected on the website of the Brazilian Institute of Geography and Statistics (IBGE).
The variables "GDP sectoral" and "energy consumption" were collected in the report
of the National Energy Balance (BEN), available on the website of the Ministry of
Mines and Energy (MME). The variable "CO
2
emissions from fossil fuels" was
calculated using the top-down method, internationally recognized and recommended
by the UN (United Nations) (IPCC, 1996). In order to calculate the carbon emissions of
the Brazilian energy system, the MCT (2006) adapted the top-down method,
recommended by IPCC (1996), for the particular characteristics of the Brazilian energy
system. Thus, in this work, for the calculation of CO
2
emissions, many of the data
used were drawn from this document.
The time interval to be analyzed in this study includes a period of fourteen years
(1996-2009), and the criterion used to define it was the data availability with the same
calculation base.
Results and discussions
First, in order to verify if the "CO
2
emissions from fossil fuels" and "energy
consumption" contributed to the formation of the output variables, we performed a
statistical analysis. Thus, we constructed a correlation matrix in order to verify the
linear correlations among all variables with their statistical significance. The software
used to perform such analysis was Stata MP 11.
Table 1 Correlation Matrix
Energy
consumption
CO
2
emissions
GDP by
sector
Persons
employed
Personnel
expenses
Energy
consumption
1,0000
p-value
CO
2
emissions 0.7235 1,0000
p-value 0,0000
GDP by sector 0.8833 0.4977 1,0000
p-value 0,0000 0,0000
Persons employed 0.4352 -0.2278 0.6602 1,0000
p-value 0,0000 0.0241 0,0000
Personnel
expenses
0.5230 -0.0099 0.7532 0.8399 1,0000
p-value 0,0000 0.9231 0,0000 0,0000
Proceedings of the 10
th
International Conference on DEA Brazil2012 102
Table 1 presents the results of the correlation matrix which showed that the all
variables of inputs were significant with p-value below 10%, for at least one output
and vice versa.
From the application of the variant additive model of DEA, window analysis was
performed to verify the efficiency of industrial sectors from 1996 to 2009, considering
both the reduction of inputs "energy consumption" and "CO
2
emissions from fossil
fuels", as the increase of outputs "GDP by sector ","persons employed" and "personnel
expenses". The results of this study indicated that the Textile sector was the one with
the highest average efficiency with respect to the variables considered, followed by
the sectors: Foods and Beverages, Chemical, Mining, Paper and Pulp, Nonmetallic,
and Metallurgical. It is important to remember that in the DEA-additive, the higher the
value of the objective function, the greater the inefficiency of the DMU. As for the
standard deviation of efficiency, the highest value was that of the Metallurgical sector,
followed by the sectors: Chemicals, Foods and Beverages, Mining, Nonmetallic, Paper
and Pulp, and Textile.
Thus, the Metallurgical sector, considering the variables analyzed in this work, are the
least efficient, being far below the others and with greater variability. From Table2, we
can see that this sector was becoming more inefficient as the oldest years were being
excluded and the most recent ones being contemplated in the windows, and the
average of the sum of the slacks increased from 1.99 in the first window (1996 -2002)
to 2.61 in the last one (2002 - 2009).
Table 2 - Window Analysis Sectors
Efficiency Means
Windows (means)
Total Means Standard deviation
1 2 3 4 5 6 7
Metallurgical 1.99 2.23 2.30 2.20 2.44 2.63 2,61 2.34 0.10
Paper and Pulp 0.60 0.70 0.74 0.76 0.78 0.88 0.90 0.76 0.01
Nonmetallic 0.66 0.73 0.75 0.75 0.75 0.82 0.80 0.75 0.01
Mining 0.41 0.45 0.46 0.41 0.36 0.33 0.32 0.39 0.03
Chemical 0.14 0.29 0.27 0.27 0.27 0.31 0.32 0.26 0.04
Foods and Beverages 0.06 0.19 0.22 0.18 0.13 0.22 0.17 0.17 0.03
Textile 0.01 0.02 0.01 0.01 0.01 0.02 0.02 0,02 0,0004
Then, as second to last in the efficiency ranking of Brazils industrial sectors, there is
the Paper and Pulp sector, which like the Metallurgical sector, its efficiency decreased
as the oldest years were excluded from the analysis and the most recent ones were
contemplated in the windows, as shown in Table 2. It is noteworthy, however, that
the variability of this sector was significantly lower than the metallurgical sector, with
the standard deviation equal to 0.01.
The third in the inefficiency ranking is the Nonmetallic sector. This sector also was
becoming more inefficient as the oldest years were being excluded and the most
Proceedings of the 10
th
International Conference on DEA Brazil2012 103
recent ones being contemplated in the windows. As the fourth sector, both in
efficiency and inefficiency, the Mineral Extraction sector, presented only in the last
four windows analyzed, increased efficiency as the oldest years were excluded and
the youngest years were included in the analysis. Thus, it can be said that the Mining
industry sector has shown improvement in recent years, in relation to their
contribution to sustainable development. The results showed that this sectors leap in
quality began in 2006.
Although the Chemical sector showed a good level of efficiency, being the third in the
ranking, it also showed a great variability over the years. An important feature to be
considered when analyzing the results of this sector is the fact that despite it has, in
general, worsened from window to window, the last year of each window was always
efficient, which demonstrates the sector has shown significant and rapid improvement,
with the most recent year of the window being always more efficient than the
previous ones. It should be mentioned that the only exception to this fact was the
years of 2008 and 2009, when the sector had a significant worsening. It is also
noteworthy that the years this sector stood out most, efficient in all the windows, were
the years 2004 and 2007.
Similar to the Chemical sector, the Foods and Beverages sector also showed high
variability in relation to efficiencies, and also worsened from window to window. This
sector, as well as the Chemical sector, showed significant and rapid improvement in
recent years, which can be corroborated by the fact that the last year of each window,
without exception, was efficient in relation to the others. It is important to mention
that the years the Foods and Beverages sector most stood out, being efficient in
multiple windows, were the years 2004 and 2006.
Finally, the sector that was most efficient by reducing inputs and increasing outputs, in
this work was the Textile sector. This sector showed high average efficiency in all
windows, besides having the lowest standard deviation relative to the other sectors. It
is noteworthy that the years this sector most stood out, effective in multiple windows,
were the years 2001, 2003 and 2004.
Conclusions
Adopting the concept of sustainability requires not only the viability of the economic
approach, but also the social and environmental variables in order to achieve a better
spread of the gains acquired by the use of the natural resources with minimum
damages to the planet and to humanity. A fair development process demands the
interaction of sustainability dimensions to harmonize different interests involving
economical growth and a social and ecological perspective.
Therefore, this article analyzed through the concept of efficiency, the contribution of
seven Brazilian industrial sectors for the sustainable development, which addressed
the three basic pillars of the triple bottom line, which are economic, social and
environmental development. Thus, we analyzed the efficiency of the Brazilian
industrial sectors from 1996 to 2009, to reduce the inputs "energy consumption" and
"CO
2
emissions from fossil fuels" and increase the outputs "sector GDP", "persons
employed" and "personnel expenses", simultaneously. The outcome of this study
showed that the Textile sector is the most efficient in terms of contribution to the
Proceedings of the 10
th
International Conference on DEA Brazil2012 104
sustainable development in Brazil, followed by the sectors of: Foods and Beverages,
Chemical, Mining, Nonmetallic, Paper and Pulp, and Metallurgical.
These results, besides the caution needed in interpreting the trends due to limited
data, must be seen as a first attempt to illustrate the contribution of the Brazilian
industrial sectors to sustainable development. In this sense, besides the variables
analyzed in this study that addressed the three pillars of sustainability, for future
works, it is possible to consider that adopting the sustainability concept includes one
of the main goals of achieving a greater fairness in income distribution, which would
imply the inclusion of new variables in the study.
According to the results of this work, it is important notice that all industrial sectors,
especially the Metallurgical, the Nonmetallic, and the Paper and Pulp, can still
contribute significantly to the complex challenge of promoting economic development
with social improvement and environmental respect. For this, the current and future
developments must be closely associated to the sustainable, efficient and secure use of
energy based on environmentally and economically viable approaches for the future
of society in the short and long term.
References
BEN - Balano Energtico Nacional, 2010. Divulga informaes relativas ao binmio
oferta consumo de fontes de energia. Available at: <https://ben.epe.gov.br>.
(Accessed 04.11.10).
Hamzah, M.O., Jamshidi, A., Shahadan, Z., 2010. Evaluation of the potential of
Sasobit to reduce required heat energy and CO2 emission in the asphalt industry.
Journal of Cleaner Production 18: 1859-1865.
IBGE Instituto Brasileiro de Geografia e Estatstica., 2011. Pesquisa Industrial Anual
Empresa. Available at: <http://www.sidra.ibge.gov.br>. (Accessed 04.05.11).
IPCC Intergovernmental Panel on Climate Change., 1996. Greenhouse gas inventory
reporting instructions Revised IPCC Guidelines for national greenhouse gas
inventories. In: United Nations Environment Program, the Organization for Economic
Co-operation and Development and the International Energy Agency, London.
IPCC - Intergovernmental Panel on Climate Change., 2007. Climate Change 2007:
Mitigation. Contribution of Working Group III to the Fourth Assessment Report of the
Intergovernmental Panel on Climate Change. B. Metz, O.R. Davidson, P.R. Bosch, R.
Dave, L.A. Meyer (eds). Cambridge University Press, Cambridge, United Kingdom and
New York, NY, USA.
Luken, R., Castellanos-Silveria, F., 2011. Industrial transformation and sustainable
development in developing countries. Sustainable Development 19: 167-175.
Mao, J., Du, Y., Xu, L., Zeng, Y., 2011. Quantification of energy related industrial eco-
efficiency of China. Frontiers of Environmental Science and Engineering in China. doi:
10.1007/s11783-010-0289-8.
Proceedings of the 10
th
International Conference on DEA Brazil2012 105
MCT Ministrio da Cincia e Tecnologia, 2006. Emisses de dixido de carbono por
queima de combustveis: abordagem top-down. Available at: <http://www.mct.gov.br>.
(Accessed 05.11.10).
Scheneider, M., Romer, M., Tschudin, M., Bolio, H., 2011. Sustainable cement
production - present and future. Cement and Concrete Research 41(7): 642-650.
Silva, F. I. A., Guerra, S. M. G., 2009. Analysis of the energy intensity evolution in the
Brazilian industrial sector - 1995 to 2005. Renewable and Sustainable Energy Reviews
13(9): 2589-2596.
Simes, A., La Rovere, E. L., 2008. Energy Sources and Global Climate Change: The
Brazilian Case. Energy Sources Part A: Recovery, Utilization & Environmental Effects
30: 1327-1344.
Tomasula, P. M., Nutter, D. W., 2011. Mitigation of Greenhouse Gas Emissions in the
Production of Fluid Milk. Advances in Food and Nutrition Research 62: 41-88.
Wernet, G., Mutel, C., Hellweg, S., Hungerbhler, K., 2011. The Environmental
Importance of Energy Use in Chemical Production. Journal of Industrial Ecology 15(1):
96-107.
Proceedings of the 10
th
International Conference on DEA Brazil2012 106
15. Efficiency in the management
of sanitation and its impacts on
health promotion: an aplication
of data envelopment analysis
DEA
Karlos Eduardo Arcanjo da Cruz
Universidade Federal de Pernambuco, Brazil, karlosarcanjo@hotmail.com
Francisco de Sousa Ramos
Universidade Federal de Pernambuco, Brazil, fsr@ufpe.br
ABSTRACT
This paper analyzes the public administration efficiency regarding the attenuation of
infant mortality. In many Brazilian regions there is a low coverage of sanitation
systems. This deficit in supply per se is directly associated with under five mortality.
However, the poor quality supply of service also causes a similar effect. Thus, this
paper analyzes the efficiency of the Brazilian states in the management of sanitation.
To do this, it was considered as the primary objective of sanitation to improve the
population welfare, which is translated by the elevation in the number of over-five-
year children who were born survivors for each thousand. As a result, it was found
that all the states in the South region are efficient, while the Southeast are not as good
as these, but have a score close to 100%. Northeast states show a low efficiency. In
the north, most of the units was ineffective, but better than the northeast units. When
comparing these results with the indicator of infant mortality, it is perceived that in the
Northeast the scarcity of infrastructure is complemented by the inefficiency and
produces a perverse effect that makes the region have the highest Under Five
Mortality Rate nationally.
Keywords: Sanitation, Infant Mortality, Efficiency, DEA.
Introduction
Services of Water Supply and Sewage Collection (SWSSC) in many developing
countries is quite impaired and very far from achieving universal service (Turolla,
2002; RIVERA, 1996). Over 2.6 billion people worldwide lack access to adequate
sanitation system and approximately 900 million people do not use drinking water
(WHO, 2010).
According to the World Health Organization, the inappropriate use of sanitation and
water is a major risk of mortality. It also has an additional adverse effect to be more
connected to regions of low income - 99% of its occurrence is in developing
countries. It is associated to a group of five risk factors that together account for 25%
Proceedings of the 10
th
International Conference on DEA Brazil2012 107
of global mortality and 84% of these deaths associated to children (WHO, 2009).
Moreover, it is directly associated to diarrhea, which is one of the major causes of
mortality of children under 5 years (BLACK et al., 2003).
The United Nations Children's Fund (UNICEF) prepares a ranking of countries
according to the number of children under five years who die per 1,000 born in the
year, called Under five Mortality Rate - U5MR. In other words, it measures the
probability that a child has to die before its fifth birthday. This indicator is a "smaller is
better." In the list of 2007, Brazil has a U5MR of 22 children, which puts it in an
intermediate position, the 107th. This situation is worse than Peru (with 20 children of
U5MR), Colombia (20), Argentina (16) and Uruguay (14) (UNICEF, 2009), all these
countries belong to Latin America.
One possible cause for this result is the low percentage of the population that is
served by the essential services of sanitation: according to information from the
National Sanitation held in 2008, the country has only 44% of households with access
to the general network sewage and has 78.6% of households with access to the main
water supply, which represents more than 12 million households without access
(BRAZIL, 2008).
Furthermore, according to this research, in 2008, as regards the quality of water, 18%
of the Brazilian municipalities distributed water without any treatment (flocculation,
settling, filtration and disinfection) in which region that had the largest number of
municipalities in this situation was the North (20.8%) followed by the Northeast
(7.9%). In 23.4% of the municipalities water rationing occurred, and the Northeast
region with the highest percentage of water rationed, with 40.5% of its municipalities
in rationing.
Regarding to efficiency, on average, 40% of water injected into the network is lost -
are not billed (SNIS, 2008). This indicator reflects the number of leaks occurring in the
country, the low rate of micro-measurement and underpricing. It is a sign of high
inefficiency in the sector (WHO, UNICEF, 2000 p 25), which may be due to the
prevailing institutional structure, the federal conflicts and cross-subsidy.
Thus, the fact is that the coverage index of the national SWSSC is far from desired.
However, this is not the only problem in the sector: the quality of provision of this
service is another problem because it has not proven satisfactory, since the percentage
of municipalities that do not treat the water supplied is considerable or even those
who work under the regime of rationing - and that the loss rate of billing is high. This
makes clear that the sector problem is not only the deficit of the service, but also in
efficient management of sanitation. Therefore, this level of efficiency needs to be
measured, so that public agents can interfere with the sector to ensure that resources
are used in the best possible way and more lives can be saved.
The aim of this study is to measure how effective are the Brazilian state governments
in the management of sanitation to improve the population welfare with the reduction
of child mortality for children under five years old. The methodology chosen for this
was the DEA.
Proceedings of the 10
th
International Conference on DEA Brazil2012 108
Methodology
In Brazil, the value of the U5MR is high and uneven across geographic regions. In
2007, 24 of every 1,000 children born, died (DATASUS, 2007). This indicator is more
pronounced in the poorest regions of the country, North and Northeast. It is a fact that
this rate has decreased over the years - mainly in the 2000s as is shown in Graph 1.
However, this inter-regional inequality is not necessarily a management issue, it may
exist due to scarce resources, given that the same regions are the ones with the worst
indicators of coverage for water supply networks (WS) and Coverage by and sewage
collection networks (SC).
Brazilian states, as making-decision units (DMU), use the existing infrastructure to
combat the mortality of children under 5 years. The mechanism by which these
resources are managed can be described by a production function, even if unknown.
However, the product of this function is, by definition, non-decreasing in relation to
inputs. Thus, the U5MR will not be used as the output, but as the number of children
who survive (U5SR), defined by equation 1.
U5SR = 1.000 U5MR. (01)
Once the indicator U5SR defined as output, define the possible inputs to remaining
possible inputs. Sanitation is one of these because there is a significant association
between the quality of it and the product. It is composed of four elements: (1) water
supply, (2) wastewater collection and treatment, (3) solid waste management and (4)
storm drainage. WS and SC indicators will be inputs of the production function, since
they account for items 1 and 2. For items 3 and 4, Degree of Urbanization (DU) will be
the used indicator as a proxy for them, since in urban areas is expected that these
services are performed.
The aim of the work is to analyze the efficiency of sanitation, however, other
indicators may be important for this analysis, since the product is an indicator that
involves economic and social factors. Thus, two possible inputs are the number of
doctors by a group of 1,000 inhabitants (DBI), as a proxy for hospital infrastructure,
since the reduction of infant mortality depends on the medical monitoring in the early
days of life and even vaccinations and medicines, and Gross Domestic Product per
capita (GDPPC), because they believe that wealthier states may use more healthcare
resources and the richest people can have access to better conditions for their
children.
To determine the importance of each variable, an analysis of correlation between
them was performed, which is shown in Table 1. From the table, there is a positive
association between the indicator U5SR and the other variables.
Proceedings of the 10
th
International Conference on DEA Brazil2012 109
Graph 1 Mortality rate evolution in less than 5 years old.
Source: elaborated by the author by means of data extracted from DATASUS (2007)
There is a strong correlation between the indicators DU and DBI. This perhaps be due
to the more urbanized regions hold a greater attractive power, and therefore it is
natural that there is a greater concentration of physicians in these areas. Also there is a
strong correlation between DBI and GDPPC, which may occur for the same reason:
the richest regions can attract more doctors, both in the private or public
On the other hand, according to UNICEF (2010, p. 14), there is a significant
relationship between GU and underweight, so that, in developing countries, there is a
concentration twice higher for underweight children in rural areas compared to urban
areas. The underweight is directly linked to four factors: food deficit in terms of
quality and quantity; sanitation and inadequate health services, lack of care and
feeding practices.
It is also observed, from the table, that the WS and U5SR correlation is low, despite
being significant. This may indicate that the coverage of water supply systems has no
significant effect on infant mortality, or that these systems are operating inefficiently in
many states, which possibly reduces the positive effect. The second option is more
likely, and hence the indicator will be remained in the model.
The main object of study is the efficiency of sanitation on health promotion, so that
the model used in the research will be called Model 1 and it will have as inputs WS,
SC and DU. The method used was the BCC. As a guideline for the optimization, it was
considered a maximization of the product, given the levels of available inputs.
Regio Norte; 2000;
33,35
Regio Norte; 2001;
32,17
Regio Norte; 2002;
30,97
Regio Norte; 2003;
29,74
Regio Norte; 2004;
28,59
Regio Norte; 2005;
27,64
Regio Norte; 2006;
26,95
Regio Norte; 2007;
26,32
Regio Nordeste;
2000; 48,81 Regio Nordeste;
2001; 45,56
Regio Nordeste;
2002; 43,28
Regio Nordeste;
2003; 40,97
Regio Nordeste;
2004; 38,87
Regio Nordeste;
2005; 37,3
Regio Nordeste;
2006; 36,19
Regio Nordeste;
2007; 35,2
Regio Sudeste;
2000; 22,12
Regio Sudeste;
2001; 21,16
Regio Sudeste;
2002; 20,2
Regio Sudeste;
2003; 19,89
Regio Sudeste;
2004; 18,92
Regio Sudeste;
2005; 17,92
Regio Sudeste;
2006; 17,69
Regio Sudeste;
2007; 17,08
Regio Sul; 2000;
19,93
Regio Sul; 2001;
19,46
Regio Sul; 2002;
18,65
Regio Sul; 2003;
18,82
Regio Sul; 2004;
17,53
Regio Sul; 2005;
16,1
Regio Sul; 2006;
15,84
Regio Sul; 2007;
15,11
Regio Centro-
Oeste; 2000; 25,12
Regio Centro-
Oeste; 2001; 24,49
Regio Centro-
Oeste; 2002; 23,24
Regio Centro-
Oeste; 2003; 22,56
Regio Centro-
Oeste; 2004; 22,18
Regio Centro-
Oeste; 2005; 21,23
Regio Centro-
Oeste; 2006; 20,54
Regio Centro-
Oeste; 2007; 20,18
Brasil; 2000; 31,99
Brasil; 2001; 30,56
Brasil; 2002; 29,13
Brasil; 2003; 28,09
Brasil; 2004; 26,62
Brasil; 2005; 25,36
Brasil; 2006; 24,77
Brasil; 2007; 24,07
Regio Norte
Regio Nordeste
Regio Sudeste
Regio Sul
Regio Centro-Oeste
Brasil
U5MR
Proceedings of the 10
th
International Conference on DEA Brazil2012 110
Table 1 Analysis of correlation between variable
U5SR WS SC DU DBI GDPPC
SOB5 1.0000
CA 0.3911 1.0000
CE 0.5667 0.5656 1.0000
GU 0.7156 0.6527 0.4622 1.0000
MPH 0.6121 0.6443 0.6652 0.7563 1.0000
PIBPC 0.7228 0.4882 0.6009 0.6961 0.8607 1.0000
Source: elaborated by the author by means of the softwareEviews 7.0.
Results and discussion
The result of linear programming for the chosen model is shown in Table 3, in which
the Local Technical Efficiency is presented by the BCC.
Table 3 Local Technical Efficiency of the states in management in sanitation.
Rank State BCC (%) U5SR Rank State BCC (%) U5SR
1 DF 100 986.69 11 AP 99.43 975.38
1 RO 100 975.07 12 MG 99.41 979.28
1 PR 100 984.49 13 AM 99.31 974.08
1 SC 100 985.43 14 AC 99.02 966.51
1 MT 100 977.62 15 CE 98.97 969.17
2 RS 99.98 985 16 PI 98.97 966.49
3 ES 99.89 983.22 17 BA 98.7 965.62
4 GO 99.87 979.12 18 MA 98.68 965.25
5 MS 99.85 976.72 19 SE 98.58 965.16
6 SP 99.84 984.84 20 RN 98.52 965.62
7 RJ 99.67 982.85 21 PE 98.41 965.2
8 RR 99.59 979.64 22 PB 98.25 963.03
9 PR 99.57 973.32 23 AL 97.34 950.79
10 TO 99.46 973.06
Source: elaborated by the author
Proceedings of the 10
th
International Conference on DEA Brazil2012 111
Thus, one can assume that there is a clear need for improvement in the quality of
sanitation services in the northeastern states, for they have not made good use of
existing infrastructure. Possibly, there must be elements that affect efficiency, but were
not considered in the model. The DBI, which is a proxy for hospital infrastructure and
was neglected in Model 1, may be affecting the efficiency of the state. Similarly, the
GDPPC also exogenously may be affecting the efficiency of states.
One factor to be considered is that water availability in the Northeast is far from a
considerable extent in other regions of Brazil. This element can also influence the
efficiency of the state to providing the essential elements for reducing infant mortality.
Years of studies for over 25 years - which, as a proxy, can be interpreted as the level
of knowledge of health education and knowledge of rights and duties - have a great
inequality between geographic regions, so that the North and Northeast are below on
average.
For an analysis of the elements that are possibly impacting the efficiency of the
States, it was considered the linear regression model, which has TE (technical
efficiency) as the dependent variable, and as independent variables, the average of
years of studies for people over 25 years of age (ST), obtained from the IPEA; the
relation between the municipalities population under the rule of state and the total
population of the state (CONT), with the information on the population obtained in
the SNIS (2007) and IBGE, the Population Density (PD), obtained from the IBGE; Per
Capita Water availability (PCWA) (m3/inhabitant.ano) obtained in Lima (2001), and
GDPpc. The model is shown in equation 4.
= +
1
+
++
2
+
+
3
+
+
4
+
+
5
+
+
6
+
(04)
The regression result is shown in Table 4, in which it is possible to verify that the
indicator ST had no significant effect on the model or on the indicator PCWA. Thus, it
cannot join the issues of water availability the inefficiency model nor the education of
the population.
Table 4 Linear Regression of TE in relation to the socio-economic indicators
Indicator Coefficient Test-t p-value
GCPPC 0.0001 2.9580 0.0075
DBI 0.7105 2.1013 0.0479
PD -0.0079 -4.2991 0.0003
PCWA 0.0000 0.4214 0.6777
ST 0.0051 0.1298 0.8980
C 97.9028 347.2115 0.0000
Source: elaborated by the author with the use of Eviews 7.0 / R
2
adjusted= 0,6
It is noteworthy, however, that the PCWA used here may not measure the situation
well, because the water shortage problem is a micro-regional one, and aggregating by
states, this information may not be well captured by the indicator. The attention
Proceedings of the 10
th
International Conference on DEA Brazil2012 112
given to health can be important for the efficiency of sanitation systems, since the DBI
indicator shows a positive effect on the dependent variable, which indicates that not
enough investment in sanitation, but it is also necessary to improve the hospital
infrastructure so that the sanitarian policies have a higher efficiency.
The indicator population density showed a negative relation with efficiency. This
information can be an indication that a concentration of population in excess can
hinder the carrying out of the work of health teams, impeding the company of
working optimally. The GDPpc has a positive effect on efficiency, which may point to
the richest states get greater efficiency in the management of sanitation. One
possibility for this is that a richer state may involve mitigation measures for sanitation
that were not captured in the model, or a better buying power allows people to
protect themselves against the inefficiency of sanitation.
Conclusion
In this study, we analyzed the technical efficiency of the state governments in
administering the sewerage, which is directly associated with reduced infant mortality.
For this analysis, we used the DEA methodology oriented to the maximization of the
product. As a result, states with 100% of technical efficiency are divided into three
geographic regions of the country, in a total of five: two in the South, one North and
two in the Midwest. In the North, states have not been efficient, but efficiency
achieved scores above the northeastern states, even having an infrastructure below to
these, either through coverage of basic sanitation, by number of physicians or by per
capita income. This result clearly shows the inefficiency of the Northwest states, which
make them to have the highest U5MR.
Thus, we perceive the need for quality improvement in the provision of sanitation
services, for its whole service whole is not limited to providing water and sewage
services to citizens. A better control is essential over the quality of this offer, since
when comparing the effectiveness of services between the states, it is noted that there
is an inequality, especially among geographic regions, which further contributes to the
social inequality in the country.
References
BLACK, R. E.; MORRIS, S. S.; BRYCE, J (2003). Where and Why are 10 million children
dying every year? The Lancet. V. 361,p. 2.226-2.234.
BRASIL (2008). Ministrio das Cidades. Pesquisa Nacional de Saneamento Bsico,
Braslia, DF.
DEPARTAMENTO DE INFORMTICA DO SUS DATASUS (2008). Informaes de
sade epidemiolgica e de morbidade causa por local de Residncia. Disponvel em:
www.datasus.gov.br.
LIMA, J. E. F. W. (2001) Recursos hdricos no Brasil e no Mundo. Ministrio da
Agricultura Pecuria e abastecimento. Planaltina: Embrapa Cerrados.
Proceedings of the 10
th
International Conference on DEA Brazil2012 113
RIVERA, D. (1996). Private sector participation in the water supply and wastewater
sector, lessons for six developing countries. Word Bank, Washington, DC.
SISTEMA NACIONAL DE INFORMAES EM SANEAMENTO SNIS (2008).
Diagnstico dos servios de gua e esgoto. Braslia.
TUROLLA, F. A. (2002). Poltica de saneamento: avanos recentes e opes futuras de
polticas pblicas. Braslia. Ipea.
UNITED NATIONS CHIDRENS FUND UNICEF (2010). Progress for children:
Achieving the MDGs with equity. UNICEF, New York.
UNICEF (2009).The States of The Worlds Children 2009. UNICEF, New York.
WORLD HEALTH ORGANIZATION WHO (2010).water global annual assessment of
sanitation and drinking-water (GLAAS) 2010: targeting resources for
better results. WHO Library, Switzerland.
WORLD HEALTH ORGANIZATION WHO (2009). Global health risks: mortality and
burden of disease attributable to selected major risks. WHO Library, Switzerland.
WHO; UNICEF (2000). Global Water supply and sanitation assessment 2000 report.
Switzerland.
Proceedings of the 10
th
International Conference on DEA Brazil2012 114
16. Efficiency of Three Outliers
Detection Tests on Non-
Parametric Frontiers Methods
Victor Maia Senna Delgado
Al. Accias, 31275-150, Belo Horizonte, MG, Brazil - FJP, victor.maia@fjp.mg.gov.br (corresponding
author)
Igor Viveiros Melo Souza
R. Catete, 35420-970, Ouro Preto, MG, Brazil - UFOP, igorviveiros@gmail.com
Abstract
One of the most important problems concerning non-parametric frontiers is the
detection of outliers, influential information that distorts statistical analysis. This topic
has been present for a long time in the literature of non-parametric statistics. Wilson
(1995) used the test proposed by Andrews and Pregibon (1978) to apply one of the
first empirical statistical tests to detect outliers on convex sets of non-parametric
frontiers. Since then, a series of proposals for the detection and treatment of outliers
has emerged in the field of non-parametric frontiers. The purpose of this research is to
highlight three methods of outlier detection: Wilson (1995), Simar (2003), Sousa and
Stosic (2005), and also answer which one is more efficient in statistical terms, lower
bias and variance.
Keywords: Outliers Detection Methods; Non-Parametric Frontiers; Data Envelopment
Analisys; mask effect; Monte-Carlo simulation.
Introduction
One of the most important problems concerning non-parametric frontiers is to control
its borders to outliers, influential information that distorts statistical analysis. Any
Outlier Detection Method (ODM) must help the researcher to isolate wheat from the
weeds, and because most of them grow together, many times, its very hard to
separate which one observation is not part of a particular bunch of data, is necessary
a queer eye-detector to do it right with no help of detection procedures.
The ODMs has been present for a long time in the non-parametric statistics literature.
Wilson (1995) used the test proposed by Andrews and Pregibon (1978) to apply one
of the first empirical statistical tests to detect outliers in convex sets of non-parametric
frontiers.
Since then, a series of proposals for the detection and treatment of outliers has
emerged in the field of non-parametric frontiers. The purpose of this research is to
highlight three methods of outlier detection: the already mentioned Wilson (1995),
and also Simar (2003) and Sousa and Stosic (2005). The purpose is to answer which
one is more efficient in statistical terms, lower bias and variance.
Proceedings of the 10
th
International Conference on DEA Brazil2012 115
In order to make comparisons between different methods, we used the same Data
Generator Process (DGP) to construct the frontier, and the outliers were kept fixed,
they are the same between the three methods. The methods were compared with
optimal threshold outlier detection value for each. The first results show advantages of
Wilson (1995) and Sousa-Stosic (2005) to obtain large single outliers, but with small
efficacy to detect outliers altogether, Sousa-Stosic has very large variance at some
points. Simar (2003) is the more efficient test to detect outliers altogether and to
minimal variance, but with small efficacy to detect particular individual outliers.
Methods
Three outliers detection methods (ODM) applied to non-parametric frontiers will be
studied: the AP-Wilson statistic, named APW for short; the statistic from Simar Test
(ST) developed by the approach of Cazals, Florens and Simar (2002) and Simar (2003).
And the statistics of Sousa-Stosic (SS) developed by Sousa and Stosic (2005). These
three methods have considerably creative and different approaches to the problem of
detecting outliers.
These three statistics are:
APW: | | | | ) 1 ( * *' det * *' det
) (
X X X X
i
L S
i
L S
R
=
ST:
) 2 ( ) , (
~ 1
) , (
1
0 0 0 0
=
=
B
b
b
m
b
m
y x
B
y x
SS:
( )
) 3 (
1
; 1
2
,
=
=
n
n
i s s
s i s
i
Where X* is the [XY] matrix of information, with X representing the set of inputs and
Y representing the set of outputs. The dataset element, or decision maker units
(DMUs), is S = {1, 2, , n}, which i represents any particular observation from S. L is
a subset of S (L
*exp(-U) (4)
Where Y and X are outputs and inputs, here and in all paper, these are vectors are
(n1) dimension. The = 0.5 and U is uniform on (0,1), so, exp(-U) have mean =
, and the output-efficiency mean is 32. The OGP is also here arbitrary and
somewhat similar to Simars, but apart from paper of Simar, we chose 90 observations
of process above and 10 outliers, summing up to 100 DMUs and a proportion of 0.1
outliers in database (Table 1 brings the outliers values).
Table 1 - Values of Outlier Generating Process
Outlier id. Y X
91 0.004 0.100
92 0.025 0.190
93 0.035 0.240
94 0.100 0.700
95 0.500 0.900
96 0.700 1.000
97 0.750 1.050
98 1.000 1.000
99 1.200 1.000
100 1.100 1.100
For now on, we made means to compare different ODMs, called statistic *. This
statistic is based on simple proportion of all outliers detected by the method (
i
) over
all true outliers ():
) 5 ( 1 *
1 1
|
|
.
|
\
|
|
|
.
|
\
|
=
= =
n
i
i i
n
i
i
T
n
The term in first parenthesis is useful for one penalty to over identification. On the
limit, a weak threshold could identify all true outliers identifying all observations as
outliers. To avoid this, the penalty was suggested, if one ODM detects all n
Proceedings of the 10
th
International Conference on DEA Brazil2012 117
observations as outliers, the * will become zero. Another change from the previous
equation is the Identification vector (T
i
) that enables the term in second parenthesis
capture only true outliers.
2
With the previous statistic is possible to compare the three different ODMs, but to be
fair with all three methods, is necessary to find out the best possible threshold value to
each one, the value that maximizes the identification with no penalty loss.
To identify the maximum threshold, all the other parameters must be fixed. It was
used all data, with one particular realization of equation 6 for all 90 non-outliers
points, and 10 fixed outliers. If we had continuity and second order condition
satisfied, to discover the maximum we need the derivative of * with respect the
threshold value (): *() = 0.
Table 2 presents the values of threshold and the maximum value of * for each
method, the first is computational statistic of maximum, the second is obtained by
optimize function of R language.
Table 2 - Optimum values for the threshold of each Method.
Computational Optimize
APW 0.965 0.966
ST 1.085 1.095
SS 0.020 0.121
To compare three different methods with only one database is insufficient to build
confidence intervals and pronounce about reliability of indicators. One idea to
compare empirically three different methods is Monte Carlo simulations. The
simulation was composed by 2000 replications of equation (4) with ninety
observations each (DGP) and the ten outliers were kept fixed.
In the first procedure, the outliers were added one by one until they sum up 10. This
procedure identifies the method capacity of identify all outliers and to control mask
effects. The routine is described:
Run the equation (6) and obtain 90 observations for each.
Append to each replication the first outlier kept fixed (from table 1).
Replicate step 1 and 2 2000 times.
Save the mean and sampled standard deviation () of the * statistic.
Obtain the confidence interval (CI) with 5% for (error type I and supposing
normal distribution).
2
If the ODM is really unbiased the term in second parenthesis will be 1.00, and with n = 100, the first
penalty term should not trespass 0.9, therefore, the maximum * is 0.9 with ten outliers. For only one
outlier this maximum is 0.99.
Proceedings of the 10
th
International Conference on DEA Brazil2012 118
Do steps above more 10 times, adding to step 2: first up to second, first up to
third, and so on until the tenth outlier.
In the second procedure, outliers were added one (and only one) by time until all ten
were analyzed. This procedure identifies the method capacity to point one outlier per
time: if that was the only outlier, how the ODM should be doing? The routine is very
similar to the previous described, with exception of sixth step, which adds one outlier
per time.
Results and discussions
As results for first procedure we have that the SS method is the best result to this first
Monte Carlo procedure, mean of 0.883 for first outlier (91
th
DMU) and 0.869 to second
(92
th
). But for all three first outliers the confidence interval of SS method is very large,
maybe because its leverage properties of being in frontier, this introduces greater
variance because more sparsely data in frontier region.
The Simars ODM greatly improves from the fourth outlier beyond, its the better
method to identify outliers from 94 to 100, with relativity small variance and mean
over 0.8 for statistic (*). The APW method does not captures the first three outliers.
Despite that, the APW method improves at each outlier added, with very small, but
increasing, variance from 94
th
to 97
th
. Although the maximum value at 0.549 is far
below the ST at final outliers, with relatively same variance.
As discussed by Wilson (1995), the mask effect should be important here for APW and
SS methods. Table 3 shows the results for three ODMs pointing the central statistic
and standard deviation with confidence interval of = 5% (1.96).
In Table 4, the capacity of identify each outlier per time is investigated. Remembering
that the maximum * value is .99 to next simulation. The capacity of SS method for
identify the seven first outliers (from 91 to 97) is very high, dropping drastically for
98
th
and 99
th
. These two observations should be not declared as outliers by the SS
method. Again, the confidence interval for SS method is very abroad to 91, 92, 93, 98
and 99.
The APW shows better results identifying the outliers one per time, with the exception
of the first three (91, 92, 93) which are note declared outliers by this method. Our
hypothesis is that the identification one per time excludes the mask effect that
prejudices APW and SS ODMs. Meanwhile, ST do not so well to identify one per time,
probably because there is over identification of false outliers, and it increases the
penalty for ST method (maybe one different threshold should be applied to this part).
Next section investigates this possibility in detail.
Proceedings of the 10
th
International Conference on DEA Brazil2012 119
Table 3 - Results for Outliers added one by one until sum up ten
Ap.wilson Simar test Sousa-Stosic
Outliers * CI_min CI_max * CI_min CI_max * CI_min CI_max
91 0.000 0.000 0.000 0.000 0.728 0.037 0.656 0.800 0.883 0.268 0.358 1.407
92 0.000 0.000 0.000 0.000 0.738 0.036 0.667 0.810 0.869 0.197 0.483 1.254
93 0.000 0.000 0.000 0.000 0.738 0.035 0.669 0.808 0.583 0.123 0.342 0.824
94 0.206 0.006 0.194 0.218 0.845 0.027 0.791 0.898 0.485 0.003 0.480 0.490
95 0.335 0.010 0.315 0.355 0.888 0.023 0.842 0.934 0.579 0.003 0.573 0.585
96 0.423 0.012 0.399 0.447 0.903 0.018 0.868 0.939 0.484 0.001 0.482 0.486
97 0.485 0.012 0.461 0.509 0.903 0.015 0.873 0.933 0.415 0.001 0.413 0.417
98 0.530 0.013 0.505 0.555 0.893 0.016 0.862 0.924 0.363 0.001 0.362 0.365
99 0.564 0.015 0.535 0.593 0.883 0.015 0.853 0.913 0.323 0.001 0.322 0.324
100 0.589 0.019 0.552 0.626 0.875 0.015 0.846 0.904 0.325 0.045 0.236 0.414
Table 4 - Results for Outliers added one per time
Ap.wilson Simar test Sousa-Stosic
Outliers * CI_min CI_max * CI_min CI_max * CI_min CI_max
91 0.000 0.000 0.000 0.000 0.726 0.037 0.653 0.799 0.899 0.243 0.422 1.375
92 0.000 0.000 0.000 0.000 0.731 0.037 0.659 0.803 0.954 0.097 0.765 1.144
93 0.000 0.000 0.000 0.000 0.737 0.038 0.661 0.812 0.966 0.011 0.945 0.987
94 0.814 0.026 0.764 0.864 0.831 0.033 0.766 0.895 0.972 0.007 0.959 0.986
95 0.805 0.027 0.753 0.858 0.781 0.034 0.714 0.849 0.969 0.010 0.949 0.990
96 0.804 0.027 0.752 0.857 0.760 0.034 0.692 0.827 0.968 0.010 0.950 0.987
97 0.805 0.026 0.755 0.856 0.758 0.036 0.687 0.829 0.969 0.010 0.949 0.988
98 0.792 0.026 0.741 0.843 0.713 0.038 0.638 0.788 0.738 0.408 -0.062 1.537
99 0.790 0.026 0.740 0.840 0.714 0.040 0.636 0.791 0.277 0.435 -0.576 1.131
100 0.795 0.025 0.746 0.844 0.713 0.039 0.637 0.788 0.954 0.101 0.755 1.152
Conclusions
Its hard to be conclusive to answer the question Which ODM is better? There is
some indication that Simars method (2003) is the better one for detect bunch of
outliers, and do it with great confidence, because small interval and small
misidentification. With correct threshold, the SS is the better one to detect outliers 91,
92 and 93 in two Monte-Carlo procedures and the more powerful method to detect
Proceedings of the 10
th
International Conference on DEA Brazil2012 120
one outlier per time. With exception of observations 98 and 99, that SS method
virtually does not detect.
As SS, the APW method does better for identify outliers one per time. With the
advantage of have smaller variance than the others for 91, 92 and 93 and for 98, 99
and 100. The APW appears to be the better method to see super inefficient outliers,
points very internal in production set. This also appears in ROC analysis and suggests
powerful performance for large outliers in large data. Due to the mask effect, both
methods (APW and SS) did not dispense visual inspection of researcher, which
becomes a limitation for large databases.
One additional future advance for this research is to investigate how these three
methods behave for more dimensions (more than 1-output and 1-input framework)
and for other technology process, increasing and mixed returns. Its also an extra topic
to apply this analysis to other well-know databases that could also lead to interesting
results.
And, finally, it seems, for us, that a promising method can blend the best qualities of
each ODM discussed here. One possibility of doing this is to take the maximum
expectancy of each method in its best extend of outlier detection.
References
Andrews, D., D. Pregibon (1978) Findings the outliers that Matter, Journal of the Royal
Statistical Society, series B, 40 (1): 85-93.
Cazals, C., J. Florens, L. Simar (2002) Nonparametric frontier estimation: a robust
approach, Journal of Econometrics106 (1): 1-25.
Simar, L. (2003) Detecting outliers in frontier models: a simple approach, Journal of
Productivity Analisys, 20 (3): 391-424.
Sousa, M.C.S., B. Stosic (2005) Technical efficiency of the Brazilian municipalities:
correcting nonparametric frontier measurements for outliers, Journal of Productivity
Analysis 24 (2): 157-181.
Wilson, P.W. (1995) Detecting Influential Observations in Data Envelopment Analysis,
Journal of Productivity Analysis 6 (1): 27-45.
Acknowledgements
The authors thank the support of Centro de Estudos de Polticas de Pblicas of
Fundao Joo Pinheiro (CEPP/FJP) for technical and finance support to the
presentation in the 10
th
DEA international congress. Like many others Operational
Research Professionals, we also regret the passing of Professor W.W. Cooper, one of
those who made DEA meeting possible. All possible remaining flaws are responsible
exclusively by the authors and not their institutions and collaborators.
Proceedings of the 10
th
International Conference on DEA Brazil2012 121
17. Evaluation of the
Benchmarking Model Proposed
by the Brazilian Electricity
Regulator for Energy
Distribution Companies: The
Case of Tariff Revision
Giordano Bruno Braz de Pinho Matos
CEPEAD, UFMG, Brazil, giordano.matos@cemig.com.br
Marcelo Azevedo Costa
UFMG, Brazil, azevedo@est.ufmg.br
Ana Lcia Miranda Lopes
CEPEAD, UFMG, Brazil, analopes.ufmg@gmail.com
Roberta de Cssia Macedo
CEPEAD, UFMG, Brazil, roberta.c.macedo@gmail.com
Abstract
In 2011 the Brazilian Electricity Regulator - ANEEL implemented a benchmarking
model to evaluate the operational efficiency of the electrical power utilities in Brazil.
This model is based on two benchmarking methods widely applied by other
regulators: Data Envelopment Analysis - DEA and Corrected Ordinary Least Square -
COLS. The aim of this paper is to identify the cause of discrepancies between the
results obtained by applying DEA and COLS models and also to discuss the use of a
non- decreasing returns to scale(NDRS) DEA model by the regulator. It is shown that
the differences between the parametric (COLS) and non-parametric (DEA) models are
mainly due to the unsuitability of the Cobb-Douglas model as a cost function, by the
effect of the sample size which shifts the COLS's efficiency scores towards smaller
values, and because of the DEA model type NDRS.
Introduction
In September 10, 2010, the Brazilian National Electric Energy Agency - ANEEL started
to debate with the society about the rules and methodologies for defining the
revenues of electricity distribution utilities in the 3rd Periodic Tariff Review Cycle
(3PTRC), through the public hearing 040/2010 (AP040). By means of the technical
note 265/2010 the regulator proposed a full review on the model which calculates
regulatory operational costs. The definition of efficient operating costs is a central
point in the incentive regulation, because it has been chosen for regulating natural
Proceedings of the 10
th
International Conference on DEA Brazil2012 122
monopolies in the Brazilian electricity sector after their privatization in the nineties.
The incentive regulation requires the definition of a level of revenue or rate for a fixed
period of time, which is defined in a formal contract. Given the level of rates or
revenues defined by the regulator, the companies are encourage to reduce their costs
to lower levels in order to achieve higher financial returns. After the concession
period defined in the agreement contract, the costs of the companies are revised, i.e.,
the regulator defines new levels which are considered more efficient. In this case,
these new efficiency levels are proposed by the regulator for the benefit of
consumers. Therefore, it is crucial for both the regulated company as for consumers a
consistent methodology for the definition of operational costs.
The Data Envelopment Analysis (DEA) benchmarking method was first considered by
ANEEL because it has been successfully applied by other regulatory agencies from
Austria, Britain, Belgium, Finland, Netherlands, among others. Therefore, it is of
interest of ANEEL to replace previous benchmarking models by a concise and robust
approach, like DEA. By applying DEA, the operating cost of each company is
compared to the costs/results of the remaining companies and, by means of a linear
system of equations, an efficiency frontier is calculated. The final results of this model
are efficiency scores that indicate the efficiency of each company in transforming
inputs (operational cost) in outputs (electricity consumption, number of customers
and network extension), when compared with similar companies.
Previous studies apply DEA to assess efficiency and productivity change of electrical
power distribution companies (e.g. Arocena, 2008; Bagdadioglu, Price, and Weyman-
Jones, 1995; Hjalmarsson and Veiderpass, 2002; Kumbhakar and Hjalmarsson, 1998;
Pacudan and Guzman, 2002). Estellita et al. (2007) presents a case study of electrical
power distribution companies in which two different DEA models are proposed. The
first modelassess efficiency using previous information from regulators and the second
model assess efficiency using previous information from companies. Thegrell, Bogetoft
and Tind (2005) proposed a multi-period and multi-output model based on costs
related to productivity analysis and a theoretical model of the company (power
distribution companies). Zhou, Ang, and Poh (2008) present results of a survey which
applies DEA in the energy sector and environment.
Based on previous literature review, the aim of this paper is to evaluate the
benchmarking model proposed by ANEEL and to discuss major inconsistencies of the
methodology. Specifically, we aim at identifying the causes of the discrepancy
between DEA and COLS efficiency scores. We also evaluate the use of non-decreasing
returns to scale in the DEA model presented by ANEEL
Methodology
The most commonly used benchmarking frontier techniques include Data
Envelopment Analysis (DEA), Corrected Ordinary Least Squares (COLS) and Stochastic
Frontier Analysis (SFA) (Haney and Pollitt 2009). DEA is a non-parametric method
which requires assumptions of increasing and concave production function (Banker,
Charnes and Cooper 1984). It provides a very flexible function, robust to errors of
misspecification. Both SFA and COLS are parametric methods which require the
specification of a functional mathematical equation, such as Cobb-Douglas or translog
Proceedings of the 10
th
International Conference on DEA Brazil2012 123
functions. The main difference between SFA and COLS is that COLS implies that
deviations from the frontier are due to inefficiency only, while SFA considers that
deviations from the efficiency frontier are due to technical inefficiencies and to a
random noise. Although SFA may have a theoretical advantage over COLS model, it is
hard to fit SFA in small samples (CEPA, 2003). Recently, advances in DEA models
allowed deviations from the efficiency frontier to be related to both, inefficiency and
random noise, as in stochastic frontier (BANKER and NATAJARAN, 2008).
The Data Envelopment Analysis - DEA methodology was first introduced by Charnes,
Cooper and Rhodes in 1978 (CHARNES, COOPER and RHODES, 1978) and then
extended by Banker, Charnes and Cooper in 1984 (BANKER, CHARNES and COOPER,
1984). This methodology has been widely used for estimating technical efficiencies of
Decisions Making Units (DMU). DEA is a mathematical programming method that
provides a single measure of efficiency. It is calculated with the use of multiple inputs
and multiple outputs information and results in a frontier which represents the best
practice. From this best efficiency frontier, the relative efficiency of DMUs is
calculated. For each DMU, DEA presents an efficiency score, typically ranging
between zero and 1, which indicates inefficiencies. Furthermore, the DEA efficiency
frontier can be used as a guideline so that inefficient companies can improve their
inputs and outputs and reach the efficiency frontier.
The following DEA models calculate the efficiency score of each DMU j by imposing
different returns to scale. Let be the efficiency score of the reference DMU j. Let y
rk
be the variable that represents the output r (r = 1, ..., R) and x
ik
be the input variable
(i = 1, ...I) for each DMU k. Let be the weights of the observations used as
benchmarking. Model (1) is the CCR model (CHARNES, COOPER and RHODES, 1978),
which assumes constant returns to scale (CRS). Model (2) assumes variable returns to
scale (VRS or BCC) (BANKER, CHARNES and COOPER, 1984). Model (3) assumes
non-increasing returns to scale (NIRS), while model (4) assumes non-decreasing
returns to scale (NDRS).
(1)
(2)
1
1
max
. . , 1,..
, 1,... ;
0, 1,..
CRS
j
N
k rk rj
k
N
k ik ij
k
k
S t y y r R
x x i I
k N
=
=
=
= ;
=
=
1
1
1
max
. . , 1,..
, 1,... ;
1, 0, 1,..
VRS
j
N
k rk rj
k
N
k ik ij
k
N
k k
k
S t y y r R
x x i I
k N
=
=
=
=
= ;
=
= =
Proceedings of the 10
th
International Conference on DEA Brazil2012 124
(3)
(4)
ANEEL Proposal
ANEEL proposed, in a first stage of the public hearing (040/2010), two DEA models
organized in two stages. Both models for the first stage used the operational cost as
their input variable and the network extension (km) as the output variable. Model 1
includes the number of customers as the second output variable, while model 2
includes the energy consumption (MWh) as the second output variable. Figure 1
shows the inputs and outputs of each model. Both models assume a non-decreasing
returns to scale (NDRS).
Inputs Outputs
Model 1
Operational Costs
(R$)
Number of Customers
Network Extension (km)
Model 2
Operational Costs
(R$)
Energy Consumption
(MWh)
Network Extension (km)
Figure 1. Proposed model by ANEEL in the first stage of the Public Hearing 040
The data used was from 2003-2010. The electrical distribution companies were split
into two groups. Group A is composed by companies with an annual energy
consumption greater than 1 Terawatt-hour (TWh), whereas group B is composed by
companies with an annual energy consumption below 1 TWh. The DEA method was
applied separately on each group in order to estimate the final efficiency scores.
After to reach the efficient operational cost for each distribution energy company
ANEEL proposed to adjust these scores by environmental variables in a two stage
model.
1
1
1
max
. . , 1,..
, 1,... ;
1, 0, 1,..
NIRS
j
N
k rk rj
k
N
k ik ij
k
N
k k
k
S t y y r R
x x i I
k N
=
=
=
=
= ;
=
=
1
1
1
max
. . , 1,..
, 1,... ;
1, 0, 1,..
NDRS
j
N
k rk rj
k
N
k ik ij
k
N
k k
k
S t y y r R
x x i I
k N
=
=
=
=
= ;
=
=
Proceedings of the 10
th
International Conference on DEA Brazil2012 125
After public contributions, ANEEL proposed one DEA model in the first stage,
aggregating all previous output variables. In addition, the regulator proposed a second
benchmarking model known as Corrected Ordinary Least Squares (COLS), and
presented in Technical Note 101/2011. COLS is a parametric model which fits a
regression model by means of ordinary least squares. In sequence, the regression
model is shifted towards the smallest observed value of operational cost, creating the
lower bound or the efficiency frontier of the operational cost. In this case, the
regression model is the Cobb-Douglas production function.
This method is currently used by a few regulators (Denmark and Great Britain), and it
is known to be more restrictive, i.e., it strongly penalizes companies which are not on
the frontier (Bogetoft & Otto, 2011). To overcome this limitation, ANEEL proposed to
average the DEA and COLS efficiency scores and to use the mean value as the first
stage outcome.
In the final decision(november 2011), the energy consumption (MWh) output variable
was replaced by a weighted energy consumption variable which aggregates high,
medium and lower voltage energy consumptions. The weights were chosen to be
proportional to the amount of consumption in the markets of high, medium, and
lower voltage, of each company.
Figure 2 shows the different models presented by the ANEEL for the estimate of
operational costs in each stage of the public hearing.
Figure 2. Phases and models suggested by the Brazilian regulator
Although the methodology proposed by ANEELs was based on the experience of
leading European regulatory agencies, it was subject to criticisms and suggestions
from the Brazilian community and power distribution companies. One of the major
concerns was to use the DEA model with non-decreasing returns to scale (NDRS) as a
replacement to the most commonly used model, the DEA with variable returns to
scale -VRS (BANKER, 1984). Later, a technical report (BANKER, 2011) was submitted
Method Input Varibles Output Variables
DEA Operational Costs (R$) Network lenght (km), Number Customers
DEA Operational Costs (R$) Network lenght (km), Electricity Consumption
Method Input Varibles Output Variables
DEA Operational Costs (R$) Network l enght (km), Number Customers, El ec. Consumpti on (Mwh)
COLS Operational Costs (R$) Network l enght (km), Number Customers, El ec. Consumpti on (Mwh)
Method Input Varibles Output Variables
DEA Operational Costs (R$) Network l enght (km), Number Customers, Wei ghted Consumpti on (Mwh)
COLS Operational Costs (R$) Network l enght (km), Number Customers, Wei ghted Consumpti on (Mwh)
Technical Note 265/2010
Technical Note 101/2010
Technical Note 294/2010
Proceedings of the 10
th
International Conference on DEA Brazil2012 126
to ANEEL, providing proper evidence that the VRS model is the appropriate model.
Briefly, even if economic theory argues that non-decreasing returns to scale prevails in
situations of natural monopoly, empirical evidence strongly suggests that VRS achieves
better fit. This is because a mathematical model is an abstract representation, and
without the proper choice of production function, outputs and environmental
variables, such model should not hold on strong assumptions, which is the case of
NDRS
A separate analysis of the estimated efficiency scores of DEA and COLS, presented by
ANEEL, shows that the efficiency scores present major inconsistencies. The efficiency
scores using the COLS model are outstandingly smaller than DEA estimates for all
companies, except for one. Figures 3 and 4 show that the differences between the
results of COLS and DEA reach up to 21% in Group A, and 41% in Group B.
Figure 3. Differences en percentage between COLS and DEA scores for Group A.
-1%
-4%
-5%
-8%
-9%
-1%
-5%
-6%
-12%
-11%
-15%
-3%
-2%
-14%
0%
-17%
-21%
-16%
-6%
-5%
-6% -6%
1%
-18%
-9%
-7%
-12%
-2%
-9%
-23%
-18%
-13%
-8%
-3%
2%
Proceedings of the 10
th
International Conference on DEA Brazil2012 127
Figure 4. Differences en percentage between COLS and DEA scores for Group B.
Analysis of inconsistencies of the methodology presented by ANEEL
The COLS model, presented in Technical Note 101/2011, is known to be more
restrictive than other benchmark methodologies, as illustrated in Figure 5. Succinctly,
COLS generates an efficiency frontier which is generally more distant to the data and,
therefore provides smaller efficiency scores for the DMUs.
Figure 5. Comparison of Benchmarking Methods
(SYRJANEN, M., BOGETOFT, P., AGRELL, P. Efficiency benchmarking project B:
Analogous efficiency measurement model based on Stochastic Frontier Analysis, Final
Report, 2006.)
-10%
-19%
-10%
-12%
-16%
-8%
-30%
-13%
-28%
-6%
-11%
-8%
-31%
-12%
-8%
-37%
-11%
-5%
-19%
0%
-4%
-9%
-5%
-9%
-41%
-9%
-7%
-13%
-2%
-4%
-45%
-40%
-35%
-30%
-25%
-20%
-15%
-10%
-5%
0%
Proceedings of the 10
th
International Conference on DEA Brazil2012 128
Analysis of the Cobb-Douglas functional
The unsuitability of the Cobb-Douglas function as a cost function is already known in
the literature. Bogetoft and Otto (2011) present the results of a benchmarking case
study of the energy sector in Germany. The authors were evaluating different
functional forms for both Stochastic Boundary (SFA) and DEA frontiers and concluded
that a Normed Linear function provided better results. The work also concludes that
the Cobb-Douglas function is not a cost function but rather a production function.
We could, of course, have handled this heteroscedasticity problem using a log-
linear specification, but we did not do so to avoid the specifications curvature
problem; the output-isoquants in a log-linear specification curve the opposite
way than do usual output-isoquants. This is not surprising since the log-linear
model corresponds to a Cobb-Douglas model, which is really a production
function and not a cost function (BOGETOFT e OTTO, 2011, p. 31).
Syrjanen, Bogetoft and Agrell (2006), pointed this problem in the report to the Finnish
Regulator[1]: "the CobbDouglas function is not a cost function". After concluding that
the linear function has heteroscedasticity problems that can be solved with the use of
a log-linear function (Cobb-Douglas), the authors argue that this has significant
conceptual problems suggesting not using it in the regulatory cost modeling for
Finnish distributors. Due to conceptual problems related to the log-linear model we
suggest discarding this model"(p. 50).
Figure 6 shows that although in the log scale, the Cobb-Douglas function seems to fit
properly the data (figure in the left side), in the original space it produces a curve
taking an opposite direction (blue curve) than what it should be expected as a cost
function (pink curve). Therefore, the Cobb-Douglas is not appropriate as a cost
function.
Figure 6 . Analysis of Log-linear function as a cost function
SYRJANEN, M., BOGETOFT, P., AGRELL, P. Efficiency benchmarking project B:
Analogous efficiency measurement model based on Stochastic Frontier Analysis,
Final Report, 2006.
Proceedings of the 10
th
International Conference on DEA Brazil2012 129
Furthermore, Cobb-Douglas function has an increasing marginal rate for the outputs
which contradicts economic theory. In fact, based on economin theory, production
functions are required to be convex.
The problem of the sample size
According to ANEEL, the COLS efficieny scores are calculated in three steps. In the
first step, the companies are grouped into two groups: Group A has 29 distribution
companies which presented a total electricity consumption larger (or equal) than 1
TWh in year 2003. Group B has 30 distribution companies which presented a total
electricity consumption smaller than 1 TWh in year 2003. In the second step, a
multiple regression model is fit separately to each group (A and B). The response, or
dependent variable, is the total operational cost of each company, for years 2003 to
2010. The predictors are: (1) total number of consumer units, (2) the total length of
the network, (3) total weighted consumption (for consumers of high, medium and low
voltage), for years 2003 to 2010. The model applies the logarithm (or log) scale which
adjusts the heteroscedasticity within each group. In the third step, for each group, the
estimated regression model is shifted towards the company with the smallest residual.
The efficiency scores are calculated with respect to this new shifted regression model.
In can be showed that the efficiency scores estimated using the model COLS Cobb-
Douglas are extremely sensitive to the sample size. The larger the sample size, the
smaller the efficiency scores of all companies, except the company which achieved
the smallest residual in each group. The company with the smallest residual has an
efficiency score of 1 (100%). By accounting the operational cost over a period of time,
say year 2003 to 2010, the sample size increases. Therefore, the larger the time period,
the larger is the sample size. In this case, there might exist temporal dependencies
which were not accounted in the Cobb-Douglas model. As a consequence of the
sample size, most of the efficiency scores were between 20% and 50% for most
companies. For instance: Boa Vista company (Group B) achieved an efficieny score of
19%, Joo Cesa company (Group B) achieved an efficiency score of 21% and Cemig
company (Group A) achieved and efficiency score of 39%. If those efficiency scores
were estimated for a single year, say 2010, most of the scores would be, on average,
close to 60%, which is incredible higher than current estimates. In brief, with the
increase of the sample sample, the scores are mostly between 20% and 50%, as can be
shown in Figure 7. Figure 7 shows that by shifting the regression model towards the
smallest residual, the larger the sample size the farther is the shift of the regression
model from the data and, as a consequence, the smaller are the estimated efficiency
scores. Therefore, the efficiency scores using the score COLS Cobb-Douglas are
inapropriate for benchmarking. This asymmetric distribution of the efficiency scores,
in the presence of a larger sample size, is known in the literature as the extreme value
distribution, and it is caused primarily by the minimal operator in the COLS procedure.
Proceedings of the 10
th
International Conference on DEA Brazil2012 130
Figure 7. Effect of the minimum operator on the distribution of independent and
identically distributed normal random variables.
Furthermore, by assuming some basic statistical assumptions on the distribution of the
residuals, the estimates of the regression model can be used to generate confidence
intervals for the efficiency scores. Table 1 shows the efficiency score of Cemig
company and its confidence intervals (lower and upper limits) using COLS Cobb-
Douglas model and the sample size adjustment. It worh mentioning that without the
sample size adjusment the efficiency score estimate is 39%.
Table 1: Confidence intervals for the efficiency score of Cemig using the
adjusted multiple regression model
Description Efficiency Score
Estimated Value 0.4519956
Lower Limit 0.3412855
Upper Limit 0.5986191
amostra de
tamanho n
Sample
size n
Proceedings of the 10
th
International Conference on DEA Brazil2012 131
Hypothesis testing of the correlation between DEA and COLS Cobb
Douglas efficiency scores for groups A and B
ANEEL reports that DEA-NDRS and COLS Cobb-Douglas efficiency scores have a
correlation coefficient of 0.94 (94%) (paragraph 68, PG. 13) and therefore, they are
similar.
Figure 8 shows the relationship between the estimated using DEA and COLS for
groups A and B. For Group A the linear correlation coefficient is 0.9539 (95.38%). For
Group B, the linear correlation coefficient is 0.8369 (83.69%). To test the hypothesis
that the DEA and COLS scores are statistically equivalent, it is possible to fit a simple
linear regression model with no intercept for each group of the form:
=
n
i
i i
d u S u
d u
1
,
) ( max
,
, ,
(1)
The optimum value of function F(N, S, d) satisfy four properties: Pareto efficiency,
Invariance with respect to affine transformation, Independence of irrelevant
alternatives, and Symmetry.
In the first model (2), we used the Nash Bargaining problem in order to negotiate
the desired levels of input-output in each stage of Network DEA, as expressed
below. This model (2) is the first phase of our analysis.
3 2 1
max z z z + + . .t s (2.1)
1 1
1
1
P P
Y y
o
(2.2)
) ( ) 1 (
1 2 2
2
2
P P
Y y
o
(2.3)
) ( ) 1 (
2 3 3
3
3
P P
Y y
o
(2.4)
3 4 4
P P
X x
o
(2.5)
1 2 2 1 1
) )( ( z = (2.6)
2 3 3 2 2
) )( ( z = (2.7)
3 4 3 3
) )( ( z =
(2.8)
1 , 1 , 1
3 2 1
= = =
(2.9)
4 3 3 2 2 1 1
, , , (2.10)
. 0 , 0 , 0 , 0
3 2 1
(2.11)
Proceedings of the 10
th
International Conference on DEA Brazil2012 140
In this model, P1 is the set of indicators assigned to Patient Perspective, P2 is the set
of indicators assigned to Internal Processes Perspective, P3 is the set of indicators
assigned to Learning & Growth Perspective, P4 is the set of indicators assigned to
Financial Perspective, y
o
is the vector of output generated by the DMU
o,
x
o
is the vector
of input used by the DMU
o
, Y is the vector of outputs generated by the set of DMU, X
is the vector of inputs used by the set of DMU, is the vector of intensities variables
for the set of DMU, is the scalar coefficient that promotes the maximum
equiproportional reduction of all inputs and } 3 , 2 , 1 { : = i
i
, are scalar coefficients that
promotes the maximum equiproportional increase all outputs. We add the convexity
constraints (2.9) for each stage in order to adopt the variable returns to scale
assumption. From the optimum solution obtained by the first model, we proposed
the second model in order to negotiate the desired levels of input-output from one
stage to another. This is the second phase of our analysis.
i i
i
i i
f
*
:
) ( ) ( max
. .t s
(3.1)
=
=
. , 1
), (
) (
*
4
*
4 4
if
if
f where
(3.2)
1 1
1
1
P P
Y y
o
(3.3)
) ( ) 1 (
1 2 2
2
2
P P
Y y
o
(3.4)
) ( ) 1 (
2 3 3
3
3
P P
Y y
o
(3.5)
3 4 4
P P
X x
o
(3.6)
1 , 1 , 1
3 2 1
= = =
(3.7)
4 3 3 2 2 1 1
, , , (3.8)
. 0 , 0 , 0 , 0
3 2 1
(3.9)
In this model,
*
and
*
i
is the value of optimum solution for the 1
st
model. Note
that if some
*
i
and some
*
do not satisfy
i i
>
*
and
i
<
*
, then the respective
output associate to
*
i
and, the input associate to
*
do not participate as a player in
model (3).
In the third phase, we used the model (4) in order to optimise
i
and that do not
participate as a player in second model (3), in order words, this model (4) optimise
the values of
*
i
and
*
if
i i
=
*
and
i
=
*
in the first model.
+ +
3 2 1
max
. .t s
(4.1)
1 1
1
1
P P
Y y
o
(4.2)
) ( ) 1 (
1 2 2
2
2
P P
Y y
o
(4.3)
) ( ) 1 (
2 3 3
3
3
P P
Y y
o
(4.4)
Proceedings of the 10
th
International Conference on DEA Brazil2012 141
3 4 4
P P
X x
o
(4.5)
1 , 1 , 1
3 2 1
= = =
(4.6)
* *
3 3
*
2 2
*
1 1
, , ,
(4.7)
. 0 , 0 , 0 , 0
3 2 1
(4.8)
In this model (4),
*
and
*
i
are the optimum solution for the 2
nd
model, but it does
not provide information on the efficiency of each individual stage. We calculated the
efficiency of each stage by the multiplier model (5).
. .
min
*
4
*
3 3
*
2 2
*
1 1
* *
3
*
2
3
3
2
2
t s d d d d
v u u y u y u
P
o
P
o
+
+ + + +
(5.1)
1
1
1
1
= d y u
P
o
, 1
2
2
2
= d y u
P
o
(5.2)
1
3
3
3
= d y u
P
o
, 1
4
4
d vx
P
o
(5.3)
0
*
2
2
2
1
1
u Y u Y u
P P
(5.4)
0
*
3
3
3
2
2
u Y u Y u
P P
(5.6)
0 , 0 , 0 , 0
3 2 1
v u u u (5.7)
0 , 0 , 0 , 0
4 3 2 1
d d d d (5.9)
This last model is the dual to the model (4). According to Cook et al. (2010), due to
the usual procedure of adjusting the virtual inputs or virtual outputs by the efficiency
scores obtained from model (5), the model (4) does not necessarily supply
information about frontier projection for intermediate outputs. Thus, we considered
max{y
o
P2
u
2
; y
o
P2
u
*
} and max{y
o
P3
u
3
; y
o
P3
u
*
} as targets for intermediate outputs. We
calculated the efficiency for each stage according to:
*
2
2
2
1
1
3
u y u
y u
e
P
o
P
o
+
= stage 3 (5.1)
*
3
3
3
2
2
2
u y u
y u
e
P
o
P
o
+
= stage 2 (5.2)
* 4
3
3
1
v vx
y u
e
P
o
P
o
+
= stage 1 (5.3)
According to Cook et al. (2010) and Chen et al. (2009), because of extra free-in-sign
variable in the VRS-DEA model becomes difficult the use of geometric mean of the
efficiency scores of the three individual stages (KAO and HWANG, 2008), once the
geometric efficiency decomposition of the overall efficiency is restricted to CRS
situations. Thus, we adopted Additive efficiency decomposition (arithmetic mean)
approach (CHEN et al. 2009).
Proceedings of the 10
th
International Conference on DEA Brazil2012 142
Results and discussions
The models were run in the software GAMS (General Algebraic Modeling System),
and Excel 2010, using a notebook with processor Intel Core i5-2410M, CPU with
2.30 GHz, and 6.00 GB RAM. The studied city had five HBU, and they will just be
referred as HBU A, B, C, D and E. The
Table 6 presented the obtained data.
Table 6.The data of medical specialities for five Health Basic Units
In our preliminary study, we considered as disagreement point 1 =
i
, for i={1, 2, 3, 4},
thus it is possible guarantee that S was a feasible set. We obtained the efficiency
scores (Table 2) for each stage by equations from (5.1) to (5.3).
DMU Satisfaction Medical service time Wait time Medicine Available Capacity used
A-Internal medicine 86.22% 6.1 77.8 61.83% 81.25%
A-Pediatrics 95.14% 7.8 116.5 84.38% 81.25%
A-gynecology 77.38% 9.4 106.3 53.27% 64.58%
B-Internal medicine 74.13% 5.3 108.6 44.94% 65.25%
C-Pediatrics 58.87% 7.7 268.0 82.59% 68.75%
D-Internal medicine 80.38% 7.7 75.6 100.00% 68.75%
E-Internal Medicine 67.46% 14.2 171.2 62.92% 62.50%
E-Pediatrics 46.41% 15.2 96.8 54.64% 56.25%
E-Ginecology 80.02% 21.5 66.3 100.00% 25.00%
DMU Working conditions Physicians Functionaries Expenses
A-Internal medicine 68.57% 6 40 R$ 1240.18
A-Pediatrics 68.57% 2 40 R$ 1132.34
A-gynecology 68.57% 4 40 R$ 1186.26
B-Internal medicine 23.18% 4 13 R$ 929.76
C-Pediatrics 25.70% 2 12 R$ 513.17
D-Internal medicine 14.39% 4 15 R$ 1688.13
E-Internal Medicine 27.29% 2 16 R$ 588.04
E-Pediatrics 27.29% 2 16 R$ 588.04
E-Ginecology 27.29% 3 16 R$ 620.71
Proceedings of the 10
th
International Conference on DEA Brazil2012 143
Table 7. Efficiency scores for each stage.
Table 8 presents the value of scalar coefficients corresponded to the perspectives of
BSC. Note that, although the DMU B-Internal medicine was considered efficient at
third stage, according to
Table 7, the proposed model can identify an improvement opportunity.
Table 4 shows the targets for medical specialties.
Table 8. The value of scalar coefficients
DMU Stage 3 Stage2 Stage1 Overall (arithmetic mean)
A-Internal medicine 0.9487 0.6863 0.9018 0.8456
A-Pediatrics 1 1 1 1
A-gynecology 0.9183 0.6359 0.9484 0.8342
B-Internal medicine 1 0.6633 0.8432 0.8355
C-Pediatrics 0.7033 0.7628 1 0.8220
D-Internal medicine 0.9621 1 0.4749 0.8123
E-Internal Medicine 0.7094 1 0.8837 0.8644
E-Pediatrics 0.5043 0.9675 0.8837 0.7851
E-Ginecology 1 0.6590 0.8234 0.8275
DMU Beta 1 Beta 2 Beta 3 Alpha
A-Internal medicine 1 1 1.4277 0.6584
A-Pediatrics 1 1 1 1
A-gynecology 1 1.1115 1.3968 0.6687
B-Internal medicine 1.0085 1.1199 1.4882 0.9231
C-Pediatrics 1.4192 1 1.4402 1
D-Internal medicine 1 1.0383 1.7864 0.8000
E-Internal Medicine 1.1129 1.1284 1.4557 1
E-Pediatrics 1.6140 1.1278 1.4812 1
E-Ginecology 1 1.0446 1.4146 0.8268
Proceedings of the 10
th
International Conference on DEA Brazil2012 144
Table 9. Targets for medical specialities.
The dual values of
k
that represents the weights for a linear combination (virtual
DMU that is a benchmark) of medical specialties that are considered efficient. Making
an analogy with the BSC theory, it would be interesting to establish objectives,
measures and targets for the medical specialties, as well to develop the Strategic Map.
The BSC allows the identification of good initiatives, which should be adopted by the
inefficient DMU, based on implemented actions by a DMU that is a benchmark.
Table 10. Benchmarks for each medical specialty.
DMU Satisfaction Medical service time Wait time Medicine Available Capacity used
A-Internal medicine 86.22% 9.2 77.8 91.42% 81.25%
A-Pediatrics 95.14% 7.8 116.5 84.38% 81.25%
A-gynecology 77.38% 10.4 89.1 89.94% 71.78%
B-Internal medicine 74.76% 7.7 75.6 100.00% 73.08%
C-Pediatrics 83.55% 7.7 75.6 100.00% 68.75%
D-Internal medicine 80.38% 8.0 72.8 103.83% 71.39%
E-Internal Medicine 75.08% 16.1 91.7 88.33% 70.53%
E-Pediatrics 74.91% 17.2 85.8 90.33% 67.12%
E-Ginecology 80.02% 22.5 63.4 104.46% 68.75%
DMU Working conditions Physicians Functionaries Expenses
A-Internal medicine 97.90% 2 25 R$ 816.53
A-Pediatrics 68.57% 2 40 R$ 1132.34
A-gynecology 95.77% 2 24 R$ 793.29
B-Internal medicine 34.49% 2 12 R$ 513.17
C-Pediatrics 37.02% 2 12 R$ 513.17
D-Internal medicine 25.70% 2 12 R$ 513.17
E-Internal Medicine 39.73% 2 15 R$ 588.04
E-Pediatrics 40.43% 2 15 R$ 588.04
E-Ginecology 38.61% 2 12 R$ 513.17
Stage 1 A-Pediatrics C-Pediatrics
A-Internal medicine 49.0% 51.0%
A-Pediatrics 100.0% 0%
A-gynecology 45.2% 54.8%
B-Internal medicine 0% 100.0%
C-Pediatrics 0% 100.0%
D-Internal medicine 0% 100.0%
E-Internal Medicine 12.1% 87.9%
E-Pediatrics 12.1% 87.9%
E-Ginecology 0% 100.0%
Benchmarks
Proceedings of the 10
th
International Conference on DEA Brazil2012 145
Conclusions
This research demonstrated how is possible the use of the DEA to measure (and to
rank) the relative efficiency of the medical specialties offered by Health Basic Units
(HBU) in a medium size town of Brazil. The ranking of efficiencies provides important
information to the Health Secretary decides regarding actions to distribute better the
available resources, making more efficient the HBU, therefore improving the overall
performance of the Health System as a whole.
Regarding the set of medical specialties analyzed, it should be noted that the results
obtained are based on selected inputs and outputs and their priorities. Sometimes, the
targets proposed by the DEA technique cannot be applied in practice or they are very
difficult of achieve them, and, in these cases, managers ought to take them as a
reference. Each medical specialty must be analyzed according to its reality or
organizational context; therefore, even those, which achieved 100% of relative
efficiency, sometimes do not have the performance desired by the organization.
This occurs because the efficiencys concept represents the best relation between the
benefits obtained and the available resource used, but does not means these benefits
are the desired results.
References
Amado C.A.F., Santos S. P., Marques P.M. (2012) Integrating the Data Envelopment
Analysis and the Balanced Scorecard approaches for enhanced performance
assessment, Omega (40): 390-403.
Chen Y, et al. (2009) Additive efficiency decomposition in two-stage DEA, European
Journal of Operational Research (196): 1170-1176.
Stage 2 A-Pediatrics D-Internal medicine E-Internal Medicine
A-Internal medicine 0 76.9% 23.1%
A-Pediatrics 100.0% 0% 0%
A-gynecology 0% 72.9% 27.1%
B-Internal medicine 0% 100.0% 0%
C-Pediatrics 0% 100.0% 0%
D-Internal medicine 0% 100.0% 0%
E-Internal Medicine 0% 68.5% 31.5%
E-Pediatrics 0% 73.9% 26.1%
E-Ginecology 0% 100.0% 0%
Stage 3 A-Pediatrics B-Internal medicine E-Ginecology
A-Internal medicine 53.1% 30.9% 16.0%
A-Pediatrics 100.0% 0% 0%
A-gynecology 10.6% 71.8% 17.6%
B-Internal medicine 0% 89.3% 10.7%
C-Pediatrics 42.5% 49.3% 8.2%
D-Internal medicine 27.3% 64.0% 8.7%
E-Internal Medicine 0% 83.9% 16.1%
E-Pediatrics 0% 86.8% 13.2%
E-Ginecology 26.7% 68.6% 4.7%
Proceedings of the 10
th
International Conference on DEA Brazil2012 146
Cook, et al. (2010) Measuring performance of two-stage network structures by DEA: A
review and future perspective, Omega: The International Journal of Management
Science (38): 423-430.
Eilat H. et al. (2008) R&D project evaluation: An integrated DEA and balanced
scorecard approach. Omega (36): 895-912.
Emrouznejad A. et al. (2008) Evaluation of research in efficiency and productivity: A
survey and analysis of the first 30 years of scholarly literature in DEA, Socio-Economic
Planning Science (42): 151-157.
Fre R., Grosskopf S. (2000) Network DEA, Socio-Economic Planning Sciences (34):
35-49.
Hadad S. et al. (2011) Determinants of healthcare systems efficiency in OECD
countries, The European Journal of Health Economics 2011; DOI 10.1007/s10198-011-
0366-3.
Kao C., Hwang S-N. (2008) Efficiency decomposition in two-stage data envelopment
analysis: An application to non-life insurance companies in Taiwan, European Journal
of Operational Research (185): 418-429.
Kaplan R.S., Norton D.P. (1992) The Balanced Scorecard: measures that drive
performance, Harvard Business Review (70): 71-79.
Nash, J. F. (1950) The bargaining problem, Econometrica (18): 155-16.
Proceedings of the 10
th
International Conference on DEA Brazil2012 147
19. Iteratively Weighted Least
Squares in Stochastic Frontier
Estimation Applied to the Dutch
Hospital Industry
Jos L. T. Blank
Delft University of Technology, Netherlands, j.l.t.blank@tudelft.nl
Aljar J. Meesters
University of Groningen, Netherlands, A.J.Meesters@rug.nl
Abstract
This paper proposes an alternative class of stochastic frontier estimators. Instead of
making distributional assumptions on the error and efficiency component in the
econometric specification of a cost function model (or any other model), this class is
based on the idea that some observations contain more information about the true
frontier than others. If an observation is likely to contain much information, it will get
a large weight in the regression analysis. In order to establish the weights, we
propose an iterative procedure. In each step weights can be determined by the
residuals obtained earlier and a user-specified weighting function. In each step the
weights will be updated and a next stage WLS regression will be carried out.
The advantages of this approach are its high transparency, the easy application to a
full-specified model and its flexibility. It allows to directly observing which
observations determine the frontier for a large part. The easy expansion to a full-
specified model refers to a model that includes a cost function and its corresponding
share equations. Its flexibility refers to the use of several alternative weighting
functions and the easiness of testing for the sensitivity of the outcomes.
The model has been applied to a set of Dutch hospital data. The outcomes of this
application are promising. The model converges rather quickly and presents reliable
estimates for the parameters, the cost efficiencies and the error components.
Key words: weighted least squares. SFA, hospitals
Introduction
The methodology Stochastic Frontier Analysis (SFA), suggested by Aigner, Lovell and
Schmidt (1977) and Meeusen and Van den Broek (1977), has become a standard in
econometric estimation of production and cost (or any other value) function. It is
based on the idea that empirically, production (or cost) can be described as a function
of a number of inputs (or outputs and input prices), a stochastic term, reflecting
errors, and another stochastic term, reflecting efficiency. With maximum likelihood
techniques, the parameters of the function and the parameters of the distribution of
Proceedings of the 10
th
International Conference on DEA Brazil2012 148
the stochastic components can be estimated. For extensive discussions on this
technique, see e.g. Kumbakhar & Lovell (2000) and Fried, Lovell & Schmidt(2008).
Since the 1977 publications, SFA has become very popular and has been applied in
many empirical work (for extensive literature reviews, see also Fried, Lovell and
Schmidt (2008) and Blank (2000)). Nevertheless, the approach has also widely been
criticized. The criticisms focus on two major points. The first type of criticism debates
the a priori specification of the production (or cost) function, the second type
concerns the assumptions on the distribution of the stochastic terms. Although both
criticisms can, to a certain extent, be overcome, by using flexible forms and using
different assumptions on the distribution of the stochastic variables in the analysis, the
rigidity might be seen as a problem. Although not mentioned very often, there is a
third type of criticism, which can be considered to be of a conceptual nature. In a
rather complex econometric framework, the methodology suggests to observe an
unobservable (the efficiency), which can be distracted from another unobservable (the
measurement and specification error). Those, who try to explain this approach to non-
initiated persons, such as managers and policy makers, will be confronted with
scepticism and unbelief. A technique like Data Envelopment Analysis, which is
actually seeking for observations that form the envelope, is far more appealing and
more transparent. This explains that in real-life problems, DEA has become a very
popular tool in applied work. Another conceptual framing of SFA may tackle the
problem and make the technique more accessible for non-experts.
The original work of Aigner, Lovell and Schmidt (1977) derives the stochastic frontier
approach in case of a single equation model. In a single equation model, we can only
estimate the technical or cost efficiency. If we are not only interested in technical or
cost efficiencies, but also in allocative efficiencies, a multiple equations approach,
allowing to derive the under- or overutilization of inputs, is needed. However, the
estimation of a multiple equations model, with a far going decomposition of the
underlying stochastic variables for measurement errors, technical and allocative
efficiency, is very troublesome. In particular, the theoretical linkage between the cost
function and the input demand equations is extremely difficult to handle (the so-called
Greene problem). Although some interesting solutions have been proposed by
applying shadow cost models (see Blank, 2009; Kumbhakar, 1997) or using Bayesian
estimation techniques, new estimation problems occur. Obviously, these approaches
suffer even more from a lack of transparency.
An alternative for the original SFA approach is the thick frontier analysis (TFA),
developed by Berger & Humphrey (1991). This approach allows to estimate a single
equation or a multiple equation. The technique is based on the selection of firms in
the top 10% (or any other percentage) and the bottom 10%. For both subsamples, the
production (or cost) function is estimated separately. Consequently, cost efficiencies
are derived by taking the ratio of the average cost of the worst practice firms and the
best practice firms. TFA has a number of advantages. Seemingly Unrelated Regression
allows for a straightforward estimate of a system of a cost function and the
corresponding share equations. TFA doesnt require any rigid assumptions on the
distributions of the error components and it does not suffer from the Greene problem.
Conceptually, it is a very transparent and appealing approach. On the other hand, it
Proceedings of the 10
th
International Conference on DEA Brazil2012 149
also has some serious drawbacks. It does not provide firm-specific cost efficiencies,
but only rather general efficiency scores. From an econometric point of view, there is
a loss of information, due to the discard of a large subset of observations. It is
questionable whether the researcher has the luxury of losing so many degrees of
freedom.
Estimating a production, cost, or profit frontier (hereinafter: frontier) would become
trivial if all firms would operate with full efficiency. One could use ordinary least
squares to estimate the parameters of the model. However, in reality some firms are
inefficient, which makes the estimation of the frontier a challenging task. This problem
could be solved by only taking account of efficient firms for the estimation of the
frontier, and by neglecting all inefficient firms. However, this method implies the
requirement of a priori knowledge about whether or not a firm is efficient and
obviously, knowledge about the efficient firms is, in general, not available prior to the
estimation of a production frontier and therefore, other methods for addressing this
problem have been proposed.
One method that is often used for estimating a frontier is the aforementioned
stochastic frontier analysis (SFA). In this approach, a stochastic term is added to an
OLS equation, where it is assumed that it follows a distribution with a non-negative
support. This stochastic term is supposed to pick up the inefficiency for each firm.
Although SFA includes the concept of inefficiency while estimating frontiers, it does
have its shortcomings. Firstly, it is often being criticized for its distributional
assumption for the efficiency component (see e.g. Ondrich & Ruggiero, 2001).
Secondly, although SFA allows for the estimate of cost functions, the concept of cost
efficiency doesnt seem to fit in with this type of estimation. Cost efficiency is built on
technical and allocative efficiency, and yet, under the SFA specification of a cost
function, all firms should be completely allocatively efficient (Greene, 1980).
Another approach for estimating a frontier was provided by Wagenvoort and Schure
(2006). They show the way in which efficient firms can be identified, if panel data are
available. They use a recursive procedure, by dropping the most inefficient firm at
each iteration. In each step, the firm-specific efficiency is calculated by averaging the
residuals of a firm over the whole time period. Their final step consists of using the
fully efficient firms for estimating the frontier. Our approach is similar to this idea;
however, we argue that the fact whether or not a firm is fully efficient doesnt concern
a zero-one-probability. We show the way in which a weighting scheme can be
implemented, in order to determine which firms are likely to be efficient and which
firms are likely to be inefficient. This concept also translates to a cross-section setting
so as to avoid the necessity of panel data. This also implies that we dont need to
assume that inefficiency is time-invariant, which can also be regarded as a rather
restrictive assumption in many efficiency models that are based on panel data.
In this paper, we propose an alternative that is related to the concept of stochastic
frontier analysis, while being far more appealing conceptually. As in TFA, it can also
be applied to multiple equation systems, while avoiding the Greene problem. Our
alternative incorporates information from all the data available. It allows decomposing
cost efficiency into technical and allocative efficiency without any additional
distributional assumptions. The method is based on an Iterative Weighted Least
Proceedings of the 10
th
International Conference on DEA Brazil2012 150
Squares (IWLS) method and can easily be programmed in standard econometric
software.
The outline of the paper is as follows. In Section 2, we discuss a few conceptual
issues of our method. In Section 3, we introduce a formal description of the model
and the estimation procedure. In Section 4, we apply the method to a set of Dutch
hospital data. Section 5 concludes the paper.
Methods
We start from the cost function, although the method may be applied to any other
model (see e.g. Fre & Primont, 1995). We assume that the firm is cost-minimizing and
that the total cost can be represented by a cost function c(y,w) that meets all the
requirements it entails. Input demand equations x
n
(y,w) can be derived from the cost
function, by applying Shephards Lemma. For reasons of convenience, we rewrite the
cost equations and input demand equations in terms of logarithms and cost shares,
and add an error term.
ln() = (ln() , ln()) +
0
(1)
(ln() , ln()) +
(2)
With:
C = total costs;
y = vector of outputs;
w = vector of input prices;
S
n
= optimal cost share for input n (n = 1,.., N).
0
,
n
error terms
Equations (1) and (2) can be estimated by a certain minimum distance estimator or, if
one wants to check for heterogeneity, with fixed or random effects, which will result
in consistent estimates of the parameters if [|, ] = 0. However, if some firms are
inefficient, i.e. they have a higher cost than what can be explained, the cost function
or random noise [] > 0, causing biases in the parameters of Equations (1) and (2).
Our suggestion for reducing these biases consists of estimating Equations (1) and (2)
with weighted least squares and assigning the ill-behaving observations with a low
weight, while the well-behaving observations will be assigned with a high weight.
Weighted least squares (WLS), which is also referred to as generalized least squares
(GLS), is a widely used econometric technique; however, since the weights are
generally not observable, they have to be estimated (see e.g. Verbeek, 2012). Our
proposed weighting scheme is based on the residuals obtained after Equations (1) and
(2) have been estimated with LSQ
3
, as we know that firms that are highly inefficient,
and thus likely to bias the results, will have a large residual , where is the estimate
of .
3
If Equation (1) and (2) are estimated with fixed effects, the weights can also be based on the fixed
effects, which would make our estimator to a generalized version of the estimator, suggested by
Wagenvoort and Schure (2006).
Proceedings of the 10
th
International Conference on DEA Brazil2012 151
Obviously, the way in which should be transformed into weights is as debatable as
the distributional assumption for the efficiency component in SFA. The weighting
scheme should reflect the tradeoff between noise and inefficiency. If one expects all
firms to be efficient, the deviation from the frontier, captured by, ismostly determined
by noise. If a weighting scheme, reducing the weight rapidly if is increasing, is being
used, the assessment of the level of the frontier will prove to be overly optimistic,
since firms that perform very well due to luck will get a larger weight. On the other
hand, if many firms are inefficient, while a weighting scheme that is virtually flat for
all is being used, the estimation of the frontier will prove to be too low, since firms
that effectively are very inefficient will still be considered to be quite efficient. One
way to determine the amount of noise versus inefficiency is to examine the skewness
of the LSQ residuals. It is easy to implement other weighting schemes and see if the
results differ. This is another advantage of our approach over the SFA approach,
which requires to calculate the convolution of two random variables and to derive the
maximum likelihood, if one wants to use another distribution.
Since the weighting scheme depends on ,which can be updated after the second
step, an iterative reweighted least squares procedure can be implemented. This
procedure is used for some robust regression estimators, such as the Huber W
estimator (Guitton, 2000). This similarity is no coincidence, since our suggested
estimator can also be considered as a robust type of regression. This implies that, after
each WLS estimation, new s are calculated, which are then used to generate new
weights, which, on their turn, are used in a next stage WLS estimation, until the
convergence criterion upholds. The convergence criterion we use requires that the
parameter estimates dont differ more than one percent from the previous stage.
Normally, we also want to know the levels of inefficiency and not only the parameters
of the cost function. In our suggested model, it is not obvious how these levels can be
calculated since, for the calculation of E[|], where is the level of efficiency, we
need at least the probability density distributions of and . However, our estimator
doesnt require making an assumption for the distribution of . This doesnt mean that
we are not able to say something about the efficiency of each firm.
Ondrich and Ruggiero (2001), for instance, show that, if a normal distribution is
assumed for the noise, the ranking of is equal to the ranking of and therefore, our
model enables us to specify the efficiency ranking for each firm.
Although the distributional assumptions about the efficiency term are not necessary
for the estimation, we still might use them for deriving the efficiency scores.
Therefore, we introduce the two usual unobservables u and v, representing the (in)
efficiency and the error term, respectively. We simplify the original problem of Aigner
et al. (1977), by only estimating the distribution of the error term, instead of both
components simultaneously. Since we did identify the cost frontier, we are able to
select a subsample of observations that satisfy u = 0, i.e. all observations with
observed cost lower than or equal to frontier cost (v 0). Note that we are not able to
identify observations that satisfy u = 0 and v 0, i.e. efficient firms with observed cost
greater than frontier cost. Therefore, we assume that |v| in the subsample is
distributed as
+
(0,
2
). The variance
2
can now be estimated by the sum of
Proceedings of the 10
th
International Conference on DEA Brazil2012 152
squared residuals, divided by the number of observations in the subsample (denoted
as
2
). Furthermore, in the full sample, we assume that the subsample is
representative with respect to the variance of the random errors and that random
errors are distributed as (0,
2
). Since we now have an estimate for the variance of
the random errors, we are also able to conditionally derive the expected efficiency
from the residuals, by applying, for instance, Materovs formula:
(
) =
2
if
0; = 0 otherwise (3)
with:
2
=
2
The efficiency score then equals:
= exp ( (
) (4)
Obviously, there are other alternatives available (see e.g. Kumbakhar & Lovell, 2000).
Note that, in comparison with the original Jondrow et al. (1982) paper, in the model,
we have swapped the roles of the random error and efficiency components. It is
important to stress that we dont apply the distributional assumptions to the errors and
efficiency components in the estimation procedure, but only in the derivation of the
efficiency scores. However, this procedure has some consequences for the weighting
scheme in the estimation procedure. Since we assume that all negative residuals
indicate efficient firms and that the residuals only reflect random noise, there is no
reason for assigning different weights to these observations; therefore, for reasons of
consistency, we use a weighting scheme that assigns a weight equaling 1 to all the
observations in the efficient subset.
Results and discussions
Model specification
We apply the well-known translog cost function model (Christensen et al., 1973;
Christensen & Greene, 1976). The cost function model consists of a translog cost
function and the corresponding cost share equations. The model includes first and
second order terms, as well as cross-terms between outputs and input prices, on the
one hand, and a time trend, on the other hand. These cross-terms with a time trend
represent the possible different natures of technical change. Cross-terms with outputs
refer to output-biased technical change and cross-terms with input prices refer to
input-biased technical change.
Proceedings of the 10
th
International Conference on DEA Brazil2012 153
( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( )
( ) ( ) + +
+ + +
+ + +
+ + + + + =
= =
= = = =
= = = = = =
= = = = =
N
n
n i
M
m
m i
O
o
M
m
m o ij
O
o
N
n
n o on
N
n
N
n
M
m
N
n
n m mn
O O
o
o o oo n n nn
M
m
M
m
m m mm
O
o
o o
N
n
n n
M
m
m m
W T j Y T i
T h Y Z g W Z f
W Y e Z Z d W W c
Y Y b Z d W c Y b a C
1
1
1
1
0
1 1 1 1
1 1 ' 1 1 1 0 ' 1 '
' ' '
1 1 '
' '
1 1 1
0
ln ln
ln ln
2
1
ln ln
ln ln ln ln
2
1
ln ln
2
1
ln ln
2
1
ln ln ln ln
(1)
With:
C = total costs;
Y
m
= output m (m = 1,.., M);
T = year of observation;
W
n
= price of input n (n = 1,.., N);
Z
o
= fixed input o (o = 1,.., O).
n m om on mn oo nn mm o n m o
j i h g f e d c b d c b a
1 1 0 ' ' '
, , , , , , , , , , , ,
parameters to be estimated.
By applying Shephards Lemma, we see that the optimal cost share functions can be
presented as:
( ) ( ) ( )
= = =
= + + + + =
N
n
M
m
O
o
in o on m mn n nn n n
N n T j Z f Y e W c c S
1 ' 1 1
'
) ,.., 1 ( ln ln ln
(2)
With:
S
n
= optimal cost share for input n (n = 1,.., N)
Homogeneity of degree one in prices and symmetry is imposed by applying
constraints to some of the parameters to be estimated. In formula:
o o oo n n nn m m mm
d d c c b b
' ' ' ' ' '
; ; = = =
= = = = =
= = = = =
N
n
n
O
o
on
N
n
mn
N
n
nn
N
n
n
j k f m e n c c
1
1
1 1 1
'
1
0 ; ) ( 0 ); ( 0 ); ' ( 0 ; 1
(3)
Equations (1) and (2) can be estimated by OLS or if one wants to control for
heterogeneity, with fixed or random effects, which will result in consistent estimates of
the parameters if [|, ] = 0. Yet, if some firms are inefficient, i.e. they have higher
cost than what can be explained, the cost function or random noise [] > 0, causing
a bias in
0
. Moreover, if the input mix of a firm is partly determined by the firms
expectations of its efficiency level we even have [|, ] where is a constant.
This causes biases, also in the other parameters of Equations (1) and (2).
Data
The data for this study cover the period 2003-2009 and were obtained from the Dutch
Hospitals Association. Annual financial, patient and personnel data were collected by
means of surveys. The surveys cover all the general hospitals in the Netherlands. For
Proceedings of the 10
th
International Conference on DEA Brazil2012 154
the purpose of this study, the data were checked for missing or unreliable data.
Various consistency checks were performed on the data, in order to ensure that
changes in average values and in the distribution of values across time were not
excessive. In particular, observations with unit value (for instance the personnel costs
per FTE for each type of personnel) less than 0.5 times or exceeding 2 times the
median value were identified. After the elimination of observations, whose dataset
contained inaccurate or missing values, we obtained an unbalanced panel dataset of
406 observations over the 5 years of study. There are approximately 80 observations
for each year. The year 2005 has the highest coverage with 84 observations out of 89,
2007 has the lowest coverage with 75 observations out of 86.
The main service delivery of hospitals consists of the treatment of patients. Therefore,
the output of hospitals is measured by the number of discharges, including day-care
patients and outpatients (not followed by an admission). The discharges have been
divided into over 30 medical specialties, in order to measure case-mix. Since it is not
possible to use such a large number of categories, they have been aggregated into
three categories, on the basis of average stay homogeneity and the distinction
between surgery/non-surgery specialties. Therefore, we distinguish the following three
groups of specialties:
Surgery and Non-surgery with average stay less than 4 days;
Non-surgery with average stay more than 4 days;
Surgery with average stay more than 4 days.
This results in four types of output, three types of inpatients (including day-care
patients) and outpatients. Although these four types of production explain a very large
part as well see later of variations in cost, the services are much more nuanced
than just the number of outpatients and discharges. The health outcome of patients
seems to be a particularly important component of hospital production. Nevertheless,
it seems reasonable to assume that the quality has not decreased, as it is constantly
monitored, for instance, by the health inspectorate, patient associations and media,
and subjected to quality-improving interventions by physicians and hospital
management. Therefore, the estimates of productivity change can be regarded as a
lower bound.
Resources include staff, administrative and maintenance personnel (including security
and cleaning), nursing personnel, paramedical personnel (such as lab technicians),
material supplies, maintenance and capital. Physicians are not included in these
personnel variables, in order to ensure that hospitals with hospital-employed
physicians and hospitals with self-employed physicians are treated equally. The costs
of physicians (wages) are not included in the cost or price variables either.
Material supplies include items such as medical supplies, food and general cost.
Maintenance includes energy costs, costs related to grounds and buildings. The
maintenance costs are rather low and have, for reasons of simplicity, been added to
the material supplies. Personnel and material supplies are treated as variable
resources, since the hospital can change these in the short term.
Capital refers to capital assets, such as buildings and medical equipment. The volume
of the capital is measured as a weighted aggregate of beds, intensive care beds,
Proceedings of the 10
th
International Conference on DEA Brazil2012 155
psychiatric beds, square meters and number of radiotherapists (a proxy for the
number of linear accelerators and cobalt units).
There are data on the costs and the quantity for each resource personnel category. For
each region and time period, wages are defined as the average wage per full time
equivalent. This is considered to be the market price for labour; qualitative differences
between hospitals are included in the volume of labour.
Since for material supplies, there is no natural unit of measurement, they are
presented by means of a circumventing concept. The price of material supplies is a
weighted index, based on components of the consumer index, calculated for the
Netherlands by Statistics Netherlands. The weights are derived from cost shares.
The price of capital is defined as a unit value, derived from capital costs and the
aforementioned volume of capital.
Estimation results
The models are estimated as multivariate regression systems with various equations
with a joint density, which we assume to be a normal distribution. Because
disturbances are likely to be cross-equation-correlated, Zellners Seemingly Unrelated
Regression method is being used for estimation (Zellner, 1962). As usual, as the shares
add up to one, causing the variancecovariance matrix of the error terms to be
singular, one share equation in the direct cost function model is eliminated. Since we
are dealing with a relatively large number of cross-sectional units and a limited
number of periods, we ignore the fact that we are dealing with a panel data (with
respect to intra-firm correlations). It is obvious that the between variance is far more
important than the within variance.
In our analysis, we use the following weighting scheme:
=
1
1+
| |
|
\ . =
(6)
Armed with (6), we define a general least distance p-norm measure by
{ }
( , ) min ( , ) ( , ) ( , )
p
p
f E x y x y x y x y | | (7)
which is an inefficiency measure that identifies the closest point on E from an activity
( , ) x y . Pastor and Aparicio (2010) showed that an efficiency measure defined by
{ }
2
1 min ( , ) ( , ) Z E x x y y x y (8)
is not a strongly monotonic efficiency measure that seeks closest points over the
efficient frontier E . Recently, Ando et al. (2012) considered the following axiom
(Axiom C ) by relaxing Axiom C.
Axiom ' C : For ( , )
a a
T x y and ( , )
b b
T x y , ( , ) ( , )
a a b b
x y x y implies
( , ) ( , )
a a b b
f f x y x y .
Ando et al. (2012) developed a least distance inefficiency measure satisfying weak
monotonicity over E by modifying (7) as follows:
Proceedings of the 10
th
International Conference on DEA Brazil2012 166
( )
{ }
, min ( , ) ( , ) ( , ) , ( , ) ( , )
p
p
f E D = x y x y x y x y x y x y
,
(9)
where
{ }
( , ) = ( , ) ( , ) ( , ) . D x y x y x y x y (10)
The situation where an activity ( , ) x y belongs to ( , ) D x y means that x can product
y if ( , ) x y is feasible. Hence, ( , ) D x y represents a free disposal input-output set of
an activity ( , ) x y . ( , ) D x y is utilized to define a slacks-based supper-efficiency
measure in Cooper et al. (2007, p. 313). By incorporating ( , ) D x y into
{ }
min ( , ) ( , ) ( , )
p
E x y x y x y | | , Ando et al. (2012) shows that
p
f , defined in (9), is
weakly monotonic over E for all [1, ] p .
2.2 An extension of ( , ) D x y
This study extends (9) to a least distance efficiency measurement setting, in which
strong monotonicity is satisfied, i.e., all of Axioms A, B and C are satisfied. For this
purpose, we replace ( , ) D x y of (10) by by ( , ) D
| |
+
|
|
= | |
`
|
|
|
|
\ .
)
\ .
x y x d y d
x y x y d
d
.
.
. (11)
In what follows, we show the profits associated with any activities in ( , ) D
x y do not
exceed the profit of ( , ) x y . That is, we utilize shadow prices of inputs and outputs of
a DMU in place of observed prices to interpret ( , ) D
x y be
{ }
min ( , ) ( , ) ( , ) , ( , ) ( , )
p
E D
x y x y x y x y x y , (12)
then it follows from
0
( , ) = ( , ) D D x y x y that
0
( , ) = ( , )
p p
f f x y x y and hence, ( , )
p
f
x y
is a natural extension of (9).
Now, let us explain how ( , ) D
= =
= =
+ + + +
`
+ =
)
v w
. .
(13)
We assume each component of
( ) , v w in (13) is positive. The following proposition
establishes an equivalent relationship between ( , ) D
x y and
( )
Proposition 1: ( ) ( ) , , D
x y x y wy vx wy vx ( ) ( ) , v w .
Proposition 1 indicates that, in
( ) , D
(14)
It is known that, for any efficient activity ( , ) E x y , there is at least one positive
shadow price vector ( ) , v w satisfying
( ) , T wy vx wy vx x y . Furthermore,
since the efficient frontier is the union of finitely many faces, we can limit to finitely
many positive shadow price vectors for each efficient activity. Consequently, if we
select a sufficiently small 0 > , then the existence of
( ) is guaranteed. The
collection of shadow price vectors contains at least one positive price vector
( ) , v w
satisfying (a) in (14) for any efficient activity ( , ) E x y . That is,
( ) contains
information on shadow prices enough to describe the efficient frontier E . For all
shadow price vectors ( ) ( ) , v w , any activities ( ) , x y in
( ) , D
x y cannot have
higher shadow profits than the evaluated DMU ( , ) x y . Mathematically, we can
express this situation as follows:
( ) ( )
( ) ( ) ( )
( ) ( )
( ) ( )
( )
, ,
, , ,
,
max 0 , , ,
max 0 ,
D
D
T
E
=
=
x y x y
x y x y
v w
wy vx wy vx v w x y
wy vx x y
Proceedings of the 10
th
International Conference on DEA Brazil2012 168
3. Least distance inefficiency measure satisfying strong
monotonicity
Consider two cones translating the origin to ( , ) x y , ( , ) D x y and ( , ) D
x y , that satisfy
( , ) ( , ) D x y x y {( , )} ( , ) ( , ) D int D
x y x y x y (15)
for any positive , where int S is the interior of S .
Lemma 1: For any >0 , (15) holds.
Lemma 2: Choose ( , ) T x y and >0 arbitrarily and let
* * * *
( , , , ) x y x y be an
optimal solution of (12). If ( , ) = E D
x y or
{ } ( , ) = ( , ) E D
x y x y , then
* *
( , ) ( , ). int D
x y x y (16)
Lemma 3: Choose ( , ) T x y and >0 arbitrarily and let
{ }
( , ) ( , ) \ ( , ) D x y x y x y .
If ( , ) = E D
x y or
{ } ( , ) = ( , ) E D
x y x y , then
( , ) < ( , ) for all [1, ].
p p
f f p
x y x y (17)
The inequality (17) means that ( , )
p
f
x y x y x y (18)
( , ) \ ( , ) = . T E D E
x y x y (19)
The existence and choice of the positive value were discussed and provided in our
previous work (Fukuyama and Sekitani 2012) in a different context. The existence of
>0 guarantees Axiom C as well as Axiom A. For any face F of T , we define
1 =1
=1, =1, ,
( ) ( , ) .
= ( , )
N M
n m j j
n m
v w j J
VW F
F
=
+
`
)
vx wy
v w
vx wy x y
.
(20)
A face F is called a maximal efficient face if F E and a face G with G F
and G E implies G F . Since E consists of a finite number of maximal efficient
faces, we have L maximal efficient faces, say
1
, ,
L
F F , such that
=1
=
L
l l
E F . For any
( , ) E x y there exists an index {1, , } l L such that ( , )
l
F x y . Consider ( , ) T x y .
There exists ( , ) ( )
l
VW F v w such that > vx wy if and only if ( , )
l
F x y . Hence,
( , ) \ T E x y if and only if for every =1, , l L . there exists ( , ) ( )
l
VW F v w satisfying
Proceedings of the 10
th
International Conference on DEA Brazil2012 169
> vx wy . Let
{ }
*
1 1
=1, , ( , ) ( )
= min , , , , ,
max min
N M
l L VW F
l
v v w w
v w .
. . . (21)
It follows from
1 1
min{ , , , , , } 1/ ( )
N M
v v w w N M + . . , the definition of E and Motzkin's
theorem of the alternative that
*
0< 1/ ( ) N M + . Let the maximal be obtained by
( )
1
* 1
= 1 N M
x y .
( )
B B
, N x y and
( )
C C
, N x y are normal cones of
( )
B B
, x y and
( )
C C
, x y , respectively. The set of all allowable shadow prices
( ) is
shown in the shaded area in two places. is chosen so that
( )
B B
, N x y and
( )
C C
, N x y are contained in ( ) . The shadow profit-based dominated set ( , ) D
x y ,
which is a proper superset of ( , ) D x y for any 0 >
and is shown as the area
described by two heavy broken lines, can be constructed from the set
( ) . For
example,
3 3
( , ) D x y
x y x y , then
for every =1, , l L . , there exists ( , ) ( )
l l
l
VW F v w such that
0.
l l l l
v x w y v x w y (22)
Moreover, =0
l l
v x w y if and only if ( , ) =( , )
l
F x y x y .
Lemma 5: Choose ( , ) T x y and (0, ) arbitrarily. If ( , ) E x y , then
( )
E E
, D x y
( )
E E
, D
x y
( )
C C
, D
x y
( )
B B
, D
x y
( ) C C
, N x y
( )
( ) B B , N x y
1
x
2
x
Proceedings of the 10
th
International Conference on DEA Brazil2012 170
{ } ( , ) = ( , ) D E
x y x y . If ( , ) \ T E x y , then ( , ) = . D E
x y
Proposition 2: Choose (0, ) and [1, ] p arbitrarily. Then, ( , )
p
f
x y satisfies
strong monotonicity over E .
4. Summary and Conclusions
The present paper has developed a least distance p-norm efficiency (inefficiency)
measure satisfying strong monotonicity over the efficient frontier as well as efficiency
indication, zero-unity requirement, units invariance and translation invariance. For
the development of our measures, we exploit the shadow profit-based dominated set,
which has the interpretation of shadow prices and profits for the DMU being
evaluated. The shadow profit-based dominated set is an extension of the free
disposal input-output set that is utilized in the development of super-efficiency
measures. The fundamental difference between the two dominated sets is that the use
of the first set enables us to construct strongly monotonic least distance inefficiency
measures, whereas the use of the latter only guarantees weak monotonicity.
References
Ando K., A. Kai, T. Maeda, K. Sekitani (2012) Least Distance Based Inefficiency
Measures in the Pareto-Efficient Frontier in DEA, Journal of the Operational Research
Society of Japan 55: 73-91.
Briec, W. (1998) Hlder Distance Function and Measurement of Technical Efficiency,
Journal of Productivity Analysis 11: 111-130.
Cooper W.W., K. S. Park, d J. T. Pastor (1999) RAM: A Range Adjusted Measure of
Inefficiency for Use with Additive Models and Relations to Other Models and Measures
in DEA, Journal of Productivity Analysis 11: 5-42.
Cooper W.W., L.M. Seiford, K. Tone (2007) Data Envelopment Analysis, A
Comprehensive Text with models, applications, references and DEA-Solver Software,
second ed., Springer, New York
Fukuyama H., K. Sekitani (2012) An Efficiency Measure Satisfying the Dmitruk-
Koshevoy Criteria on DEA Technologies, Journal of Productivity Analysis 38: 131-143.
Pastor J.T., J. Aparicio (2010) The Relevance of DEA Benchmarking Information and
the Least-Distance Measure: Comment, Mathematical and Computer Modelling 52:
397-399.
Proceedings of the 10
th
International Conference on DEA Brazil2012 171
21. Maximal Allocated Benefit and
Minimal Allocated Cost and its
Application
Mozhgan Mansouri Kaleibar
*
Young Researchers Club, Tabriz Branch, Islamic Azad University, Tabriz, Iran,
mozhganmansouri953@gmail.com
Sahand Daneshvar
Tabriz Branch, Islamic Azad University, Tabriz, Iran, sahanddaneshvar1@yahoo.com
Abstract
In this paper, we investigate the problems of consensus-making among institution in
stock exchange with multiple criteria for evaluating performance when the players
(institutions) are supposed to be egoistic and the score for each criterion for a player
is supposed to be a positive score. Each player sticks to his superiority regarding the
criteria. This paper introduces the models for computing minimal cost ratio and the
maximal benefit for institutions.
Keywords: Cooperative Game, DEA, Game Theory, Stock Exchange, Weight
selection.
Introduction
Let us suppose n players each have m criteria for evaluating their competency or
ability, which is represented by a positive score for each criterion. As with usual
classroom examination, the higher the score for a criterion is, the better player is
judged to perform that criterion. For example, there are three students A, B and C,
with three criteria of linear algebra, real analysis and numerical analysis. The scores
are their records for the three subjects, measured by positive cardinal numbers. All
players are supposed to be selfish in the sense that they insist on their own advantage
on the scores. Similar situations exist with many societal problems. We now present
some of the potential applications of DEA game. In the literature on cooperative game
theory, there have been many applications to cost or benefit sharing problems. The
proposed DEA game models are in sharp contrast to them, in that we can deal with
these problems under multi criteria environments that are common to real conflicts
in our society. This paper through allocating and imputing the given benefit [6]
propose a new scheme for computing maximal allocated benefit and minimal
allocated cost for the institutions under the framework of game theory and data
envelopment analysis (DEA). The different sections of this paper are sequenced in the
following order:
*
Corresponding author
Proceedings of the 10
th
International Conference on DEA Brazil2012 172
In section 2, the basic models are described and some properties of problem are
proved. In section 3, the extensions to the basic model and computing the ratios by
proposed models are discussed. In section4, a numerical presentation of real data
from a Stock Exchange of Tehran is elaborated on. Finally, section 5 includes
conclusions and some remarks.
Basic models of the game
In this section, we introduce the basic models and structures of the game.
Selfish behavior
Let ( )
m n
ij
X x R
+
= be the score matrix, scores of player j in the criterion i for
1,..., i m = and 1,..., j n = and
0
ij
x >
. It is assumed that the higher score for a criterion
is, the better player is judged to perform as regard to that criterion. Each person k has
a right to choose two sets of nonnegative weights
1
( ,..., )
k k k
m
w w w = to the criteria that
are most preferable to the player. Using the weight
k
w , the relative scores of player k
to the total score
are defined as follows:
1
1 1
(1)
( )
m
k
i ik
i
m n
k
i ij
i j
w x
w x
=
= =
The denominator represents the total score of all players as measured by player k s
weight selection, while the numerator indicates player k self evaluation using the same
weight selection. Hence, the expression (1) demonstrates player k relative importance
(share) under the weight (or value) selection
k
w . We assume that the weighted scores
are transferable. Player k wishes to maximize this ratio by selecting the most
preferable weights, thus resulting in the following fractional program.
( )
1
1 1
( 2)
( )
. 0
k
m
k
i ik
i
m n
w
k
i ij
i j
k
i
w x
Max
w x
s t w i
=
= =
The motivation behind this program is that player k aims to maximize his relative
value as measured by the ratio: the weighted sum of his records vs. the weighted sum
of all players' records. This arbitrary weight selection is the fundamental concept
underlying DEA initiated by Charnes et al. [2]. DEA terms this as variable weight, in
contrast to a prior fixed one. Refer to cooper et al. [10] for further explanation of this
issue.
Before continuing, we reformulate the problem as follows, without losing generality.
We normalize the fuzzy data set X so that it is row-wise normalized, i.e.,
1
1 ( )
n
ij
j
x i
=
=
.
Proceedings of the 10
th
International Conference on DEA Brazil2012 173
For this purpose, we divide the row
1
( ,.., )
i in
x x by the row-sum
1
n
ij
j
x
=
for 1,..., i m = .
The program (2) is not affected by this operation. Thus, using the Charnes-Cooper
transformation scheme, the fractional program (2) can be expressed using a linear
program as follows:
1
1
( )
. 1, 0 (3)
m
k
i ik
i
m
k k
i i
i
c k Max w x
s t w w i
=
=
=
=
Apparently, the optimal solution is given by assigning 1 to
( )
k
i k
w for the criterion i(k)
such that } {
( )
1,.....,
i k ik
x Max x i m = =
and assigning 0 to the weight of remaining
criteria. We denote this optimal value by c(k).
( ) 1,...,
ik
c k x k n = = (4)
The c(k) indicates the highest relative score for player k which is obtained by the
optimal weight selecting behavior. The optimal weight
( )
k
i k
w may differ from one
player to another.
Theorem 1.
1
( ) 1 (5)
n
k
c k
=
That is, sum of maximized scores is greater or equivalent 1.
Proof. Let the optimal weight for player k is
* * * *
1 ( )
( ..., ), 1
k k mk i k k
w w w w = = and
0( ( ))
ik
w i i k
= then, we have
*
( ) 1
1 1 1 1 1
( ) 1
n n m n n
i k ik i k k k
k k i i k
c k w x x x
= = = = =
= = =
The inequality above follows from
( ) 1 i k k k
x x
and the last equality follows from the
rowwise normalization.
This theorem asserts that, if each player sticks to his egoistic sense of value and insists
on getting the portion of the benefit as designated by c(k), the sum of shares usually
exceeds 1 and hence c(k) cannot fulfil the role of division of the benefit. If eventually,
the sum of c(k) turns out to be 1, all players will agree to accept the division c(k),
since this is obtained by the players most preferable weight selection. The latter case
will occur when all players have the same and common optimal weight selection.
More concretely, we have the following theorem.
Theorem 2.
The equality
1
( ) 1
n
k
c k
=
=
.
Coalition with additive property
Let the coalition S be a subset of player set (1,..., ) N n = . The record for coalition S is
defined by ( ) ( 1,..., ) (6)
i ij
j S
x S x i m
= =
These coalitions aim to maximize the outcomes c(S).
1
1
( ) ( )
. 1 , 0 (7)
m
i i
i
m
i i
i
c S Max w x S
s t w w i
=
=
=
=
The c(S) with ( ) 0 c = , defines a characteristic function of the coalition S. Thus this
game is represented by (N,c).
Definition 1. A function f is called sub additive if for any S N and TN with ST
= the following statement holds: ( ) ( ) ( ) f S T f S f T + .
Definition 2.A function f is called super additive if for any S N and T N with S
T = the following statement holds: ( ) ( ) ( ) f S T f S f T + .
Theorem 3. The characteristic function of c is sub additive, for any S N and T N
with ST = we have ( ) ( ) ( ) (8) c S T c S c T +
Proof. By renumbering the indexes, we can assume that { } 1,..., , { 1,..., } S h T h k = = +
and { } 1,..., S T k = . For these sets, it holds that
1 1 1
( ) ( ) ( )
k h k
ij ij ij
i i i
j j j h
c S T Max x Max x Max x c S c T
= = = +
= + = +
Theorem 4. ( ) 1 c N = .
Proceedings of the 10
th
International Conference on DEA Brazil2012 175
Proof.
1 1 1
( ) 1
m n m
i ij i
i j i
c N w x w
= = =
= = =
.
A DEA minimum game
The opposite side of the game can be constructed by (N,d) as follows :
1
1
( )
. 1, 0 (9)
m
k
i ik
i
m
k k
i i
i
d k Min w x
s t w w i
=
=
=
=
The optimal value d(k) assures the minimum division that player k can expect from
the game .
Theorem 5.
1
( ) 1
n
k
d k
=
(10)
Analogous to the max game, for the coalition S N, we define
1
1
( ) ( )
. 1 , 0 (11)
m
i i
i
m
i i
i
d S Min w x S
s t w w i
=
=
=
=
Theorem 6. The min game (N,d) is super additive we have ( ) ( ) ( ) d S T d S d T + for
each S, T N with S T =
Proof. By renumbering the indexes, we have { } 1,..., , { 1,..., } S h T h k = = + and
{ } 1,..., S T k = . For these sets it holds that
1 1 1
( ) ( ) ( )
k h k
ij ij ij
j j j h
d S T Min x Min x Min x d S d T
= = = +
= + = +
Thus this game starts from d(k)>0, 1,..., k n = and enlarge the gains by the coalition
until the grand coalition N with d(N)=1 is reached .
Theorem 7. ( ) ( \ ) 1 d S c N S S N
+ =
Proof. By renumbering the indexes, we can assume that { } 1,..., , {1,..., } S h N n = = and
{ } \ 1,..., N S h n = + .For this sets, it holds that
1 1 1 1 1
1 1 1 1
( ) ( \ ) ( )
(1 ) 1 1
h n n n n
ij ij ij ij ij
i i i i
j j h j j h j h
n n n n
ij ij ij ij
i i i i
j h j h j h j h
d S c N S Min x Max x Min x x Max x
Min x Max x Max x Max x
= = + = = + = +
= + = + = + = +
+ = + = +
= + = + =
Proceedings of the 10
th
International Conference on DEA Brazil2012 176
Extensions
In this section, we extend the basic model to maximal allocated benefit and minimal
allocated cost.
Maximal allocated benefit
Suppose that there are s criteria for representing benefits. Let ( 1,..., )
ij
y i s = be the
benefits of player ( 1,..., ) j j n = where u
1
( ,..., )
s
u u is the virtual weights for benefits.
Analogous to the expression (1) we define the relative score of player j to the total
scores as:
1
1 1
(12)
( )
s
i ij
i
s n
i ij
i j
u y
u y
=
= =
Player j wishes to maximize his benefits. We can express this situation by the linear
program below:
1
1 1 1
. ( ) 1, 0 ( 1,..., )
0 (13)
s
i ij
i
s n s
i ij i ij
i j i
i
Max u y
s t u y u y j n
u i
=
= = =
= =
The weights of benefits are nonnegative. A characteristic function of the coalition S is
defined by the linear program below:
1
1 1 1
( )
. ( ) 1, 0 ( 1,..., )
0 (14)
s
i ij
i j S
s n s
i ij i ij
i j i
i
c S Max u y
s t u y u y j n
u i
=
= = =
=
= =
In the program (14), the benefits of all players are nonnegative. Since the constraints
of program (14) are the same for all coalitions, we have the following theorem.
Theorem 8. The maximal allocated benefits game satisfies a sub- additive property.
Proof. For any S N andT N withS T = , we have:
1 1
1 1 1 1
( ) ( )
( ) ( ) ( ) ( )
s s
i ij i ij ij
i j S T i j S j T
s s s s
i ij i ij i ij i ij
i j S i j T i j S i j T
c S T Max u y Max u y y
Max u y u y Max u y Max u y c S c T
= =
= = = =
| |
= = +
|
\ .
| |
= + + +
|
\ .
Proceedings of the 10
th
International Conference on DEA Brazil2012 177
Minimal allocated cost
Suppose that there are m criteria for representing costs. Let ( 1,..., )
ij
x i m = be the costs
of player ( 1,..., ) j j n = where v
1
( ,..., )
m
v v is the virtual weights for costs. Player j
wishes to minimize his costs then we have:
1
1 1 1
. 1, 0 ( 1,..., )
0 (15)
m
i ij
i
m n m
i ij i ij
i j i
i
Min v x
s t v x v x j n
v i
=
= = =
= =
The weights of costs are nonnegative. A characteristic function of the coalition S is
defined by the linear program below:
1
1 1 1
( )
. 1, 0 ( 1,..., )
0 (16)
m
i ij
i j S
m n m
i ij i ij
i j i
i
d S Min v x
s t v x v x j n
v i
=
= = =
=
= =
In the program (16), the costs of all players are nonnegative. Minimal allocated costs
game satisfies a superadditive property.
Theorem 9. The maximal allocated benefit game (N,c) and min game (N,d) are dual
games, for any S N, we have ( ) ( \ ) 1 d S c N S + = .
Proof.
1 \ 1 1 1
1 1
( \ ) ( ( ( )) ( )
(1 ) 1 ( ) 1 ( ).
s s s s
i ij i ij ij i ij i ij
u u u
i j N S i j N j S i j N i j S
s s
i ij i ij
u u
i j S i j S
c N S Max u y Max u y y Max u y u y
Max u y Min u y d S
= = = =
= =
= = =
= = =
In above programs we presented a scheme for computing maximal benefit ratio and
minimal cost ratio for coalitions. Also we can compute maximal benefit ratio and
minimal cost ratio for members of coalition, using extended programs in this paper.
These results are applied by the following section.
Avoiding occurrence of zero weights and setting preference on
weights
In above programs we presented a scheme for determining the weights through the
program (13), (15). Some weight may happen to be zero for all optimal solutions. This
means that the corresponding criterion is not accounted for in the solution of the
game at all. Let us suppose that all players agree to put preference on certain criteria.
The zero weight issue can thus be solved in this way. If all players agree to
incorporate preference regarding criteria, we can apply the following "assurance
region method". For example , we set constraints on the ratio w
1
, w
i
(i=2,,m) as:
Proceedings of the 10
th
International Conference on DEA Brazil2012 178
1
i
i i
w
L U
w
, (i=2,,m) where L
i
and U
i
denote lower and upper bounds of the ratio
1
i
w
w
, respectively. These bounds must be set by agreement among all players. The
program (3) is now modified as:
1
1
1
( ) ( )
. 1, ( 2,..., )
0 (17)
m
i i
i
m
i
i i i
i
i
c S Max w x s
w
s t w L U i m
w
w i
=
=
=
= =
Similarly, we can avoid occurrence of zero weight in linear programs of maximal
allocated benefits and minimal allocated costs. Then, we have:
1
1 1
1
1
. ( ) 1,
0 ( 1,..., ) (18)
( 2,..., ), 0
s
i ij
i
s n
i ij
i j
s
i ij
i
i
i i i
Max u y
s t u y
u y j n
u
L U i s u i
u
=
= =
=
=
=
=
1
1 1
1
1
. 1,
0 ( 1,..., )
( 2,..., ), 0 (19)
m
i ij
i
m n
i ij
i j
m
i ij
i
i
i i i
Min v x
s t v x
v x j n
v
L U i m v i
v
=
= =
=
=
=
=
The (13), (15) are modified in (18), (19) respectively.
1
1 1 1
1
( )
. 1, 0 ( 1,..., )
( 2,..., ) 0 (20)
s
i ij
i j S
s n s
i ij i ij
i j i
i
i i i
c S Max u y
s t u y u y i n
u
L U i s u i
u
=
= = =
=
= =
=
1
1 1 1
1
( )
. 1, 0 ( 1,..., )
( 2,..., ), 0 (21)
m
i ij
i j S
m n m
i ij i ij
i j i
i
i i i
d S Min v x
s t v x v x i n
v
L U i m v i
v
=
= = =
=
= =
=
The (14), (16) are modified in (20), (21) respectively.
Proceedings of the 10
th
International Conference on DEA Brazil2012 179
The minimal allocated cost and maximal allocated benefit in stock
exchange of Tehran
We now apply this approach to some institution in stock exchange of Tehran. There
are 15 capital institutions in this district. Clearly, using new programs shows the better
(successful) institution. Table 1 shows specifications of companies. Our sample
covers the period from 1 March 2010 to 1 September 2010. The data were obtained
from the stock exchange of Tehran.
Table 1: specifications of companies
company First
price
Stocks Remaining
capacity
Purchase
price
Change
rate
Selling
capacity
Selling
price
1 7000 4780617160 466334 6900 2.66 466334 7000
2 1428 2924848280 10606 1450 4 10606 1482
3 2406 62871272 210306 2406 3.97 1100 2441
4 1300 1271112850 5208061 1300 4 4000 1388
5 1399 741384210 168805 1399 3.93 19800 1419
6 2920 57660229402 5949865 2920 3.98 7500 3110
7 1661 172844210 4132704 1661 3.94 10000 1800
8 921 1343516580 389782 931 3.95 3168 926
9 1957 2278030900 4162032 1957 3.98 2207 2090
10 371 626186510 282201 371 3.92 1000 380
11 3396 14947079 35831910 3396 3.98 30000 3500
12 3500 4800709200 28682483 3500 3.98 1700 3699
13 955 590990420 1421379 930 3.91 1421379 955
14 1116 1970649670 5552335 1116 3.91 19960 1145
15 3139 861200190 475297 3139 3.97 10000 3189
Now we compute the inferiority and superiority criteria for institution.
Each institution uses 2 inferiority and 2 superiority criteria. In Tables 2 and 3
inferiority and superiority for these institutions (players) are given, respectively. Also
in Table 4, the results of approach are presented.
Proceedings of the 10
th
International Conference on DEA Brazil2012 180
Table 2: Inferiority criteria for the 15 institutions of stock exchange
Player
j
x
1j
= First price
Stocks
2j
sellingcapacity
x
sellingprice
=
1 3.34643201210
13
67.5846
2 4.3346170310
11
7.314482759
3 1.511268280410
11
87.40897756
4 1.65244670510
12
4006.200769
5 1.0371965110
12
120.6611866
6 1.68368058510
13
2037.625
7 2.870944232810
11
2488.081878
8 1.2373787710
12
418.670247
9 4.45810647110
12
2126.74093
10 2.32315195210
11
760.6495957
11 5.07602816410
12
10551.21025
12 1.6802482210
13
8194.995143
13 5.64395851110
11
1528.364516
14 2.19924503210
12
4975.210573
15 2.70330739610
12
151.4166932
Proceedings of the 10
th
International Conference on DEA Brazil2012 181
Table 3: superiority criteria for the 15 institutions of stock exchange
Player j y
1j
= Change rate
2j
remainingcapacity
y
purchaseprice
=
1 2.66 66.619143
2 4 7.156545209
3 3.97 0.450634985
4 4 2.88184438
5 3.93 13.95348837
6 3.98 2.411575563
7 3.94 5.5555
8 3.95 3.421166307
9 3.98 1.055980861
10 3.92 2.631578947
11 3.98 8.571428571
12 3.98 0.45958367
13 3.91 1488.354974
14 3.91 17.17030568
15 3.97 3.13577924
Proceedings of the 10
th
International Conference on DEA Brazil2012 182
Table 4: Maximal allocated benefit and Minimal allocated cost for Institutions
of stock exchange
Player
j
Maximal allocated
benefit
Minimal allocated
cost
1 0.0458 0.3859
2 0.0689 0.0052
3 0.0684 0.0017
4 0.0689 0.0190
5 0.0677 0.0151
6 0.0685 0.2478
7 0.0678 0.0329
8 0.0680 0.0254
9 0.0685 0.0512
10 0.0675 0.0027
11 0.0685 0.0583
12 0.0685 0.1928
13 0.9166 0.9166
14 0.0673 0.0065
15 0.0684 0.0351
First company has the minimal allocated benefit and the 13th has the maximal
allocated benefit. Clearly, this scheme is a way for attaining reasonable and fair
division for institutions.
Conclusions
In this paper, we have studied the common weight issues that connect the game
solution with arbitrary weight selection behaviors of the players (institutions). In this
regard, we have proposed a method for computing maximal allocated benefit and
minimal allocated costs for institutions. An extension of problem, in which both
superiority and inferiority criteria could be considered to players (institutions), has
been discussed. Furthermore a numerical example, in which some institutions of
stock exchange evaluated with the proposed ways, has been considered.
Proceedings of the 10
th
International Conference on DEA Brazil2012 183
References
A. Charnes, W.W. Cooper, Programming with fractional function, Naval Res. Logist.
Quart. 9 (1962) 181-185.
A. Charnes, W.W. Cooper, E. Rhodes, Measuring the efficiency of decision making
units, European Journal of Operational Research. 2 (1978) 429-440.
G. Owen, On the core of linear production games, Mathematical Programming. 9
(1975) 358-370.
G.R. Jahanshahloo, F. Hosseinzadeh Lotfi, S. Sohraiee, Egoists dilemma with interval
data, Applied Mathematics and Computation. 183 (2006) 94-105.
I. Nishizaki, M. Sakawa, Solution based on fuzzy goals in fuzzy linear programming
games, Fuzzy Sets and System. 115 (2000) 105-119.
K. Nakabayashi, K. Tone, Egoist's dilemma: a DEA game, The International of
Management Science. 36 (2006) 135- 148.
M. Sakawa, Fuzzy sets and Interactive Multiobjective Optimization, Plenum Press,
New York, 1993.
R. Allen, A.D. Athanassopoulas , R.D. Dyson , E. Thanassoulis, Weights restriction and
value judgment in data envelopment analysis, Annals of Operations Research. 73
(1997) 13-34.
S. Daneshvar, M. Mansouri Kaleibar, The minimal cost-benefit ratio and maximal
benefit-cost ratio, presented at the Int Conf. Engineering System Management and
Application Sharjah, UAE, 2010.
W.W. Cooper, L.M. Seiford, K. Tone, Data envelopment analysis, a comprehensive text
with models application references and DEA- solver software, Boston: Klawer
Academic Publishers, 2000.
Acknowledgements
The authors acknowledge the support from Young Researchers Club. This research is
supported by Tabriz Branch, Islamic Azad University research budget and
sponsorship.
Proceedings of the 10
th
International Conference on DEA Brazil2012 184
22. Measurement of returns to
scale using a non-radial DEA
model
Vladimir E. Krivonozhko
National University of Science and Technology MISiS, Russia, KrivonozhkoVE@mail.ru
Finn R. Frsund,
Department of Economics, University of Oslo, Norway, f.r.forsund@econ.uio.no
Andrey V. Lychev
National University of Science and Technology MISiS, Russia, lychev@misis.ru
Abstract
There are some specific features of the non-radial DEA (data envelopment analysis)
models which cause some problems under the returns to scale (RTS) measurement. In
the scientific literature on DEA, some methods were suggested to deal with the RTS
measurement in the non-radial DEA models. These methods are based on using strong
complementary slackness conditions (SCSC) in the optimization theory. In this paper,
we propose and substantiate a direct method for the RTS measurement in the non-
radial DEA models. Our computational experiments documented that the proposed
method works reliably and efficiently on the real-life data sets.
Keywords: Data envelopment analysis; Efficiency; Non-radial models; Returns to scale
Introduction
The measurement of scale properties of frontier functions estimated using DEA models
may in some cases be problematic. In particular the non-radial DEA models (Banker
et al., 2004) can possess some specific features that give rise to estimation problems.
First, multiple reference sets may exist for a production unit. Second, multiple
supporting hyperplanes may occur on optimal units of the frontier. Third, multiple
projections (a projection set) may occur in the space of input and output variables. All
these features cause certain difficulties under measurement of returns to scale of
production units.
Banker et al. (2004) proposed a two-stage approach to determine returns to scale in
the non-radial models. Sueyoshi and Sekitani (2007) showed that this approach may
generate incorrect results in some cases. An interesting approach was proposed for
measurement of returns to scale based on using strong complementary slackness
conditions (SCSC) in the non-radial DEA models (Sueyoshi and Sekitani, 2007).
However, our theoretical consideration and computational experiments show that the
SCSC non-radial model may not be efficient from the computational point of view.
The SCSC non-radial model generates ill-conditioned basic matrices during the
Proceedings of the 10
th
International Conference on DEA Brazil2012 185
solution process, which results in strange results that do not coincide with the
optimal solution of the corresponding non-radial DEA model. This naturally
contradicts the optimization theory.
In our work we propose a two-stage approach to measure returns to scale in the non-
radial DEA models. At first stage, an interior point, belonging to the optimal face, is
found using a special elaborated method. In Krivonozhko et al. (2012) it was proved
that any interior point of a face has the same returns to scale as any other interior
point of this face. At the second stage, we propose to determine the returns to scale at
the interior point found in the first stage with the help of Banker et al. (2004) method
or using the direct method of Frsund et al. (2007).
Our computational experiments documented that the proposed approach is reliable
and efficient for solving real-life DEA problems.
The plan of the paper is to state the problem to be investigated in the next section.
Then a direct method for discovering all units belonging to the minimum face is
developed. After this, some of the numerical experiments with the SCSC non-radial
model specification are reported.
Problem statement
The non-radial DEA model can be written in the following form (Banker et al., 2004;
Sueyoshi and Sekitani, 2007)
) ( max
+ +
+ = S C S C h
T T
subject to
, 0 , 0
, , , 1 , 0 , 1
,
,
1
1
1
= =
=
= +
+
=
+
=
S S
n j
Y S Y
X S X
j
n
j
j
o
n
j
j j
o
n
j
j j
.
(1)
where ) , , (
1 mj j j
x x X . = and ) , , (
1 rj j j
y y Y . = represent the observed inputs and
outputs of production units ) , (
j j
Y X , n j , , 1. = , ) , , (
1
=
m
s s S . and ) , , (
1
+ + +
=
r
s s S .
are vectors of slack variables. The superscript T indicates a vector transpose. The
components of the objective-function vectors
+
C and
The model (1) is also called range-adjusted model (RAM) (Cooper et al., 2000).
Proceedings of the 10
th
International Conference on DEA Brazil2012 186
In the model (1), an efficiency score for unit ) , (
o o
Y X is evaluated, where ) , (
o o
Y X is
any production unit from the set ) , (
j j
Y X , n j , , 1. = . If the optimal value
*
h of the
model is equal to zero, then unit ) , (
o o
Y X is considered efficient, if 0
*
> h , then the
unit is inefficient (Banker et al., 2004).
The dual problem to the model (1) is written in the form:
{ }
0
min u Y u X v
o
T
o
T
+
subject to
, ,
, , , 1 , 0
0
+
= +
C u C v
n j u Y u X v
o
T
j
T
.
(2)
where ) , (
1 m
v v v . = and ) , , (
1 r
u u u . = are vectors of dual variables associated with the
first and the second group of constraints of problem (1),
0
u is a free variable
associated with the convex constraint.
In the papers (Sueyoshi and Sekitani, 2007, 2009), it was proposed to use strong
complementary slackness conditions from the optimization theory in order to find the
set of optimal solutions in the primal and dual space. The SCSC non-radial
model (Sueyoshi and Sekitani, 2007) is written in the following form
= +
= +
=
+ +
+ = +
= +
=
= =
= +
+ +
+ +
+
+
=
+
=
=
r i c u s
m k c v s
n j
u Y u X v
u Y u X v S C S C
C u C v
n j u Y u X v
S S n j
Y S Y
X S X
i i i
k k k
j
T
j
T
j
o
T
o
T T T
j
T
j
T
j
n
j
j o
n
j
j j
o
n
j
j j
, , 1 ,
, , , 1 ,
, , , 1
,
,
, ,
, , , 1 , 0
, 0 , 0 , , , 1 , 0
, 1 ,
,
max
0
0
0
1 1
1
.
.
.
.
.
(3)
In paper (Sueyoshi and Sekitani, 2007), the problem (3) is used in order to find the
minimum face that contains the set of optimal solutions (a projection set) on the
efficient hyper-surface of production possibility set T in the space of input and output
variables. Next, two additional fractional-linear optimization problems are determined
for measurement of returns to scale.
The SCSC non-radial model is very interesting as a theoretical idea. However, our
computational experiments show that the model (3) may generate strange results even
for medium-size problems using well-reputed optimization software. The size of the
model (3) increases significantly in comparison with the model (1). Indeed, the size of
the model (3) is equal to ) 2 2 2 ( ) 2 2 3 3 ( + + + + + + n r m n r m , where m is the number
of inputs, r is the number of outputs, and n is the number of production units.
Proceedings of the 10
th
International Conference on DEA Brazil2012 187
Remember that the size of the model (1) is equal to ) ( ) 1 ( n r m r m + + + + , and n is
usually much greater than ) ( r m+ in real-life models.
In order to investigate our suspicions about what will happen when using larger real
datasets, we conducted computational experiments using two middle-sized models.
For the first model, call it Model 1, we took the data for electricity utilities in Sweden
1987; see Frsund et al. (2007). For Model 2 we took the data from 920 Russia banks
financial accounts for January 2009.
In the computational experiments we used the well-reputed optimization software
CPLEX, the software Mathematica that is very popular among mathematicians, and the
software FrontierVision, a specially elaborated program for DEA models that enables
one to visualize the multidimensional frontier with the help of constructing two- and
three-dimensional sections of the frontier.
The main discrepancies in solving model (1) and model (3) with the help of the
program CPLEX (though theoretically optimal solutions of these problems have to
coincide) are as follows: a) efficiency scores of model (1) and model (3) may differ
significantly; b) reference sets obtained in the solution of model (3) may contain
inefficient units.
Direct method for discovering all units belonging to the minimum
face
In the linear programming problem (1), the set of optimal points
*
(the set of
projections of unit ) , (
o o
Y X on the frontier) are situated on the boundary (frontier) of
the production possibility set T. The boundary consists of a number of faces.
The solution set
*
cannot belong to different faces that do not have common points,
otherwise interior points of T would belong to the solution set. Thus, optimal
solutions of set
*
can belong only to the intersection of some faces of the set T .
Lemma 1. Let two different faces
1
and
2
of the set T intersect. Then faces
1
and
2
do not have common interior points, i.e. =
2 1
ri ri .
Corollary 1. Let two different faces
1
and
2
of the set T intersect. Then only one
from the following cases occurs:
(i) one face belongs to the other face entirely, to be precise let
2 1
, and set
2 1
is
a part of the boundary of
2
;
(ii) set
2 1
is a part of the boundary of the face
1
and face
2
, moreover the set
2 1
is also a face and its dimension is less than the dimensions of the face
1
or the
face
2
.
From the assertions written above, it follows that faces can intersect only along the
boundaries of these faces. Taking into account also that the number of faces of the
set T is finite, we obtain that there exists a face of minimum dimension
min
Proceedings of the 10
th
International Conference on DEA Brazil2012 188
containing set
*
. By virtue of the non-negativity constraints on slack variables and the
specific objective function associated with all slack variables in problem (1), set
*
may constitute only some part of face
min
.
Now, we proceed to the construction of the procedure that finds all production units
belonging to the minimum face
min
and to the set
*
.
Let problem (1), respectively problem (2), be solved by the simplex-method (Dantzig
and Thapa, 2003) and optimal primal variables ; , , 1 , {
*
n j
j
. = ; , , 1 ,
*
m k s
k
. =
} , , 1 ,
*
r i s
i
. =
+
and dual variables ; , , 1 , {
*
m k v
k
. = } ; , , 1 ,
*
0
*
u r i u
i
. = be obtained.
Determine the following index sets for primal variables
}. , , 1 , 0 | {
}, , , 1 , 0 | {
}, , , 1 , 0 | {
*
*
* *
r i s i I
m k s k I
n j j I
i y
k x
j
.
.
.
= > =
= > =
= > =
+ +
(4)
Introduce the index sets associated with the dual variables
}. , , 1 , | {
}, , , 1 , | {
}, , , 1 , 0 | {
*
*
*
0
* * *
r i c u i J
m k c v k J
n j u Y u X v j J
i i u
k k v
j
T
j
T
.
.
.
= = =
= = =
= = + =
+
(5)
Since the optimal solution is obtained with the help of the simplex-method, every
non-basic variable is equal to zero. However, some basic variables may be equal to
zero also, then the optimal solution is considered degenerate.
For the dual problem (2) all indices belonging to the set ) (
*
u v
J J J contain a basic
set of indices. However, the set ) (
*
u v
J J J may also contain non-basic indices, in
this case the dual problem (2) is considered degenerate. Thus, the following relations
hold
) ( ) (
* *
u v B y x
J J J J I I I
+
,
where
B
J is a set of optimal basic variables of the problem (1).
Variables
*
j
,
*
I j determine only one point on the minimum face. To find all points
belonging to the face
min
it is necessary to solve additional problems.
Problem
ol
Q (
*
J l ):
l l
f = max
subject to
Proceedings of the 10
th
International Conference on DEA Brazil2012 189
, , 0 , , 0
, , 0 , 1
,
,
*
*
*
*
u i v k
j
J j
j
o
J i
i i
J j
j j
o
J k
k k
J j
j j
J i s J k s
J j
Y s e Y
X s e X
u
v
=
=
= +
+
(6)
where
m
k
E e
and
r
i
E e
+
are identity vectors associated with variables
k
s and
+
i
s ,
respectively.
Notice that problem (6) includes only those variables for which the corresponding
dual constraints hold strictly equations for optimal dual variables (5). According to the
dual theorems of linear programming, this means that optimal variables of problem (6)
will also be optimal variables of problem (1).
The Procedure that finds all production units belonging to the minimum face
min
and
to the set
*
is described as follows:
Initialize sets =
o
J ,
*
J JH = , =
1
J . If the set JH is not empty, then go to the
next step. If set JH is empty then go to step 3.
Choose index JH l , if the set JH is empty, then go to step 3. Solve the problem (6).
If 0
*
>
l
f , then determine l J J
o o
= . If 1
*
=
l
f , then l J J =
1 1
. Delete index
l
from
the set l JH JH \ = . Go to the beginning of the step.
If 0
*
=
l
f , then delete index l from the set l JH JH \ = . Go to the beginning of step 2.
Set
o
J determines the set of units belonging to the face
min
. Set
1
J determines the set
of units belonging to the set
*
.
The Procedure is completed.
The standard present-day optimization software generates only one point in the
multidimensional space as an optimal solution. However, this may be not sufficient in
order to determine returns to scale on the whole minimum face, since different
vertices of the face may display different returns to scale. Any unit from set
*
J may
belong to the minimum face. The standard software generates set
*
J as a by-product.
So, the Procedure enables one to check whether some unit from set
*
J belongs to the
minimum face or not. The validity of this assertion is based on the theorems given
below.
After running the Procedure, the minimum face
min
, containing the optimal set
*
,
can be written in the form:
Proceedings of the 10
th
International Conference on DEA Brazil2012 190
. , 0 , 1
, , ) , (
)
`
= = =
o j
J j
j
J j
j j
J j
j j min
J j
Y Y X X Y X
o
o o
(7)
However, some points of the set
min
may not belong to the set
*
on the frontier.
The set
*
is written as:
{
}. ) (
) (
, , , ) , ( ) , (
*
0
*
*
*
u Y C u
X C v X C Y C
Y Y X X Y X Y X
o
T
o
T T T
o o min
+
=
=
+
+
(8)
The Procedure enables one to find all units belonging to the face
min
and to the set
*
. The validity of this assertion is based on the following theorems.
Theorem 1. Let unit
r m
t
E Z
+
be an interior point of polyhedron
r m
E
+
, let also
unit
r m
p
E Z
+
be any point of this polyhedron, which is distinct from point
t
Z . Then
unit
t
Z can be represented as a convex combination of ) 1 ( + + r m units of set and
unit
p
Z enters this combination with a nonzero coefficient.
Theorem 2. The optimal value of problem (6) is strictly positive 0
*
>
l
f if and only if
unit ) , (
l l
Y X belongs to the minimum face
min
that contains the set
*
.
In essence, Theorem 1 says that, if some unit
p
Z is a vertex of face
min
or belongs to
the face, then it is necessary that there exists such solution that variable
*
p
enters this
solution with a nonzero coefficient.
Corollary 3. If the optimal value of problem (6) 1
*
=
l
f , then unit ) , (
l l
Y X belongs to
the set
*
.
If 1
*
=
l
f , then 1
*
=
l
, this means that
*
l
is the only non-negative -variable in the
optimal basis, hence unit ) , (
l l
Y X belongs to the set
*
.
It was proved in (Krivonozhko et al., 2012) that interior points of a face have the same
returns to scale, so it is sufficient to determine returns to scale at any interior point of
this face. An interior point ) , ( Y X of the face
min
can be chosen as a strong convex
combination of units from the set
o
J , that is
. , 0 , 1
, ,
o j
J j
j
J j
j j
J j
j j
J j
Y Y X X
o
o o
> =
= =
Returns to scale of unit ) , ( Y X can be measured by two methods at least. In the first
(indirect) method (Banker et al., 2004) the BCC model is solved at the first step, the
dimension of this problem is equal to ) ( ) 1 ( n r m r m + + + + , then at the second step
Proceedings of the 10
th
International Conference on DEA Brazil2012 191
two additional problems are solved, the dimension of these two problems coincides
with the dimension of the BCC problem. Returns to scale is determined with the help
of dual variables.
In the second (direct) method (Frsund et al., 2009), an intersection of the set T and
two-dimensional plane is constructed with the help of some algorithms at the first
step. At the second step returns to scale of any point on the graph is measured by
using derivatives of this graph.
Computational experiments
However, the question arises: What is the payment for discovering all vertices of the
face
min
?
We calculated the number of iterations accomplished by CPLEX software in order to
solve problem (1) for all units ) , (
j j
Y X , n j , , 1. = of the Model 1. This number is
equal to 2876 = iterations. Next, we calculated the number of iterations
accomplished by CPLEX in order to solve problems (6) for all units ) , (
j j
Y X ,
n j , , 1. = . This number makes up 62 . 0 1782= = iterations in the problems of the
type (6). That is really not a heavy burden, even for an ordinary notebook.
Finding. The non-radial DEA models possess some specific features. However this is a
not a problem in order for find returns to scale of the set of optimal points. For this
purpose it is sufficient to find an interior point of the minimum face that contains the
set of optimal solutions of problem (1) and to determine returns to scale of this
interior point. Such solution requires much less computations than to solve problem
(1) for the specific unit.
Conclusions
In Sueyoshi and Sekitani (2007; 2009), a method was proposed in order to measure
returns to scale in the non-radial DEA models using strong complementary slackness
conditions. However, the size of the SCSC non-radial model (3) increases significantly
in comparison with the model (1). In particular, for the banks data set the size of basic
matrices during the solution process becomes ( 1840 1840 ) instead of ( 7 7 ) in the
model (1). In addition, some constraints in model (3) do not make sense from an
economic point of view. Unreliable solutions may follow due to these reasons.
In our method, it is sufficient to solve several problems of the form of model (1),
however such problems have much less variables than problem (1). Thus, the
proposed approach is reliable and efficient for solutions of real-life problems.
Moreover, it was stressed in Sueyoshi and Sekitani (2007) that the method of Banker et
al. (2004) cannot always generate reliable results because of difficulties described
above in the non-radial DEA models. However, the method of Banker et al. (2004)
can also be used to measure returns to scale from our point of view. For this purpose,
it is sufficient to take an interior point of the minimum face, which can be found by
the method proposed in this paper, after this one can use the method proposed
in Banker et al. (2004).
Proceedings of the 10
th
International Conference on DEA Brazil2012 192
References
Banker, R. D., Cooper, W. W., Seiford, L. M., Thrall, R. M., and Zhu, J. (2004) Returns
to scale in different DEA models. European Journal of Operational Research 154(2),
345362.
Cooper, W. W., Park, K. S., and Pastor, J. T. (2000) RAM: A range adjusted measure of
efficiency. Journal of Productivity Analysis 11: 542.
Dantzig, G. B. and Thapa, M. N. (2003) Linear programming 2: Theory and Extensions.
New York: Springer-Verlag.
Frsund, F. R., Hjalmarsson, L., Krivonozhko, V. E., and Utkin, O. B. (2007) Calculation
of scale elasticities in DEA models: direct and indirect approaches. Journal of
Productivity Analysis 28: 4556.
Frsund, F. R., Kittelsen, S. A. C., and Krivonozhko, V. E. (2009) Farrell revisited
Visualizing properties of DEA production frontiers. Journal of the Operational
Research Society 60: 15351545.
Krivonozhko, V. E., Frsund, F. R., and Lychev, A. V. (2012) Returns-to-scale properties
in DEA models: the fundamental role of interior points. Journal of Productivity
Analysis 38(2): 121130.
Sueyoshi, T. and Sekitani, K. (2007) Measurement of returns to scale using a non-radial
DEA model: A range-adjusted measure approach. European Journal of Operational
Research 176: 19181946.
Sueyoshi, T. and Sekitani, K. (2009) An occurrence of multiple projections in DEA-
based measurement of technical efficiency: theoretical comparison among DEA
models from desirable properties. European Journal of Operational Research 196: 764
794.
Acknowledgements
The research is carried out with financial support of the Programme of Creation and
Development of the National University of Science and Technology MISiS. The
reported study was partially supported by RFBR, research projects No.11-07-00698 and
No.12-07-31136.
Proceedings of the 10
th
International Conference on DEA Brazil2012 193
23. Multimethodology applied to
the assessment of
municipalities health
performance in Brazil
Marcos Pereira Estellita Lins
Production Engineering Program POLI-COPPE/UFRJ, Post Office Box 68507, CEP 21945-970 - Rio
de Janeiro, RJ, Brasil, estellita@pep.ufrj.br
Sergio Orlando Antoun Netto
Department of Cartography/Engineering School/UERJ - COPPE/UFRJ, 524 So Francisco Xavier
Street, 5
st
floor - Rio de Janeiro CEP 20550-013, sergioantoun@gmail.com (corresponding author)
Abstract
This work presents a strategic approach to the formulation and structuring of health
problem in 5565 Brazilian municipalities using Concept Map and Data Mining. In
addition, using the Operational Research Method called DEA (Data Envelopment
Analysis) for the determination of counties goals and performance indicators and the
establishment of benchmarks for regulating the public health sector in Brazil. The
analytical results will corroborate to a progressive incentive for greater productivity in
the Brazilian municipalities health.
Keywords: Data Envelopment Analysis (DEA), Data Mining, Concept Map,
Multimethodology, Public Health
Introduction
Data Envelopment Analysis (DEA) has been used to support public policy in
regulatory processes because of its ability to handle multidisciplinary problems and
deliver efficient targets. However, public policy is also characterized by its inherent
complexity, which requires taking a particular arbitrary perspective regarding the
system purpose and relevant factors.
As a comparative method in its essence, DEA requires that homogeneity prevail as
one of the key issues for the Decision Making Units under assessment. Nevertheless,
not much research has been devoted to either complexity or homogeneity in DEA
applications. One reason could be the small size of most databases in regulatory
problems, which does not allow the application of clustering procedures to data.
This work presents a strategic approach to the formulation and structuring of the
health problem, taking advantage of the large size database comprising 5565 Brazilian
municipalities. First it uses Concept Maps to represent the many factors that
characterize the complex issues involved in health performance management in Brazil.
Then heterogeneity is investigated and a Data Mining technique is applied to find
appropriate clusters of around a thousand municipalities.
Proceedings of the 10
th
International Conference on DEA Brazil2012 194
Actually, the philosophy behind these two approaches are emphasized by the US and
UK Operational Research schools, as shown in the respective societies websites; the
first prescribes the use of analytics in order to explore different approaches to the
database, while the second includes a critical step that consists of the analysis of
different approaches to problem structuring. Thus the use of multiple methodologies
is an inherent trend in both schools of OR.
This work employed a strategic approach to the formulation and structuring of
problems through the use of conceptual maps. Actually, in solving problems of
systems, "Soft" and "Hard" tools can be employed, and both may be used in a
complementary manner (Oral & Reisman, 2005). It improves the modeling of
efficiency in many ways, since stakeholders can acquire an accounting of the whole
system, while putting the quantitative indicators in a comprehensive and still
accessible context.
The application to municipality health comprises main causes of death in three
different age ranges, each one assessed by a different DEA model. Different profiles
according to life threatening and care provided by governmental authorities emerge
from the analysis.
Methods
According to Rosenhead and Mingers (2001), Concept Map is important to make
explicit when we are seeking proposals for dealing with complex problems and not
just solving a simplified part of a problem under a particular perspective. In this
former approach the structuring of matters, issues and situations is one of the stages of
the modeling at the very beginning of the decision-making process.
For Weis and Indurkhya (1999), data mining is the search for valuable information in
large databases, as a cooperative effort between men and computers. Men design
databases, describe problems and define goals. Computers check data and look for
patterns that match the goals established by men.
The Data Envelopment Analysis (DEA) has been used in the calculation of
performance indicators and to establish benchmarks for regulation of public sectors.
The method lends itself to use in multidisciplinary issues and multiagents may be used
in the estimation of production frontier functions or for incorporating the opinion of
specialists, like a multicriteria method.
Multimethodology is the "art" of use, combined, more than a methodology or part of
methodologies, in order to consider, in the best way, the various problems, as
proposed by Mingers and Brocklesby (1997). The multimethodology approach
assumes that there is a specific method that is more appropriate, but that all methods
have advantages and disadvantages that can be balanced.
Results and discussions
We developed the formulation and structuring of Public Health in Brazilian
municipalities using information from Public Health experts reported in legal
documents as Special Checking Accounts and press interviews with government
authorities as Federal Prosecutors, National Health Council, Parliament House, Minister
Proceedings of the 10
th
International Conference on DEA Brazil2012 195
for Science, Technology and Innovation, Comptroller General of the Union, Board of
Policies, Performance and Competitiveness Management linked to the Governing of
the Republic Presidency Council and members of non-governmental sectors as SUS
National Audit Department, SUS Auditors National Union and Brazilian Collective
Health Association. The formulation and structuring of Public Health in Brazilian
municipalities can make use of the concept maps shown in Antoun Netto (2012).
To implement the clustering technique we employed the Profile of Brazilian
Municipalities Database (MUNIC), from Brazilian Institute of Geography and Statistics
(IBGE), covering the year 2009. It consists of a survey of Municipal basic information
published annually by IBGE with information about 16 themes of 5565 Brazilian
municipalities, such as: human resources, legislation and municipal planning tools,
education, culture, sports, housing, transportation, health, environment, among others.
The software used for clustering was the Waikato Environment for Knowledge
Analysis (WEKA), a set of algorithms and programs produced by the University of
Waikato in New Zealand. The Weka is implemented in the Java language, which has
as main characteristic its portability, this way you can use it on different operating
systems. The WEKA is free software, i.e. is under GPL license and is available at http:
//www.cs.waikato.ac. nz /ml/weka.
In the Table 1 is a summary with the main feature of each grouping set by clustering
technique, using the WEKA.
Table 1: Main feature of each cluster
Cluster Main feature
1 High Public consortia participation with the Federal and State Government
2 Participation of variables near the average of the municipalities examined
3
Municipalities have an improved infrastructure in the area of health
compared to those of other groups, though not necessarily the ideal
4 Considerable participation of public Consortium between municipalities
5
Participation of the variables below average of the municipalities
examined
Antoun Netto (2012) has more details about the clustering technique in Brazilian
municipalities.
For the determination of municipal development targets and indicators in the area of
health we will consider clusters 1 and 3. Such a choice results from the following
facts:
a. Regarding Cluster 1, we want to investigate if municipalities that present high
participation of State and Federal Governments resources are employing those
resources efficiently and effectively.
Proceedings of the 10
th
International Conference on DEA Brazil2012 196
b. Analysis of Cluster 3 is particularly interesting because this cluster is composed
of municipalities with better infrastructure, compared to the other clusters. In this
work we will considered only the municipalities with population greater than or equal
to 50,000 for cluster 1 and 100,000 for cluster 3, aiming at minimizing possible
anomalies in the DEA models to be implemented.
To analyse the level of human development in a particular country we need to
conduct studies about various social indicators. A very important indicator for the
analysis is infant mortality, which corresponds to the number of children who die
before reaching one year of age. According to the IBGE, there has been a 50%
reduction in child mortality between 1990 and 2008, from 47 to 23.3 per 1,000 live
births. The proportion, however, is still not low according to the standards of the
World Health Organization (less than 20 per thousand).
It is worth mentioning that in Nations like Sweden, Norway and Canada the ratio is,
respectively, 3, 10.4 and 4.63 per 1,000 live births. The index can vary between
different cities, States and regions, as well as on the basis of income, i.e. the low-
income sector presents always higher child mortality.
Another important indicator refers to diseases of the circulatory system that are the
leading causes of death in Brazil and in the world. The two main causes of mortality
within this group are ischemic heart disease and cerebrovascular disease.
The risk factors for cardiovascular diseases are very prevalent in urban populations,
currently being cardiovascular risk factors: smoking, overweight, excess meat
consumption of fats, physical inactivity, alcohol abuse, etc.
Mortality ratios are not distributed homogeneously in space. The inequality in
mortality from diseases of the circulatory system, whether on a national, regional or
local basis, is strongly associated with social and economic inequality and the
distribution of health equipment.
Finally, in view of violence growth in Brazilian municipalities, search in this study
consider the indicator of mortality from external causes, which include, according to
the World Health Organization, all types of accidents, suicides and homicides. High
rates of mortality are associated with higher prevalence of risk factors specific to each
type of external cause. Traffic accidents, homicides and suicides respond together for
about two-thirds of deaths from external causes in Brazil. The rates are considerably
higher in the young adult population, especially males.
It is worth emphasizing that the country's economic growth in recent years and
spending on public security higher than in some developed countries, such as
Germany and Spain, have not contributed to the reduction of homicides in Brazil,
according to the Yearbook of Brazilian Public Security. Given the above, the choice of
variables sought to contemplate the dimensions of child mortality, external causes and
diseases of the circulatory system. It is important to report that restrictions on weights
were not introduced in the templates.
Table 2 shows the variables to be considered within the model. The choice of
variables in DEA is a key issue for the determination of efficiency.
Proceedings of the 10
th
International Conference on DEA Brazil2012 197
Table 2 Variables identified in the case study
DIMENSION
External Causes
Circulatory
Diseases
Infant Mortality
INPUT
Deaths Chapter XX
- External causes of
morbidity and
mortality
Deaths Chapter IX
- Circulatory
Diseases
Deaths Chapter XVI -
Some disorders
originating in the perinatal
period
Deaths Chapter XVII -
Congenital malformations,
deformities and
chromosomal
abnormalities
OUTPUT
Resident Population
20 to 29 years old
Resident
Population
40 to 59 years old
Resident Population
< 1 year old
In determining targets and indicators of municipal development in the area of health,
each municipality is represented as a DMU (Decision Making Unit) endowed with
autonomy. We adopted a design of hierarchical modeling in 2 (two) stages, as
proposed by Lins et al. (2007) and Ozcan et al. (2010), namely:
In the first stage, we considered the output-oriented VRS basic models for
the three dimensions: infant mortality, circulatory diseases and external
causes.
In the second stage, we determined the final efficiencies for these
municipalities, using the weighted average.
In Table 3, we will present the first stage efficiency of municipalities in various
dimensions, by cluster, through the DEAFrontier software, developed by Professor Joe
Zhu of WPI (Worcester Polytechnic Institute).
Proceedings of the 10
th
International Conference on DEA Brazil2012 198
Table 3- Efficient Municipalities by dimension in the first stage
Dimension
Circulatory Diseases External Causes Infant Mortality
C
l
u
s
t
e
r
1
Manaus (AM)
Abaetetuba (PA)
Barcarena (PA)
Monte Santo (BA)
Manaus (AM)
Santarm (PA)
Monte Santo (BA)
Manaus (AM)
Abaetetuba (PA)
Belm (PA)
Frutal (MG)
Porto Ferreira (SP)
Almirante Tamandar (PR)
Concrdia (SC)
Laguna (SC)
Santa Cruz do Sul (RS)
Viamo (RS)
Aparecida de Goinia (GO)
C
l
u
s
t
e
r
3
Santana (PA)
Belo Horizonte (MG)
Uberlndia (MG)
So Paulo (SP)
Valinhos (SP)
Jaragu do Sul (SC)
Juiz de Fora (MG)
Pouso Alegre (MG)
So Paulo (SP)
Juiz de Fora (MG)
Maca (RJ)
Ourinhos (SP)
Salto (SP)
So Paulo (SP)
Curitiba (PR)
Porto Alegre (RS)
Santa Maria (RS)
In Table 4 we present the benchmarks municipalities resulting from aggregated
efficiencies of the three dimensions.
Proceedings of the 10
th
International Conference on DEA Brazil2012 199
Table 4- Benchmarks Municipalities
Cluster Benchmark
1 Manaus (AM)
3 So Paulo (SP)
Conclusions
This work illustrated the use of conceptual maps for the design and structuring of the
Brazilian public health assessment. Knowledge mapping was used to establish a
context for the qualitative and quantitative variables, to be used in Data Mining and
Data Envelopment Analysis, respectively.
The approach used encompasses different major diseases that are prevalent and
representative of death risks at three different age ranges. The methodology intends to
evaluate three different processes and respective sets of variables that are not directly
related, and thus assessed in three different DEA models, constituting a first stage.
Differently from other approaches, we restrained the analysis to health closures,
represented by the worst situation, namely death occurrence. We intend to extend the
method to include other minor diseases.
Other recommendations for future research consider the evolution of efficiency along
the period 2009-2012, since the first coincided with the beginning of the current
mandate of mayors in Brazilian municipalities. We suggest the use of Malmquist
index to evaluate the actions and programs in the health municipalities during their
mandate, considering the change in the health performance of municipalities between
the two periods of time (2009 and 2012).
Finally, it is important to facilitate the validation of these results by qualified
healthcare professionals, regarding the potential for improvements and the
observation of the recommendations in the National Policy of Health Promotion,
whose general objective is to promote the quality of life and reduce vulnerability and
health risks related to its determinants and conditioning ways of living, working
conditions, housing, environment, education, leisure, culture and access to essential
goods and services.
References
Antoun Netto, S. O. (2012). O uso de Multimetodologia para a determinao de Metas
e Indicadores de Desenvolvimento Municipal na rea da Sade. Tese (doutorado)
UFRJ/ COPPE/ Programa de Engenharia de Produo.
Lins, M. E. et al. (2007). O uso da Anlise Envoltria de Dados (DEA) para avaliao
de hospitais universitrios brasileiros. Cincia & Sade Coletiva, 12(4): 985-998.
Mingers, J. ; Brocklesby, J. (1997). Multimethodology: Towards a Framework for
Mixing Methodologies, Omega, International Journal of Management Science, 25, 5
(pp. 489509).
Proceedings of the 10
th
International Conference on DEA Brazil2012 200
Ozcan, Y.A.; Estellita Lins M.P., Lobo, M.S.C.; Silva A.C.M.; Fiszman, R.; Pereira, B. B.
(2010) Evaluating the Performance of Brazilian University Hospitals Annals of
Operations Research; ISSN:0254-5330; Springer.
Reisman, A.; Oral, M. (2005). Soft systems methodology: A context within a 50-year
retrospective of OR/MS." Interfaces 35.2, 164-78.
Rosenhead, J.; Mingers, J. (2001). Rational analysis for a problematic world: problem
structuring methods for complexity, uncertainty and conflict. 375p. 2. ed. West Sussex:
John Willey & Sons.
Weis S. M. e Indurkhya, N. (1999) Predict Data Mining; Morgan Kaufmann
Publishers, Inc.
Proceedings of the 10
th
International Conference on DEA Brazil2012 201
24. On the Measurement of Social
Efficiency in Microfinance
Institutions
Breno Sampaio
Department of Economics, Federal University of Pernambuco, brenorsampaio@gmail.com
*
Lcio Silva
Department of Production Engineering, Federal University of Pernambuco,
lucio_camara@hotmail.com
Abstract
Do Microfinance Institutions trade-off between financial and social efficiencies?
Previous evidence shows that a positive correlation exists. In this paper we argue that
previous results are biased because they do not account for differences between
countries. We propose a decomposition in which efficiency is broken down into a
MFI and a country component. Results are quite different from the ones previously
presented, given we isolate any country-specific characteristic that might bias
efficiency estimates. The correlation we estimate between financial and social
efficiency is about 35% of what was previously reported. Also, this correlation varies
within the financial efficiency distribution. Looking at profits, we obtain that they are
indeed correlated with efficiencies; the financial efficiency correlation is positive but,
contrary to previous findings, the social efficiency correlation is negative. Finally, we
show that countries indeed face different frontiers implying that efficiencies should be
assessed by comparing units within the same country.
Keywords: DEA, microfinance institutions (MFIs), social efficiency
Introduction
Microfinance Institutions (MFIs) are nowadays present in most countries around the
world. Their main purpose is to provide credit to the poor who are excluded from the
traditional banking system and, for this reason, they are widely recognized for their
contribution to the development of financial markets in the developing world.
Contrary to what is observed in the literature about the traditional banking sector, little
is known about the performance of MFIs, specially considering their mutual purpose
of being financially and socially efficient (Gutirrez-Nieto et al., 2009). Berguiga
(2009), for example, concludes after an extensive review that MFIs do not trade-off
between being financially and socially efficient, that is, the literature actually suggests
that these two requirements are compatible and may even be complementary. This
question is also addressed in Gutirrez-Nieto et al. (2009). They analyze both the
financial and social efficiency of Microfinance Institutions using data for 89 MFIs
*
Corresponding author
Proceedings of the 10
th
International Conference on DEA Brazil2012 202
located in several countries around the world. Additionally, they also look at the
relationship between these efficiencies, and how they relate to other indicators, such
as profitability, and the type of institutions - Non-Governmental Organization (NGO)
and non-NGO. Their main conclusions are that there is a positive, but low, correlation
between social efficiency and financial efficiency, NGOs are more socially efficient
than other MFIs that are run under other organizational structures, and, more
importantly, the geographical area of activity of the MFI was found to be important in
determining efficiency measures.
In this paper, our focus is to look at the interaction between financial and social
efficiencies but taking into account differences between countries, which is a
substantial improvement compared to previous studies on the subject. More
specifically, we invoke one of the most basic assumptions in DEA applications, which
require that the units being considered in the analysis must have the same production
process. That is, when comparing MFIs in different countries one must take into
account that significant differences may exist in terms of labor market conditions for
women, for example, which if not correctly accounted for may introduce significant
bias in the efficiency coefficients ultimately leading to misinterpretations of
inefficiencies given by country-specific limitations. Hence, to account for country-
specific characteristics when estimating MFIs financial and social efficiencies and to
look at how different are production possibility curves in different countries, we
propose a decomposition of efficiency similar to that taken by Portela and
Thanassoulis (2001) and recently by Amores and Contreras (2009), in which overall
efficiency is broken down into a MFI (financial and social) efficiency and a country
(financial and social) efficiency. This idea is similar to estimating a regression model
with country fixed effects to control for any unobserved country characteristics (see
Sampaio (2012)).
After this brief introduction, in the next section we present the data. In the third
section we describe the methodology and in the fourth section we present the results.
Finally, in the fifth section we conclude.
Data
The data source, as in most of the recent studies looking at MFIs (Gutirrez-Nieto et
al. (2009), among others), comes from the Microfinance Information Exchange (MIX)
database. The sample we use in this article was obtained by first dropping all MFIs
that had missing values on any of the variables used in the analysis and then
restricting our sample to include only countries that had at least 20 MFIs, given we
want to construct country-specific frontiers. All the data we use consider only the year
of 2008. This leaves us with 483 MFIs distributed within 14 countries. The two largest
countries, according to the number of MFIs analyzed, are Russia, with 78, and India,
with 57 microfinance institutions.
Proceedings of the 10
th
International Conference on DEA Brazil2012 203
Methodology
The Data Envelopment Analysis (DEA) Model
We estimate DEA efficiency scores using the CCR model of constant returns to scale,
which has been the model used in this specific literature. Given the main purpose of
our article is to improve upon what has been done in the literature, we follow the
input and output selection of Gutirrez-Nieto et al. (2009). There are three inputs: total
assets (A), represented by the total of all net asset accounts ($), taken directly from the
Mixmarket dataset; Operating Cost (C), defined as expenses related to operations,
such as transportation, office, and personnel expenses; and number of active
employees (E) employed by the MFI. The two financial outcomes considered are:
gross loan portfolio (L), defined as the outstanding balance of the all of the MFI's
outstanding loans including current, delinquent and restructured loan, but not loans
that have been written off (it does not include interest receivable); and financial
revenue (R), defined as the revenue generated from the gross loan portfolio and from
investments plus other operating revenue. The two social outcomes considered are:
number of active borrowers who are female (W); and an indicator of benefit to the
poorest (P), defined through the average loan balance per borrower and the per
capita Gross National Income.
The number of active borrowers who are female is taken directly from the Mixmarket
dataset. We interpret this as a measure of female empowerment at home or, in some
degree, within her society. The indicator of benefit to the poorest is another social
outcome of MFIs widely considered in the literature, since MFIs are partially designed
to provide financial services to the poorest in a given country. We follow Gutierrez-
Nieto et al. (2009) and define an indicator to benefit the poorest as the ratio between
the average loan balance per borrower and per capita Gross National Income
(pcGNI). The intuition is that wealthier individuals are able to borrow larger amounts.
Thus, if the average loan balance per borrower is low, then a larger fraction of poor
individuals are being financed. The division of the average loan balance per borrower
by per capita Gross National Income (pcGNI) is only intended to capture differences
in countries average per capita income, hence is it a normalization. This indicator is
then normalized again to lie on the interval [0,1] and inverted (one minus the
indicator), such that a number close to 1 represents a higher percentage of loans to
the poor. Finally, this new indicator is multiplied by the number of active borrowers,
such that one can construct an approximate measure of the number of poor
borrowers. In this paper we consider two main empirical DEA specifications: the
financial frontier, represented by inputs (Assests, Costs and Employees) and outputs
(gross loan portfolio and financial revenue), named as ACE-LR; and the social frontier,
represented by the same inputs but considers as outputs (number of active borrowers
who are female and the indicator of benefit to the poorest) and is named as ACE-WP.
The Efficiency Decomposition
The decomposition we adopt in this paper follows closely the one proposed by
Portela and Thanassoulis (2001). For our case, we will first compare MFIs within the
same country, such that country characteristics are isolated from final estimates,
Proceedings of the 10
th
International Conference on DEA Brazil2012 204
certainly not introducing any bias in the DEA efficiencies. Then, we will compare all
MFIs in a way that differences between countries production possibility curves will be
identified. Figure 1 shows how the decomposition works graphically. For simplicity
assume there are only two countries A and B and that there is only one input and one
output. Each triangle is a DMU, in this case a MFI, that is established in country A and
each square is a MFI established in country B. Solid lines represent the DEA efficiency
frontiers for each country (which we label as local efficiency - EFL), while the dashed
line represent the DEA efficiency frontier considering both countries analyzed (which
we label as global efficiency - EFG).
Now consider the MFI labeled as Z established in country B. Its DEA efficiency with
respect to other MFIs in the same country may be assessed by
1
, which we label as
MFI Efficiency (MFIE) and, in this case, is clearly smaller than 1 making Z an
inefficient MFI. This MFI, however, is not as inefficient as it would appear if all MFIs
efficiencies of all countries were assessed together, which we label as MFI Overall
Efficiency (MFIOE). In this case, Zs efficiency would be
2
<
1
. Thus, when pulling
every MFI together in a unique production frontier, previous analysis on this specific
topic, such as Gutirrez-Nieto et al. (2009), are introducing a negative bias in all MFIs
efficiencies.
Figure 1: Decomposition of Overall Efficiency
Proceedings of the 10
th
International Conference on DEA Brazil2012 205
Now we turn to the measurement of how efficiency frontiers vary within countries.
Note that the coefficient obtained by Gutirrez-Nieto et al. (2009), namely
2
,
represents two effects. First, it represents the lack of effort or ability of MFI Z that
prevents it from being efficient in comparison to all other MFIs in the same country.
And second, the possibilities of the country to provide/produce outputs, which we
label as Country Efficiency (CE). The CE, which is equal to the distance between each
country's local efficiency frontier and the global efficiency frontier,
2
, can be
obtained by
2
=
=
1
Results and discussions
Table 2 presents average efficiencies by country. First note that, as expected, MFIE are
always greater than MFIOE. Also, MFIE are not only a shifted version of MFIOE, that
is, there are substantial changes in the country ranking order if ones uses local or
global efficiencies measures. This is another evidence that comparisons between
financial and social efficiencies using global efficiency measures might be significantly
misleading.
Table 1: Financial and Social Efficiencies by Countries
Country Financial Efficiency Social Efficiency
MFIF MFIOE MFIF MFIOF
Bangladesh 0.895 0.679 0.831 0.425
Bolivia 0.941 0.736 0.475 0.116
Brazil 0.876 0.782 0.478 0.109
Ecuador 0.907 0.752 0.521 0.167
Ghana 0.862 0.577 0.433 0.300
India 0.892 0.723 0.666 0.642
Indonesia 0.861
0.689
0.382 0.179
Mexico 0.895 0.660 0.541 0.214
Nepal 0.894 0.682 0.550 0.423
Nicaragua 0.939 0.741 0.565 0.132
Peru
0.913 0.741 0.612 0.154
Philippines
0.835 0.643 0.547 0.197
Russia
0.870 0.837
0.291
0.089
Tajikistan 0.944
0.784 0.586 0.082
Proceedings of the 10
th
International Conference on DEA Brazil2012 206
To look at the correlation between financial and social efficiencies, we use a different
strategy than the one adopted by Gutirrez-Nieto et al. (2009). We estimate the
correlation between efficiencies by the following fixed effects model
= +.
where i and c stands for, respectively, the MFI and the country the institution is
located at; is the parameter of interest; and
and
F of the
population distribution F from the efficiency scores
{ }
, 1,2,...,
i
i n = = defined by
DEA. Bias-corrected efficiency scores
{ }
* *
, 1,2,...,
i
i n = =
are generated by
F in the
pursuit of
*
.
By applying DEA, for instance the BCC model (Banker et al., 1984), we obtain
i
efficiency scores
{ }
1
0 1
n
i i
i
=
for every DMU
i
. In the following analysis we presume
input orientation is applied.
Proceedings of the 10
th
International Conference on DEA Brazil2012 228
Based on the efficiency scores
{ }
( )
1
n
i
i
=
= assigned to the sample DMUs, or, the
weighted efficiency scores
{ }
( )
1
n
i
i
=
=
, where
=
for every DMU. Every sequence of pseudo-numbers originates from every single
efficiency score or from a combination of a selected efficiency score and the average
scores of the sample.
{ }
*
to produce 1,2,...,
o
T x
=
or
1 *
1
(1 ) to produce
1,2,..., ; 1,2,...,
n
ud ud
i i i io
i
z z n x
T
i n
=
+
= =
`
)
(1), where
{ }
* *
min ,
io i i
x x
=
In addition,
2
*
( ) ( , ) T x N se
~
and
* 2
( ) ( ( ), ( ( ) ) )
ud
i i i
T x N cv
- - ~ (2)
1
1
( ) (1 )
n
ud ud
i i i
i
z z n
=
= +
-
where
z is a user-defined score that denotes the magnitude of a single efficiency score, and
complementary of the sample mean efficiency scores, on the generation of a
trimmed random sequence of data (scores). In fact, there is inherent dependency
between the efficiency scores of the sample DMUs that is due to the comparative
assessment procedure applied through DEA.
Moreover,
*
x represents the randomly generated data, the
*
io
x
expresses selected
randomly generated replicas of the efficiency score for the -number
4
elements of the
sequence, and cv stands for the coefficient of variation which is user-defined.
The bias-corrected efficiency score for every DMU is estimated as follows
* *
( ) 1,2,..., ; 1,2,...,
i io
s x i n
= = =
MPBC
i
i i
bias =
(4) where
*
[0,1)
i
.
The standard error of the proposed MPBC method is
1/2
1 * 2
1
[ ( )]
MPBC
i
io i
se x s
=
=
`
)
- (5)
where
1 *
1
( )
i io
s x
=
=
-
Taking into account equations (3) and (5), and also a large enough sequence size ( )
of pseudo-data so that the asymptotic normality of their distribution to be assured by
4
1,..., = where denotes a proper length for the sequence rather than
{ } { }
*
1 1
n n
f f f
i i
i i
p
= =
=
, both
f
and
f
p express fixed values.
Proceedings of the 10
th
International Conference on DEA Brazil2012 229
the Central Limit Theorem, the confidence intervals of the bias-corrected efficiency
scores are constructed using Students t distribution
* (1 /2) * (1 /2)
( 1) ( 1)
,
MPBC MPBC
a a
t se t se
+
+
(
(
(6)
we prove that
{ }
*
Pr , 1,2,..., 0
ub
i i
ob i n = =
(7)
and
{ }
1 *
1
Pr , 1,2,...,
L
ub
l i i
l
L ob i n e
=
= =
(8), where the superscript ub
stands for the upper bound of the confidence interval of the bias-corrected efficiency
scores.
Acknowledging the inherit randomness in the proposed method, all the provided
proofs or statements result from iterative procedures. In formula (8), the probability,
that is the average of L=1000 iterations, is equal to an infinitesimal value.
The inherent randomness in the proposed method is regarded as a drawback because
it is a source of instability for the obtained results when the method is applied
repeatedly. To overcome this drawback, a stabilization parameter is introduced in
the procedure that eliminates up to 99% the variation of the bias-corrected scores. The
parameter expresses the number of iterations for the formulas (1)-(6). The reported
results are average scores.
The proposed method
5
for dealing with uncertainties in DEA is expressed by the
following formula
*
( , , , , , ,var )
ex ex MPBC
f cv z n
(9)
where denotes the level of significance, cv is preferably a low-variance parameter (
1 cv < ), 0 1 z , about the interval of we have already discussed, and should be
at least equal to unity, which means that no iteration to the proposed bias-correction
procedure is applied.
In formula (9), two exogenous parameters
ex
n and var
ex
are included; they denote the
number of DMUs in the original sample and the number of input and output
variables, respectively, that are utilized to define the efficiency scores through DEA.
Based on a numerical example and on results that are tested through Monte Carlo to
eliminate randomness, the proposed method yields better estimators (
*MPBC
) for the
population efficiency scores ( ) than the DEA Bootstrap (
*Boot
s to
s increases against
*Boot
(10)
The efficiency estimators obtained by the proposed method (MPBC) should converge
more to the true efficiency scores of the DMUs under evaluation than the efficiency
scores estimated by the smoothed Bootstrap
* * MPBC Boot
<
(11)
In order to test the consistency of the MPBC efficiency estimators against the efficiency
estimators obtained by the smoothed Bootstrap, we compare the efficiency scores
assigned to the 6 main samples (10, 30, 40, 50, 60, and 80 DMUs) with those of the
population sample that consists of 100 DMUs.
In the case we employ a binary scale to distinguish the success from the failure of
MPBC to yield more consistent efficiency estimators than the smoothed Bootstrap, we
utilize the following criterion
1 * *
1
,
1 if Pr 0.5
Convergence
0 otherwise
n
MPBC Boot
i i i
i
i z
ob n
=
| |
` |
\ . )
< >
=
(12)
The effects of the variability of the user-defined and the exogenously-fixed
parameters, illustrated in formula (9), on the consistency of the MPBC efficiency
estimators in comparison with the smoothed Bootstrap efficiency estimators, are
summarized in the regression models (13) and (14). The construction of the former
regression model drew on efficiency convergence scores obtained for =[0, 2, 10], and
the latter regression model was based solely on scores estimated for =10.
The independent variables used in our analysis were: (1) n, (2) variables, (3) z, (4)
binz, (5) binzn, and (6) binzvariables. The parameters , cv and are regarded as
constant ( =0.05, cv =0.25, =100). The independent variable binz is a binary variable
where binz=0 if [0.0, 0.5) z , and binz=1 if [0.5, 1.0] z . The regressors binzn and
binzvariables are mixed variables.
In practice, let [0, 2, 10] = , then
** ** **
**
% (65.219) (0.456) (0.316)
(12.921)
Convergence binzn
binz
n = +
(13)
**
p <0.01, R
2
=0.448, adjR
2
=0.445, F=141.628, p
F-stat
=0.00
Proceedings of the 10
th
International Conference on DEA Brazil2012 231
If the user selects a [0.0, 0.5) z , the MPBC method yields more consistent efficiency
scores than the smoothed Bootstrap for samples with up to 33 DMUs, regardless of
the number of variables incorporated in the identification of the DEA efficiency scores.
The sample size is extended to 36 DMUs when a [0.5, 1.0] z is selected for the bias-
correction procedure.
In the case of 10 = , then
** ** **
*
% (70.899) (0.537) (0.259)
(1.047)
Convergence binzn
binzvariables
n = +
(14)
**
p<0.01,
*
p<0.05, R
2
=0.513, adjR
2
=0.505, F=68.089, p
F-stat
=0.00
For a [0.0, 0.5) z , the MPBC efficiency estimators report better convergence than the
smoothed Bootstrap efficiencies for samples with up to 38 DMUs. In the case a
[0.5, 1.0] z is decided by the user, then the maximum sample size for which the
MPBC simulates at a higher degree than the smoothed Bootstrap the true efficiency
varies according to the number of variables. For instance, for 4 variables, the
maximum sample size is 31; for 7 and 10 variables, it is 35 and 39, respectively. In all
these cases, the maximum sample size meets the criterion/rule of thumb of
{ } max * , 3*( ) n x y x y + for preventing dimensionality effects in the DEA efficiency
estimations.
Taking into account formulas (13) and (14), we develop the following roadmap to
facilitate the application of MPBC towards the optimum relative convergence of the
efficiency estimators to the true efficiency scores. In practice,
if variables
7, and
n<31 =[0, 2, 10] and
z
[0.5, 1.0]
31
38 =10 and
z
[0.0, 0.5)
if variables = 8, and
n<32 =10 and
z
[0.5, 1.0]
32
38 =10 and
z
[0.0, 0.5)
If variables = 9, and
n<36 =10 and
z
[0.5, 1.0]
36
38 =10 and
z
[0.0, 0.5)
If variables
10,
n =10 and
z
[0.5, 1.0], [max n
6
: adjusted to the number of
variables]
6
The MPBC method yields more consistent efficiency estimators than the smoothed Bootstrap, or
relative consistent efficiency estimators, and also free of dimensionality effects, for up to 43 DMUs
when 13 variables are incorporated in the DEA evaluation process.
Proceedings of the 10
th
International Conference on DEA Brazil2012 232
so that,
{ } { }
*
1
1
n
n
p
i i
i
i
=
=
where
( )
1
* *
#
i i
=
and # denotes the number of parameters.
Conclusions
In this paper, we developed a multi-parametric method for bias correction (MPBC) of
DEA efficiency estimators. The new method enhances the applicability and reliability
of DEA as a comparative efficiency measurement technique when small or inadequate
samples are available. In the presence of small or inadequate samples, DEA efficiency
estimators are not regarded as consistent because they are biased by sampling
variations and dimensionality.
In order to prevent any confusion to the users of the method resulting from the
selection of the appropriate parameters to attain greater consistency for the efficiency
estimators than the smoothed Bootstrap, we provide a detailed roadmap. Based on
this roadmap, we prove that efficiency estimators obtained by DEA in conjunction
with MPBC converge in higher probability to the true efficiency scores than the
efficiency estimators yielded by smoothed Bootstrap, under certain circumstances.
MPBC is regarded as an appropriate bias-correction method for DEA efficiency
estimators when the samples consist at maximum of 43 DMUs.
References
Banker, R. D. (1993). Maximum-Likelihood, Consistency and Data Envelopment
Analysis - a Statistical Foundation. Management Science, 39, 1265-1273.
Banker, R. D., Charnes, A., & Cooper, W. W. (1984). Some models for estimating
technical and scale inefficiencies in data envelopment analysis. Management Science,
30, 1078-1092.
Chernick, M. R. (2008). Bootstrap Methods: A Guide for Practitioners and Researchers.
New Jersey: John Wiley & Sons.
Cooper, W. W., Seiford, L. M., & Tone, K. (2007). Data envelopment analysis: a
comprehensive text with models, applications, references and DEA-Solver software
(2nd ed.). New York: Springer Science + Business Media.
Simar, L., & Wilson, P. W. (1998). Sensitivity analysis of efficiency scores: How to
bootstrap in nonparametric frontier models. Management Science, 44, 49-61.
Simar, L., & Wilson, P. W. (1999). Estimating and bootstrapping Malmquist indices.
European Journal of Operational Research, 115, 459-471.
Proceedings of the 10
th
International Conference on DEA Brazil2012 233
28. Relative balance as a
complementary measure to
relative efficiency
Heinz Ahn
Technische Universitt Braunschweig, Institute of Management Control and Business Accounting,
Germany, hw.ahn@tu-bs.de
Ludmila Neumann
Technische Universitt Braunschweig, Institute of Management Control and Business Accounting,
Germany, l.neumann@tu-bs.de
Nadia Vazquez Novoa
Technische Universitt Braunschweig, Institute of Management Control and Business Accounting,
Germany, n.vazquez-novoa@tu-bs.de
Abstract
One of the major strengths of Data Envelopment Analysis (DEA) is the endogenous
determination of the weights of performance criteria, assigning to each decision
making unit (DMU) its best possible efficiency score. However, this property also
leads to a significant shortcoming: it allows zero-value weights that exclude criteria
from the evaluation. While many approaches that deal with this problem incorporate
value judgments into analysis, our approach supports managements efficiency
analysis with a complementary performance measure that is derived from the given
data set. The respective balance score evaluates the extent to which a DMU avoids
concentration on only some of the crucial performance criteria. One of the possible
decisions resulting from a balance analysis is to reduce the set of DMUs considered to
serve as benchmarks. For this case, a modified CCR-O model is presented.
Keywords: DEA, Balance score, Inappropriate benchmarks
Introduction
Data Envelopment Analysis (DEA) is a meaningful approach to relative efficiency
measurement of decision-making units (DMUs) that assesses DMUs performance by
aggregating their input and output values into a single efficiency score. The weights of
these performance criteria are endogenously determined, assigning to each DMU its
best possible efficiency score. This property constitutes one of the major advantages
of the basic DEA models but it also represents a source of pitfalls concerning
performance assessment and performance control. Such problems result from the
possibility of zero-value weights that eliminate from the analysis any input and/or
output criteria in which the performance of a DMU is weak, with the aim of raising its
efficiency score to its maximum level (Dyson and Thanassoulis, 1988; Allen et al.,
1997; Thanassoulis et al., 2004). Unlike approaches that deal with this issue by
Proceedings of the 10
th
International Conference on DEA Brazil2012 234
incorporating value judgments into the usual DEA analysis, we suggest a
complementary balance score that directly refers to the given input/output values.
The balance score constitutes a source of additional information about a DMUs
performance structure. As is known, it may happen that different DMUs obtain
(nearly) the same efficiency score even if they substantially differ in their pattern of
criteria achievement. We suggest to cover this aspect of performance by means of the
balance score
o
. A DMU will be characterized by a balance score of 100% if all of its
crucial performance criteria values are relative to the respective values of the other
DMUs achieved on a similar level. In contrast, the more the performance criteria
values are not achieved on a (relative) similar level, the more the balance score
decreases towards 0%.
Often, outputs can be interpreted as indicators of strategic goals and inputs as means
to achieve these goals. For such cases, the pattern of the input values is estimated to
be of minor relevance, whereas the pattern of output values is estimated to be of great
managerial importance. For this reason, managers may prefer to focus their attention
on the evaluation of the balance of the output mix. We therefore focus on an output-
oriented balance score. If the analysis of the input mix would be of main interest, an
input-oriented balance score could be calculated.
The rest of the paper addresses the following aspects: introduction of the balance
model; benefit of the model application; numerical example and concluding remarks.
The Balance Model
According to Ahn et al. (2012) the procedure to measure relative balance is comprised
of two main steps: determination of the acceptable output mix region and calculation
of each DMUs balance score. Supposing that there is no ideal output pattern or that it
is at least unknown, we use the average of all DMUs as substitute and the average
absolute deviation from the mean as basis for the specification of a tolerance region,
called the multicone C .
While all DMUs located inside this region are considered to be balanced, the balance
score for all other DMUs needs to be calculated. A reference point on the boundary of
the multicone is determined for each unbalanced DMU guaranteeing that the
maximum relative variation
o,r
of each output r necessary to obtain a balanced output
mix is minimal. The formula 1
o
reflects to what extent the extreme pattern of the
output values of an unbalanced DMU differs from that of the reference point.
Balance analysis and its consequences
The complementary information provided by the balance analysis provides insights
that substantially influence the DEA efficiency analysis.
Concerning performance assessment, the balance score can be characterized as a kind
of secondary measure. On one hand, efficiency deficits cannot be compensated by a
high balance score. On the other hand, balance deficits let the performance expressed
by a high efficiency score appear in another light. For example, it may be found that
the cause of a DMUs low balance score is the pursuit of a significantly different
Proceedings of the 10
th
International Conference on DEA Brazil2012 235
business strategy. Consequently, such DMUs should be excluded from the set of
comparable DMUs.
Concerning performance control, the management should be cautious about using
efficient but unbalanced DMUs as benchmarks. An inefficient DMU with such a
benchmark would be urged to pursue a questionable performance strategy.
We suggest excluding efficient but unbalanced DMUs from the set of possible
benchmarks. This can be achieved by adding the following constraint to the
envelopment form of the CCR-O model:
{ } n k o k
k k o
,..., 1 , 1 , 0
,
=
This modification still allows all efficient DMUs to serve as their own benchmarks,
while eliminating unbalanced DMUs from the efficiency frontier relevant for the
inefficient DMUs. Consequently, the reference sets of the inefficient DMUs only
contain balanced DMUs, and the modified efficiency frontier leads to improvements of
the efficiency scores of inefficient DMUs. These improvements result from the fact that
extreme targets set by the unbalanced benchmarks are replaced with more moderate
targets. In special cases, it may even happen that an inefficient DMU turns into an
efficient one because it becomes part of the new frontier.
The numerical example that follows in the next section illustrates the proposed
balance analysis and shows the evolution in the results.
Numerical example
We refer to the example of a European pharmacy chain originally presented in Ahn et
al. (2012). As Table 1 shows, 20 pharmacy stores were analyzed, based on two inputs:
worked hours (WH) and store square meters (SQM) and three outputs: number of
customers buying medications available only on prescription (MP), number of
customers purchasing over the counter products (OTC) and total number of
prescriptions (PR). The outputs represent the main goals to be pursued by the stores.
The main numerical results calculated by running the balance model are presented in
Table 1. The DMUs A, E, F, O, Q, R, and T are not attain all outputs in an acceptable
proportion. For example, DMU A has a balance score
A
of 0.88 since it requires a
minimal output variation of 12% to be projected onto C ; an increase of 11% on MP
and 12% on PR is necessary, while a reduction of 12% on OTC is allowed.
In the extreme case, where at least one output is not produced at all, the respective
DMU will be characterized by a balance score of 0. The DMU R represents such an
extreme DMU because it has no OTC sales. The
o,r
values project R (nearly) onto the
origin of the coordinate system, which obviously cannot be understood as a
benchmark. However, the efficiency analysis by means of traditional CCR-O model
designates R as a benchmark for F and K (see Table 2), although Table 1 shows that R
is specialized on two outputs, while F and K also take a third output into account. For
this reason we reanalyze the efficiency of the pharmacy stores by means of a modified
CCR-O model which exclude unbalanced DMUs from the reference set of other
DMUs. Table 2 presents the results of the modified CCR-O model and those of the
traditional CCR-O model.
Proceedings of the 10
th
International Conference on DEA Brazil2012 236
Table 1. Data set of 20 pharmacy stores and the results of the balance model.
DMU
Inputs Outputs Balance model (iteration 1)
WH SQM MP OTC PR
o
o,MP
o,OTC
o,PR
A 7,424 287 3,578 12,329 15,431 0.88 0.11 0.12 0.12
B 14,208 168 9,115 8,765 40,034 1 0 0 0
C 8,608 146 7,644 13,340 29,453 1 0 0 0
D 8,736 281 7,866 13,055 27,304 1 0 0 0
E 10,656 333 12,244 9,307 56,743 0.89 0.11 0.11 0.07
F 10,272 130 8,700 10,067 52,510 0.99 0 0.01 0.01
G 10,336 274 6,749 14,279 30,522 1 0 0 0
H 12,160 240 8,867 13,446 41,789 1 0 0 0
I 7,424 203 10,483 12,580 52,145 1 0 0 0
J 12,960 264 8,981 13,787 36,094 1 0 0 0
K 10,880 257 7,000 13,071 39,464 1 0 0 0
L 6,240 220 7,946 10,826 43,239 1 0 0 0
M 14,240 127 15,335 24,899 68,108 1 0 0 0
N 11,584 115 13,133 27,643 60,306 1 0 0 0
O 3,840 92 5,365 12,365 16,079 0.90 0 0.10 0.10
P 9,600 266 8,119 12,908 43,039 1 0 0 0
Q 6,400 90 7,108 24,107 35,996 0.89 0.11 0.11 0.06
R 7,040 92 7,567
1
44,104 0
2
1 + 0.57
2
1 +
S 10,176 275 8,115 12,854 26,959 1 0 0 0
T 7,040 180 2,835 11,980 18,333 0.79 0.21 0.21 0.11
The balance and efficiency analysis concerning a period 1 is represented by the results
of iteration 1. The six pharmacy stores originally identified as efficient keep this
property. While the balanced DMUs I, M and N also continue to serve as benchmarks
for other stores, the unbalanced DMUs O, Q and R are eliminated from the reference
set of other stores. This increases the efficiency score of those inefficient stores which
formerly had unbalanced DMUs in their reference sets. For example, the new
benchmark of A is N, raising its efficiency score from 0.44 to 0.70. As a special case, L
is now characterized as efficient, and it becomes an appropriate benchmark for K and
P. It should be noted that the balance analysis indicates R as a DMU following a
different business strategy. In this case, R is not comparable to the other DMUs and
should be excluded from the sample.
On the basis of the data analysis of period 1, the management of the pharmacy stores
should take decisions according to the respective results. If a store is unbalanced, a
change of its output mix in reference to the determined reference point on the frontier
of the multicone is required. Hence, the DMUs A, E, F, O, Q and T should adjust their
output values according to the proposed
o,r
values in Table 1. Such changes in the
output mix may induce a negative change in the efficiency score in the subsequent
period 2 indicating the overestimation of the actual performance of the DMUs.
Proceedings of the 10
th
International Conference on DEA Brazil2012 237
Table 2. Traditional versus modified efficiency scores and benchmarks.
DMU
CCR-O model
Modified CCR-O
model (iteration 1)
Modified CCR-O model
(iteration 2)
o
o
o
A 0.44 Q 0.70 N 0.61 N
B 0.55 I, N, O 0.55 I, N 0.55 I, N
C 0.70 I, N, O 0.72 I, N 0.72 I, N
D 0.64 I, O 0.72 I, N 0.72 I, N
E 0.81 I 0.81 I 0.72 I
F 0.86 N, Q, R 0.93 I, N 0.92 I, N
G 0.50 I, O, Q 0.58 N 0.58 N
H 0.56 I, N, O 0.57 I, N 0.57 I, N
I 1 I 1 I 1 I
J 0.53 I, N, O 0.54 I, N 0.54 I, N
K 0.55 I, Q, R 0.59 I, L, N 0.59 I, L, N
L 0.99 I, Q 1 L 1 L
M 1 M 1 M 1 M
N 1 N 1 N 1 N
O 1 O 1 O 1 O
P 0.66 I, Q 0.70 I, L, N 0.70 I, L, N
Q 1 Q 1 Q 1 Q
R 1 R 1 R --- ---
S 0.57 I, O 0.62 I, N 0.62 I, N
T 0.46 I, Q 0.71 N 0.56 N
Assuming that the balanced DMUs keep their balanced output mix, the pharmacy
stores are again evaluated in period 2, but without the incomparable DMU R. The
results of this second balance analysis are shown in Table 2, iteration 2. The DMUs A
(
A
=0.99), E (
E
=0.97), F (
F
=0.97), Q (
Q
=0.99), and T (
T
=0.99) still remain
unbalanced, while DMU O becomes balanced and the balance score of DMU B
decreases to 0.97. The efficiency analysis by running the modified CCR-O model
shows a decrease of the efficiency scores of unbalanced DMUs while the efficiency
scores of balanced DMUs as well as the reference set of inefficient DMUs remain
unaffected.
This iterative process of balance and efficiency analysis could be continued. Assuming
that the output mix of balanced DMUs remains constant, the unbalanced DMUs will
turn into balanced ones after further iterations will be conducted.
Proceedings of the 10
th
International Conference on DEA Brazil2012 238
Conclusions
This paper introduces a units-invariant model for identifying DMUs with an
unbalanced output mix and illustrates the method with a numerical example. Our
approach supports managements efficiency analysis with a complementary
performance measure that avoids setting restrictions on the weights. Based on a
typical data set for a DEA application, it allows for higher discrimination within both
efficient and inefficient DMUs by quantifying their relative balance. The approach
does not need additional value judgments to quantify the balance scores and bears no
risk of infeasible solutions.
A thorough evaluation of the achieved balance scores provides insights that
substantially influence the DEA efficiency analysis. Concerning performance
assessment, the explanation for a DMUs low balance score could be that the DMU
pursues a different business strategy and therefore is not comparable to the other
DMUs. Concerning performance control, the management may agree that efficient but
unbalanced DMUs are not adequate benchmarks for inefficient DMUs. For this case, a
modified CCR-O model is proposed, which endogenously calculates the balance score
and eliminates the undesirable benchmarks. In addition, an iterative process of
balance and efficiency analysis is suggested, which determines adequate efficiency
scores and supports the unbalanced DMUs on their way to get balanced.
A series of alternative models can be developed by modification of the procedure to
calculate o (Dyckhoff et al., 2012). For example, instead of using the average
deviation as a measure of dispersion the less restrictive standard deviation could be
used. This would lead to a broader multicone
C
. Nevertheless, this alternative
multicone would keep the feature of being sensitive to the distribution of the DMUs,
changing its position and amplitude according to the output values of the DMUs.
References
Ahn H., L. Neumann, N. Vazquez Novoa (2012) Measuring the relative balance of
DMUs. European Journal of Operational Research 221: 417-423.
Allen R., A. Athanassopoulos, R.G. Dyson, E. Thanassoulis (1997) Weights restrictions
and value judgments in Data Envelopment Analysis: Evolution, development and
future directions, Annals of Operations Research 73: 13-34.
Dyckhoff, H., A. Dirksen, E. Mbock (2012) Measuring balanced efficiency with DEA:
New approach and case study of German business schools research performance,
http://ssrn.com/abstract=1990233.
Dyson R.G., E. Thanassoulis (1988) Reducing weight flexibility in Data Envelopment
Analysis, Journal of the Operational Research Society 39: 563-576.
Thanassoulis E., M.C.S. Portela, R. Allen (2004) Incorporating value judgments in DEA.
In Cooper, W.W., L.M. Seiford, J. Zhu (eds.), Handbook on Data Envelopment
Analysis. Boston et al.
Acknowledgements
We gratefully acknowledge the financial support of the DFG (German Research
Foundation) in the context of the research project Advanced Data Envelopment
Analysis.
Proceedings of the 10
th
International Conference on DEA Brazil2012 239
29. Statistical Inference and
Efficient Portfolio Investment
Performance
Shibo Liu
Loughborough University UK, S.Liu2@lboro.ac.uk
Tom Weyman-Jones
Loughborough University UK, T.G.Weyman-jones@lboro.ac.uk
Karligash Glass
Loughborough University UK, K.A.Kenjegalieva@lboro.ac.uk
Abstract
The purpose of this paper is to apply a quadratic data envelopment analysis model
with bootstrap and second stage regression to estimate the efficiency of a sample of
investment funds, obtain the statistical inference of the efficiency scores and detect the
determinants of inefficiency. Morey and Morey (1999) developed a mutual funds
efficiency measure in a traditional mean-variance model. It is derived from the
standard data envelopment analysis but differs from it in having non-linear constraints
in the envelopment version of the models structure. This paper first applies the
procedures in Morey and Morey (1999) to a new modern data set comprising a multi-
year sample of investment funds and then utilise Simar-Wilson (2008) bootstrapping
algorithms to develop statistical inference and confidence intervals for the indexes of
efficient investment fund performance. For the second stage analysis, robust-OLS
regression, Tobit models and Papke-Wooldridge (PW) models are conducted and
compared to evaluate contextual variables affecting the performance of investment
funds. The DEA efficiency scores are regressed on potential variables to test the
statistical significance of those factors. Results and inferences are drawn from an
extensive new dataset of investment funds.
Keywords: nonlinear-DEA, portfolios, bootstrapping, second stage DEA
Introduction
The history of portfolio evaluation dates from the 1960th (Sharp, 1966; Treynor, 1965
and Jensen, 1968), with emphasis on both expected return and risk. Investment fund
managers attempt to find efficient portfolios those promising the greatest expected
return for any given degree of risk, i.e. risk-adjusted return. However, there are many
criticisms of traditional portfolio analysis which focus on their sensitivity to chosen
benchmarks. Murthi et at. (1997) were the first to apply DEA methodology to fund
performance evaluation. A large proportion of DEA models applied to investment
funds show piecewise linear correspondence between multiple inputs and outputs.
Proceedings of the 10
th
International Conference on DEA Brazil2012 240
However, according to Markowitz portfolio theory, there is correlation between
different assets which should not be ignored, and these co-movements between
different securities affect the relationship between expected return and risk of the
combined portfolio.
Based on the unique characteristics of investment trusts, Morey and Morey (1999)
developed an investment funds efficiency measure in a traditional mean-variance
model. It was based on Markowitz portfolio theory and related the non-parametric
methodologies to the foundations of traditional performance measurement in mean-
variance space. The model is derived from the standard data envelopment analysis but
differs from it in having non-linear constraints in the envelopment version of the
models structure. Although mean and variance are considered in Morey and Morey
(1999) models, they distinguish their model from traditional portfolio analysis by the
fact that there is no theoretical benchmark like the market portfolio of the Capital
Asset Pricing Model. Instead, the benchmarking fund in Morey and Morey (1999)
consists of certain funds in the group, each with a particular weight. So rather than
being compared with an idealised fund, Morey and Morey (1999) model benchmarks
the funds under evaluation again themselves. This makes Morey and Morey (1999)
model practically feasible and easier to test.
Therefore, this paper firstly applies the procedures in Morey and Morey (1999) to a
new modern data set comprising a multi-year sample of investment funds. It then
extends Morey and Morey (1999) model by adding statistical significance tests. The
motivation is that the DEA efficiency scores obtained through Morey and Morey (1999)
model have not hitherto been tested for statistical significance. Simar-Wilson (2008)
bootstrapping algorithms are utilised to develop statistical inference and confidence
intervals for the indexes of efficient investment fund performance.
After conducting the statistical test, this paper then examines the efficiency of
investment trusts, analyse the factors contributing to investment trusts performance
and detect the determinants of inefficiency. This framework involves two-stages. In
the first stage, efficiency scores are calculated using Morey and Morey (1999) quadratic
DEA model. And then in the second stage, these scores are regressed on potential
explanatory variables which may have an impact on the funds performances. Robust-
OLS regression, Tobit models and Papke-Wooldridge (PW) models are conducted and
compared to evaluate contextual variables affecting the performance of investment
funds. In this stage the DEA efficiency scores are regressed on potential variables
including Sharpe ratio, Jensens alpha, expense ratio, P/E ratio, book to market ratio
and market value of the investment funds to test the statistical significance of those
factors.
Methods
In the essence of data envelopment analysis, Morey and Morey (1999) quadratic
models use the idea of funds of funds: for each fund there is a corresponding
composite benchmarking fund, which lies on the efficient frontier. These are
hypothetical but potentially efficient combinations of the actual observations. DEA
scores are obtained by measuring the direct distance from the position of the fund in
question in mean-variance space to that of the efficient composite benchmarking fund.
Proceedings of the 10
th
International Conference on DEA Brazil2012 241
Morey and Morey (1999) presented two basic quadratic programming approaches to
identify those funds that are efficient. These two approaches are mean return
augmentation and risk contraction. Figure1 illustrate these two quadratic models.
From Figure 1, it is in the mean-variance space with risk as input and mean return as
output. These two approaches show different paths to the efficiency frontier.
Mean return augmentation method could be seen as output oriented DEA, and it
represents a vertical path towards the efficient frontier, while the risk contraction
model is input-oriented DEA which follows the horizontal path.
Consider N mutual funds to be evaluated, indexed j=1, 2,,N, where
0
j is the fund in
evaluation for each run. , ,... 2 , 1
0
N j = and there are N runs totally. Let T denote the
number of different time horizons, where t=1, 2,,T. Denote ) (
,t j
R E as the mean
return for fund j, and
2
j
as its the variance as well as ) , (
, , t j t i
R R Cov as its covariance.
Denote
j
w as the weight allocated to each fund to form the benchmarking fund in
each run. Formula of mean return augmentation is as follows:
Determine ) ,..., ,... 2 , 1 ( 0
0
N j j w
j
= so that:
) ,... 2 , 1 (
) ( ) (
) , (
1 . .
,
1
,
2
, ,
1 1 1
2
,
2
1
0
0
T t
R E R E w
R R Cov w w w
w t s
Max
t j
N
j
t j j
j t j t i
N
j
N
j i
i
N
j
j i t j j
N
j
j
=
+
=
=
=
+
= =
=
=
+
= =
=
=
Where 1 Z and the efficient frontier is composed
by funds with Z equal to one.
Results and discussions
29 funds classified by Morningstar as UK equity Mid/Small cap are tested and the
results show that both mean return augmentation and risk contraction approach
identify the same 6 efficient funds and except the efficient funds, 3 funds out of 29
have the same ranking from both approaches, with other funds rank differently for
different approaches. The correlation between two rankings is 0.829929, which is very
high. This means that although mean return augmentation approach and risk
contraction approach emphasize different aspects and have different benchmarking
fund on the efficient frontier, a fund could get similar ranking based on two
approaches. Also, the marginal contribution of the mean return and variance in each
period to the funds efficiency could be obtained by solving lagrangian functions.
Then DEA scores are bootstrapped with 2000 replicates and Bandwidth h equals
0.2186 which is calculated by least-squares cross-validation method described by
Silverman (1986). From the results, the initial DEA model gives an average
uncorrected efficiency score of 1.2470, while the bootstrap model generates an
average bias-corrected score of 1.2642. The minimum uncorrected score is 1 and the
maximum is 1.9331, while the minimum bias corrected score was 1 and the maximum
was 1.9736. For the most efficient funds, the 2000 bootstrap estimators are all equal to
one; therefore the 95% confidence intervals for these funds become a single point.
The results also reveal that all the estimated biases are negative, which is as expected,
because according to Simar and Wilson (1998), the DEA estimate is upwardly biased
using an input oriented model and downwardly biased for an output oriented model.
The original scores have a mean bias of -0.0172. And the standard deviations for all
the estimators are quite small with the maximum standard deviation equal to 0.0908.
All the funds satisfy the condition of
4
1
>
i
i
bias
Proceedings of the 10
th
International Conference on DEA Brazil2012 279
, , 1, = 0, ))]
) ( ( ( ]) [ ( )
1 =
1 =
n j d
d p d
y
l
j
t l
n
o l
l
o
y
o
j
t j
j
t
y
j
y
j
y
rl l
n
o l
l
.
+
+ + +
+ +
) ( ( [
max
)] ) [(
} | | , | {
1 = 1 =
o
x
io ij
x
j
s i
x
j
x
j
s
x
j
J
x
j
s
x
j
s
il l
n
o l
l
io o ij
m
i
d c
x x c
(1)
, , 1, = 0, ))]
) ( ( ( ]) [ ( )
1 =
1 =
n j d
d c d
x
l
j
t l
n
o l
l
o
x
o
j
t j
j
t
x
j
x
j
x
il l
n
o l
l
. +
+ +
. , 1, = 0, 1, =
1 =
n j
j j
n
j
.
It is clear that if , 1,..., = |, =| |, =| n J J
y
j
y
j
x
j
x
j
Soysters method is obtained and if
, 1,..., = 0, = = n j
y
j
x
j
we have the nominal problem. So, we have the flexibility of
adjusting the robustness of the method against the level of conservatism of the
solution by varying
x
j
and . 1,..., = , n j
y
j
The above model is non-linear. We can
obtain the linear form of Model (1) by the proposition provided by Bertsimas and Sim
(2004), as follows:
max =
o
k
, , 1, = 0,
] ) [( . .
1 = 1 =
n j q
z y y p t s
y
rj
y
j
J r
y
j
y
j rl l
n
o l
l
ro o rj
s
r
. +
+
, , 1, = 0,
] ) [(
1 = 1 =
n j q
z x x c
x
ij
x
j
J i
x
j
x
j il l
n
o l
l
io o ij
m
i
. +
+ + +
, , 1,..., = , ) (
y
j
y
ro o rj
y
rj
y
j
J r n j d p q z + , , 1,..., = ,
1 =
y
j rj l
n
o l
l
y
rj
y
j
J r n j p q z +
, , 1,..., = , ) (
x
j
x
io o ij
x
ij
x
j
J r n j d c q z + + , , 1,..., = ,
1 =
x
j ij l
n
o l
l
x
ij
x
j
J i n j c q z +
(2)
1, =
1 =
j
n
j
. , , , 1, = 0,
0, 0, 0, 0,
x
j
y
j
x
ij
y
rj
x
j
y
j j
J i J r n j q
q z z
.
It should be mentioned that Model (2) can be applied in cases where the lower
bound of each interval input and output is non-negative.
Proceedings of the 10
th
International Conference on DEA Brazil2012 280
Model (2) is solved in two different ways for
x
j
s and
y
j
s. In the first case, the
overall profit efficiency scores of DMUs are obtained from Model (2) for each
x
j
and
y
j
that are randomly generated. Then, the mean values of these overall profit
efficiency scores are considered as the robust overall profit efficiency scores. In the
second case, the overall profit efficiency scores of DMUs are obtained from Model (2)
by increasing
x
j
and
y
j
, which change from zero to the number of the uncertain
inputs and outputs, respectively. The mean values of the overall profit efficiency
scores are given as the robust overall profit efficiency scores.
Definition 1: DMU
o
is overall profit efficient if in Model (2), 0 =
* *
.
Robust Overall Profit Efficiency (ROPE) Model with the Input and
Output Price Vectors Uncertainty
In this section, a model is formulated, based on robust optimization, to measure
overall profit efficiency with the input and output price vector uncertainty. In real
applications, it is unlikely for all coefficients of price vectors to be equal to their
nominal value; it is also unlikely for them all to be equal to their worst-case value.
Thus, it is desirable to adjust the level of conservativeness of the solution, so that a
reasonable trade-off between robustness and performance is achieved. The numbers
n j
p
j
1,..., = , and n
c
j
1,..., = , are presented, which assume values in the intervals
|] | [0,
p
j
J and |] | [0,
c
j
J , where
p
j
J and
c
j
J are the index sets of the uncertain
parameters of the output and the input prices, respectively, and they are not
necessarily integer-valued.
p
j
and
c
j
are to adjust the robustness of the proposed
method against the level of conservatism of the solution. In fact, they are the
protection level for the j -th constraint. The number of coefficients allowed to vary is
at most
p
j
and
c
j
for the output and input price vectors respectively. Thus, the main
purpose is to introduce a model that is protected against the maximum
p
j
and
c
j
,
where only one output price and only input price
tj
p and
tj
c vary by at most
p
tj
p
j
p
j
]) [ ( and
c
tj
c
j
c
j
d ]) [ ( , respectively. That is, it is assumed that only a subset of
the output price and a subset of the input price change to yeild the solution that was
presented by Bertsimas and Sim (2004). We propose the robust counterpart of Model
(5) as follows:
max =
o
d
Proceedings of the 10
th
International Conference on DEA Brazil2012 281
) ( [
max
) ( . .
1 = } | | , | {
1 = 1 =
l rl
n
l
ro
p
rj
p
j
s r
p
j
p
j
s
p
j
J
p
j
s
p
j
s
l rl
n
l
ro rj
s
r
y y d
y y p t s
+
, , 1, =
0, )] ( ]) [ (
1 =
n j
y y d
l l
j
t
n
l
o
j
t
p
j
j
t
p
j
p
j
.
+
) ( [
max
) (
1 = } | | , | {
1 = 1 =
l il
n
l
io
c
ij
c
j
s i
c
j
c
j
s
c
j
J
c
j
s
c
j
s
l il
n
l
io ij
m
i
x x d
x x c
+ +
+
(3)
, , 1, =
0, )] ( ]) [ (
1 =
n j
x x d
l l
j
t
n
l
o
j
t
c
j
j
t
c
j
c
j
.
+ +
. , 1, = 0, 1, =
1 =
n j
j j
n
j
.
The above model is non-linear. We can obtain the linear form of Model (3) using the
proposition given by Bertsimas and Sim (2004), as follows:
max =
o
d
, , 1, = 0,
) ( . .
1 = 1 =
n j q
z y y p t s
p
rj
p
j
J r
p
j
p
j l rl
n
l
ro rj
s
r
. +
+
, , 1, = 0,
) (
1 = 1 =
n j q
z x x c
c
ij
c
j
J i
c
j
c
j l il
n
l
io ij
m
i
. +
+ +
, , 1,..., = ,
p
j
p
r
p
rj
p
rj
p
j
J r n j f d q z +
, , 1,..., = ,
c
j
c
i
c
ij
c
ij
c
j
J r n j f d q z +
, 1,..., = ,
1 =
s r f y y f
p
r l rl
n
l
ro
p
r
(4)
, 1,..., = ,
1 =
m i f x x f
c
i l il
n
l
io
c
i
1, =
1 =
j
n
j
Proceedings of the 10
th
International Conference on DEA Brazil2012 282
. , , , 1, = 0,
0, 0, 0,
0, 0, 0,
c
j
p
j
c
ij
p
rj
c
i
p
r
c
j
p
j j
J i J r n j q
q f f
z z
.
Model (4) is solved for
p
j
s and
c
j
s, which are once generated randomly and once
determined by being increased step by step. In both cases, the mean value of the
overall profit efficiency scores are given as the robust overall profit efficiency scores.
Results and discussions
In this section, the robust overall profit efficiency will be explained. We consider a
problem with 4 DMUs, two inputs, two outputs, and two interval input and output
price vectors. The input, output, and nominal price vector data are taken from Cooper
et al. (2006). The data for DMUs are given in Table 1. It is assumed that all deviation
values from the nominal
rj
p and
ij
c are equal to ten, that is, 10. = =
c
ij
p
rj
d d
We solve Model (2) for different combinations of
x
j
s and
y
j
s by the generalized
algebraic modeling system (GAMS). Thirty random
x
j
s and
y
j
s are generated and
the overall profit efficiency scores for each random
x
j
and
y
j
are saved.
1
j
k is given
the robust overall profit efficiency score of a DMU using the mean of overall profit
efficiency scores. Model (2) is run for different
x
j
s increasing by 0.2 in each step
from zero up to 3 and different
y
j
s increasing from zero up to 5 with step length 0.2,
after which the values of overall profit efficiency scores are stored. The mean values
of overall profit efficiency scores are given as the robust overall profit efficiency
scores, denoted by
2
j
k . The results are provided in Table 2.
Model (4) is run for different combinations of
p
j
s and
c
j
s by the generalized
algebraic modeling system (GAMS). Ten random
p
j
s and
c
j
s are generated and
overall profit efficiency scores are stored for each random
p
j
and
c
j
. The robust
overall profit efficiency scores are determined using the mean values of overall profit
efficiency scores and are denoted by
1
j
d . Moreover, the values of
p
j
s and
y
j
s are
increased from zero to 2 with step length 0.2 and the values of overall profit
efficiency scores are stored. The mean values of overall profit efficiency scores are
presented as the robust overall profit efficiency scores, denoted by
2
j
d . The results are
given in Table 3.
Conclusions
In this paper, a deterministic methodology was proposed to address the problem of
measuring overall profit efficiency subject to the input, output, and price vector
parameters being within an uncertainty set. Using the robust optimization concept, an
equivalent model was built without uncertainty of the same class. Specifically, the
Proceedings of the 10
th
International Conference on DEA Brazil2012 283
proposed model is a linear programming problem. One of the most appealing features
of this model is that it uses very little information on the input, output, and price
vector distributions, and therefore it is widely applicable. If we only know the mean
and the variance of the distributions, the robust policy often outperforms the nominal
policy.
Acknowledgement
This research is partially supported by Ardabil Islamic Azad University.
References
Banker, R., Charnes, A., Cooper, W.W. (1984) Some models for estimating technical
and scale inefficiencies in data envelopment analysis, European Journal of Operational
Research 30 (9): 1078-1092.
Bertsimas, D., and Sim, M. (2004) The price of robustness, Operations Research 52 (1):
35-53.
Charnes, A., Cooper, W. W., Rhodes, E. (1978) Measuring the efficiency of decision
making units, European Journal of Operational Research; 2 (6): 429-444.
Cooper, W.W., Park, K.S., Yu, G. (1999) IDEA and AR-IDEA: models for dealing with
imprecise data in DEA, Management Science 45: 597-607.
Cooper, W.W., Seiford, L.M., Tone, K. Data Envelopment Analysis: A Comprehensive
Text with Models, Applications, References and DEA-solver Software. 2nd ed, New
York, Springer, (2006).
Entani, T., Maeda, Y., Tanaka, H. (2002) Dual models of interval DEA and its
extension to interval data, European Journal of Operational Research 136: 32-45.
Sengupta, J.K. (1992) A fuzzy systems approach in Data Envelopment Analysis,
Computers and Mathematics with Applications 24 (8-9): 259-266.
. 4 price nominal and output, input, : 1 DMUs for data The Table
3500 600 120 600 72 180 168 27 4
2200 480 90 450 55 160 160 25 3
1800 400 80 350 50 150 131 19 2
2010 550 100 500 90 100 151 20 1
2 1 2 1 2 1 2 1 j j j j j
p p c c O O I I DMU
Proceedings of the 10
th
International Conference on DEA Brazil2012 284
. Model(2) ith ROPE : 2 w of results The Table
. Model(4) ith ROPE : 3 w of results The Table
0.06 0.00 0.00 4
1.45 0.61 0.16 3
3.07 0.21 1.16 2
0.29 0.05 0.19 1
2 1 U
j j j j
k k k DMU
0.24 0.06 0.13 4
0.46 0.15 0.29 3
0.32 0.08 0.17 2
0.12 0.00 0.00 1
2 1 U
j j j j
d d d DMU
Proceedings of the 10
th
International Conference on DEA Brazil2012 285
35. The efficiency in Tourism
Investment capture in relation
to Highways Investment in
Tourist Routes of Esprito
Santo State, Brazil
Marta Monteiro da Costa Cruz
Federal University of Esprito Santo, Brazil, Doctor in Transport Engineering, mcruz@npd.ufes.br
Josiane Baldo
Federal University of Esprito Santo, Brazil, Master in Civil Engineering, josianebaldo@hotmail.com
Abstract
Tourism becomes a sector of more important time for economic and social
development of a country, contributing for the reduction of disparity of the regions
and favouring for its growth. The transport service, as well as road infrastructure is
essential part of tourism in order to promote movement of tourists to intended
destination. This paper presents the use of DEA to measure the efficiency in capture
of Tourism Investment in relation to Investment in Highways. Government tourism
planning of Espirito Santo State is based on eight tourist routes. Each route has an
origin in Vitoria, capital of Espirito Santo State and a final destiny that define the
route. For this study were analyzed 87 stretches road linking Vitria, to each urban
community that compose these routes. These 87 stretch roads were defined as DMU
and it was divided into two groups due to the difference in type of road, simple or
double. The inputs of this problem are distance between cities, road investment,
maintenance road investment and traffic volume. It was used tourism investment as
product. The DEA Model used was BCC oriented by product. The results achieved
shows that tourism level of investment can increase comparing with roads
investments.
Keywords: Data Envelopment Analysis, Efficiency Analysis, Highways of the Esprito
Santo, Tourism Investment and Highways
Introduction
According to Porter (1990), the infrastructure is essential to the promotion of systemic
conditions of competitiveness in service systems - transport, energy, water,
telecommunications - key to economic activity.
In the economic context, according to the Brazilian Institute of Tourism (2010), the
country reached the goal of $ 5.8 billion in international currencies generated by
tourism. In the Espirito Santo State, according to a survey conducted by the State
Secretariat of Tourism of the Espirito Santo State - SETUR in February 2010, the
Proceedings of the 10
th
International Conference on DEA Brazil2012 286
number of tourists between 2008 and 2010 in high season increased by about 47%,
resulting in an increase of 109% in average spending Tourist Routes. Regarding the
mode of transport used, approximately 84% of answers used the road to move around
to the desired destination.
In the social context, when a region has, among other issues, a road infrastructure in
perfect condition has expanded its capacity to generate employment, income
distribution, providing improved quality of life in communities. Already the tourism
sector provides an endless source of knowledge of historical, cultural and social
development of a country, state or community.
Today, Brazil ranks seventh in the ranking of nations that hold more international
events, the city of So Paulo as first place in the Americas. The country reached its
target of $ 5.8 billion in international currencies generated by tourism and, little by
little, distance themselves from stereotypes related to "exotic" (MINISTRY OF
TOURISM, 2010c).
However, the World Tourism Organization says that 75% of global tourist flows are
intra-regional, i.e., the majority of tourism takes place within the tourist region. The
eminent predominance of short-distance travel and reinforces this statement
demonstrates that the practice of domestic tourism has been gaining more importance
in the economic context of tourism.
According to the National Council of Commerce (CNC) (2008), domestic tourism in
Brazil's support base of the entire production chain of domestic tourism and presents
itself also as the vector product qualification against the national tourist international
travel market. Spraying of economic activities issued by domestic tourist movement
houses several tourist segments and forces them to get better qualification of its
products and services, preparing them well for the reception of international tourist
who usually has a higher criterion of satisfaction.
In a continental country like Brazil, tourist land transport has a significant importance -
both for its affordability among consumers, and by geographic accessibility near the
tourist destinations and products. Moreover, it is of fundamental importance for
accessibility to local natural and cultural attractions, as well as stimulating the personal
interaction between tourists, travelers and tourism professionals.
The overall goal of the work is to apply Data Envelopment Analysis - DEA to evaluate
the efficiency level of the Government to attract investment in the tourism sector,
taking into account the investment made in road stretches that give access to Tourist
Routes of Espirito Santo, Brazil. They were then identified road stretches with better
technical performance.
Espirito Santo Tourist Routes
Based on the Tourism Regionalization Plan, the State Government of the Espirito
Santo State, through the State Department of Tourism (SETUR), created since 2001, the
eight tourist routes listed below along with the cities that make up (SETUR, 2009):
. Route Capara with 11 cities
. Route Immigrants with 8 cities
. Route of the Sun and Moqueca with 5 cities
Proceedings of the 10
th
International Conference on DEA Brazil2012 287
. Sea Route and the Mountains with 5 cities
. Green Route and Waters with 5 cities
. Route of the Valleys and Coffee with 5 cities
. Coastal Route and Immigration with 7 cities
. Route Marble and Granite with 20 cities
Currently, the Espirito Santo State have three routes in the international market (Route
of the Sun and Stew, Sea Route and the Green Mountains and Rota and Waters).
Access to the tourist routes is done through federal highway (BR-262, which connects
with the Espirito Santo State Minas Gerais, in the east-west, BR-101 South and North,
which links the Espirito Santo State with the Rio de Janeiro and Bahia, north-south),
privatized highways (ES-060 South, known as the "Highway of the Sun", which
connects Vitoria with the south coast of the state) and non-privatized (ES-166, ES-185,
ES -490, ES-259, ES-060 North, among others) (CRUZ, JR OLIVEIRA, 2009).
Regionalization Planning Tour of the Espirito Santo State seeks to optimize and
facilitate the mobilization efforts, resources, communication and synergy of production
arrangements, seeking to consolidate the sustainable development of tourism,
revealing a set of differential highlighted with attractive, concentrated in a
geographical space delimited. These spaces must present conditions of competition
and cooperation among actors that stimulate public investment, and define market
segments to work (SETUR, 2006).
Espirito Santo Roadways
Whereas the total area of the state is 46,187 km , its road density is 14.1 km of
highway per 100 km . All seats of its 78 counties can be accessed by paved highway.
The average distance within the state, to access a paved highway, federal or state, is
6.1 km. This distance varies average 2.5 km to 23.9 km on the basis of micro-planning
of the state. During the surveys, it was found that 207.6 kilometers were in paving and
most of the roads were unpaved natural bed (ES-DER, 2009).
Identification DMUs
The road stretches linking the capital of Esprito Santo (Vitoria) to cities that make up
each route can be considered the decision-making units, homogeneous, because they
use the same vectors inputs / product with the same goal, which is road transport
carried by passenger vehicles, public transportation vehicles, transport vehicles and
cargo motorcycles, and therefore can be evaluated by their relative efficiencies, the
units identified as being able to compose the efficient production frontier.
Note that there is the existence of road stretches connecting Vitoria to a given
municipality who share inputs and outputs of another passage that connects Vitoria to
another city. This happens due to the composition of excerpts come from one or
more cities.
The decision to take as its starting point the city of Vitoria was due to several factors:
first by its geographical location, being equidistant from most cities that make up the
routes except one route to which the municipality belongs. Second factor is important
Proceedings of the 10
th
International Conference on DEA Brazil2012 288
because this city be cut by BR-101, the main highway Brazilian, passing through the
metropolitan area of Greater Vitoria cities linking the capital to the northern region of
the state, the BR101 North and south cities of the region, the South BR101 The city of
Vitoria binds to the State of Minas Gerais and the cities of the region and Southwest
region of Capara by BR262 and the Highway of the Sun, ES-060, which connects the
southern coast of the state. These roads are of great importance to highway network
of Tourist Routes of the Espirito Santo State, since access to Routes must be done
largely through them.
Another important factor is the fact that Vitoria is the only county in the state to have
an airport with regular flights, making it a great receiver for tourists throughout the
state.
The road stretches analyzed vary greatly in terms of volume of traffic, flows ranging
from very low to very high flows. When observations of highway sections with flow
of vehicles of different intensities are used on the same sample, we expect problems
regarding the interpretation of the results.
Thus, one group was performed as auxiliary support the interpretation of results. The
stretches of highway linking the cities of Vitria Routes have distinct characteristics of
traffic volume. You can check that the distinction traffic volume occurs in stretches of
two-lane road. Thus were established two groups: the 1st group will be represented
by highway sections consisting solely of single lane highways, and the 2nd group
shown by road stretches composed of simple and double track trails.
DEA Variables
The initial selection was based on data availability in DER-ES on Secretariat State
Tourism of the Espirito Santo State, the Undersecretary of State Budget and the
literature review. Initially six variables were chosen to analyze the efficiency of road
stretches. Taking into consideration the opinion of experts, the relevant variables for
final analysis of efficiency of road stretches connecting Vitoria to each municipality
that make up the Tourist Routes of the Espirito Santo State were:
Inputs: Distance, investment in roads, investment in maintenance and traffic volume.
Products: Investment in tourism
In this study we adopted the BCC model (BANKER, CHARNES, COOPER, 1984) (with
variable returns to scale) product oriented, with and without restriction to weights,
using the software SIAD.
The SIAD (ANGULO MESA et al, 2005) was developed to calculate all the results of
DEA models, such as efficiencies of DMU's, weights for each of the targets,
benchmarks and clearances. Furthermore, it also provides the option of inserting
weight restrictions and the possibility of using up to 150 DMU's. Through SIAD also
can see which units work with constant returns and they work with variable returns to
scale.
Proceedings of the 10
th
International Conference on DEA Brazil2012 289
Results and discussions
In this study, it was found that the efficiency score of inefficient portions were very
low, portraying the deficiency of tourism investment over investment in roads. In the
1st group, 82% of the stretches were analyzed efficiency below 50%. In the 2nd group
this percentage was even higher, 87%. This means that the level of investment in
tourism can increase large-scale investment in roads and road maintenance support
this increase.
The road stretches efficient by default were Vitoria - Vila Velha (less distance, less
investment in roads, lower investment in road maintenance and increased investment
in tourism) and the Vitoria - Ibatiba, lower volume of vehicles.
One can see that besides the uniformity of efficiency scores of passages considered
efficient with and without restriction to weights, this uniformity is also observed in the
analysis of benchmarks with and without restriction to weights because practically the
same passages that were benchmarks without restriction to weights were also
restricted, with both the 1st group and for the 2nd group.
It is noteworthy that, despite several attempts planning and investment by the
Government in the Tourist Regions of the Espirito Santo State, is still far below what is
needed to ensure the efficiency of road stretches that give access to these routes.
Thus, one should seek to improve investment in the tourism sector, maintaining the
quality and efficiency of highways.
Conclusions
Besides the analysis of tourism investment over investment in roads, there are several
other important indicators in the analysis of efficiency Tourist Routes that have not
been included here. Therefore it is extremely important that prompted the sectors
responsible for the inclusion of indicators such as:
Number of tourists;
Index accidents on highways;
Cost of accidents on highways.
Another suggestion would be to analyze the efficiency of Tourist Routes in relation to
its infrastructure (power of attraction of the municipality, number of Beds, number of
Tourist attractions, number of positions of power, among other tourism products).
References
ANGULO MEZA, L;, BIONDI NETO, L.; SOARES DE MELLO, J.C.C.B.; GOMES, E. G.
ISYDS (2005) Integrated System for Decision Support (SIAD Sistema Integrado de
Apoio a Deciso): a software package for data envelopment analysis model. Pesquisa
Operacional, v.25, n.3, p 493-503.
BANKER, R.D.,CHARNES, A., COOPER W.W. (1984) Some models for estimating
Technical and Scale inefficiencies in Data Envelopment Analysis. Management
Science.
Proceedings of the 10
th
International Conference on DEA Brazil2012 290
CONFEDERAO NACIONAL DO COMRCIO CNC. (2008) Transporte turstico
terrestre / Confederao Nacional do Comrcio, Coordenao das Cmaras Brasileiras
de Comrcio, Cmara Brasileira de Turismo. Rio de Janeiro.
CRUZ, M. ; OLIVEIRA, R. (2009) . Rotas tursticas do Estado do Esprito Santo:
atratividades versus custos envolvidos. In: XXIII ANPET - Congresso de Pesquisa e
Ensino em Transportes, 2009, Vitria. Panorama nacional da pesquisa em transportes
2009. Vitria : ANPET, v. 1. p. 1-4.
DEPARTAMENTO DE ESTRADAS E RODAGENS DO ESPRITO SANTO- DER-ES.
(2009) Plano Estratgico de Logstica e Transportes do Esprito Santo: Componente
Rodoviro.Vitria. 6v.
MINISTRIO DO TURISMO. (2010) Turistas estrangeiros gastaram u$ 5,3 bilhes no
Brasil em 2009, 22/01/10. Disponvel em: http://www.turismo.gov.br/dadosefatos/.
Acesso em: Mar. 2010.
MINISTRIO DOS TRANSPORTES. (2010) Departamento Nacional de Infra-Estrutura
de Transportes. Rede Rodoviria do PNV: diviso em trechos 2010. Braslia, 2010.
ORGANIZAO MUNDIAL DO TURISMO, (1994) Banco de dados. Disponvel em:
<http//www.world-tourism.org>. Acesso em: Mar. 2010.
PORTER, M. (1990) The need for a new paradigm: the competitive advantage of
nations. New York: The Free Press, 1990.
SECRETARIA DE ESTADO DE TURISMO DO ESPRITO SANTO. (2008) Pesquisa de
turismo receptivo na regio metropolitana da grande Vitria alta temporada. SEBRAE:
FUTURA. Vitria, 2008.
Proceedings of the 10
th
International Conference on DEA Brazil2012 291
36. The investment in ports
enterprises in Espirito Santo,
Brazil
Karen Vassoler Martins
Federal University of Espirito Santo, Brazil, Mastering in Transport Engineering,
karenvmartins@gmail.com
Marta Monteiro da Costa Cruz
Federal University of Esprito Santo, Brazil, Doctor in Transport Engineering, mcruz@npd.ufes.br
Abstract
Considering the amount of resources needed for ports enterprises, it is important to
evaluate the level of infrastructure utilization at ports terminals to assist managers to
take decisions on improving performance or investing in increasing capacity. In this
line, this paper analyses the efficiency of available elements use at public port
terminals located at the state of Esprito Santo, Brazil, in order to obtain subsidies to
its planning and development. For this, it was made a performance comparison
between regional terminals using DEA modelling and the homogenization criteria of
DMUs presented by Bertoloto (2010). The results of the study indicate the potential for
increase cargo handling and can contribute to management decisions at the
operational and strategic level.
Keywords: Port efficiency; Port infrastructure; DEA.
Introduction
According to the Secretary of Ports (SEP, 2011), the Brazilian port sector handles
approximately 700 million tons of various goods per year, what strengthens its
importance in the major trade operations (IPEA, 2010, SEP, 2011).
In the context of Brazilian international trade, the port complex located at the state of
Esprito Santo has a prominent position (ESPRITO SANTO, 2006). In 2010, ports and
terminals that belong to that complex moved around 171.46 million tons of cargo
(SINDIEX, 2011), at about 24.00% of the total flow in the country.
Given its importance, the development of regional economy is closely linked to the
strategies of port activities. And to support the existing demand through public
terminals, it is indispensable to increase efficiency in the use of available infrastructure
(ESPRITO SANTO, 2006).
As defined in Brazilian port sector legislation, port facilities can be classified as private
or public, and public facilities can be operated by public or private companies, by
prior bidding and agreement (BRAZIL, 1993; IPEA, 2009; IPEA, 2010).
Among the elements that ensure port competitiveness there are the berths and depth,
both determinants of the size of ships that dock at the port, that also determine the
Proceedings of the 10
th
International Conference on DEA Brazil2012 292
amount of cargo handled (BERTOLOTO, 2010; IPEA, 2010). Considering the amount
of resources needed to build these elements, to know the level of use of existing
structures is important at the studies for ports and terminals expansion (WANKE,
2009a).
In this sense, the analysis of efficiency, defined as the ratio between what is produced
and what could be produced with the same resources (SOARES DE MELLO et al.,
2005), can be useful to identify the efforts to improve ports performance. And one of
the most known and used methods to get benchmarks for ports infrastructure is the
Data Envelopment Analysis (FONTES AND SOARES DE MELLO, 2006; COSTA, 2007;
PIRES et al., 2009; BERTOLOTO, 2010).
The Data Envelopment Analysis (DEA) is a technique for analysis and diagnosis based
on mathematical programming developed by Charnes, Cooper and Rhodes (1978),
that uses data input and output and the production function theory to estimate the
efficiency frontier in a set of decision-making units (SOARES DE MELLO et al., 2003;
BLONINGEN AND WILSON, 2008; WANKE, 2009 a; BERTOLOTO, 2010).
To analyze the efficiency of use of infrastructure utilization of public terminals located
at Esprito Santo and obtain information for its planning and development, this paper
presents a study of nine ports and terminals, with similar location and connectivity
with other modes, based on maximum draft allowed, total length of berths and total
amount of cargo handled, with application of DEA. According to the different forms of
exploitation which they are subjected, these ports and terminals were classified into
three groups - Direct Administration, Rented and Private - on which was used the
compensation technique for non-homogeneous units presented by Bertoloto(2010).
The results indicate that there is potential to increase cargo handling at public
terminals and suggest that specialization and form of operation of terminals
and ports influence its efficiency.
Performance measures obtained can help to identify the level of utilization of ports
and terminals, to trace the causes of inefficiencies, to define the best form of
exploitation and to take decisions on berths specialization.
Methods
This study proposes a combination of theory and practice through the application of
Data Envelopment Analysis model on a set of ports and terminals located at the state
of Esprito Santo, Brazil, to evaluate the efficiency at local public terminals
infrastructure utilization.
The DEA model chosen was BCC (BANKER et al., 1984), input-oriented.
Nine ports or terminals were analyzed at the period between the years 2008 and 2009.
Were defined as inputs the maximum draft allowed and total length of berths, both
measured in meters, and as output the total amount of cargo handled quarterly,
measured in tons (BERTOLOTO, 2010). Data were obtained from analyzed ports and
terminals and from the local Port Authority.
Proceedings of the 10
th
International Conference on DEA Brazil2012 293
Ports and terminals, each quarter, were considered a decision-making unit (DMU) in
order to facilitate the identification of seasonality interferences and economic
situations in the amount of cargo handled.
Due the different forms of exploitation the studied ports and terminals are subjected,
the set is not considered homogeneous. Because of this the units were classified in 3
distinct clusters: Direct Administration, Rented and Private, with the application of
compensation method approached by Bertoloto (2010), following the script laid out
by the author: 1) Running a DEA model isolated for each cluster; 2) Running a DEA
model with only the efficient units of each cluster and calculate the median of the
new efficiency values for each cluster; 3) Divide the outputs of DMUs of each cluster
by the average efficiency found in the previous step and 4) Running a DEA model
with all DMUs and calculate the efficiencies again. For this it was used SIAD
(Integrated System for Decision Support), version 3.0, Meza et al (2005).
Results and discussions
The study considered only ports and terminals with similar location and access
infrastructure, in order to minimize the effects of comparisons with terminals whose
hinterlands have larger markets or that are connected to more efficient modes.
It was obtained 108 DMUs, but only 104 were analyzed, since four units did not
handle any amount of cargo.
Since the goal of the study is to provide subsidies to the planning and development of
public terminals, the performance analysis focuses on the units belonging to clusters
Direct Administration and Rented. The results can help managers to take decisions
about the opportunity for specialization or diversification of cargo handling.
The average efficiencies of ports and terminals analyzed are presented in Table 1.
Performance measures, beside the output targets, suggest the potential of the
inefficient units to increase the volume of cargo handled.
Clusters Direct Administration and Rented concentrated 78.00% of DMUs with
efficiencies lower than 50.00% - 33.33% and 44.67%, respectively. Such information
can guide service level definitions of rented terminals or subsidize feasibility studies
on rent.
The comparison between inefficient terminals and benchmarks contributes to the
identification of decision factors for the best use of structures, assisting the operational
management of terminals operated directly by Public Administration and the
management of contracts for rented terminals. Such factors may be related, among
other things, to the amount of equipment, labor and storage area available.
At strategic level, this performance analysis can be used to evaluate the development
of port infrastructure and determine if a particular port is underused or not, before the
decision to expand it, and provide information for setting the direction and scope of
port activities (BERTOLOTO, 2010).
Proceedings of the 10
th
International Conference on DEA Brazil2012 294
Table 1: Summary table of average efficiencies of ports and terminals
Cluster Port / Terminal
Efficiency Average
Individual Cluster
Direct
Administration
DA1 27,57%
46,35% DA2 60,89%
DA3 50,59%
Rented
R1 67,15%
39,67% R2 9,47%
R3 42,20%
Private
P1 78,91%
57,27% P2 9,05%
P3 83,86%
Conclusions
Based on maximum draft permitted, extension of berths and total cargo handled data
of a group of regional ports and terminals, it was made an analysis of efficiency in
utilization of infrastructure of public terminals located at the state of Esprito Santo,
Brazil.
Ports and terminals were divided into three groups, according to the form of
exploitation they are subjected - Direct Administration, Rented and Private upon
which was applied the mathematical model of Data Envelopment Analysis, using the
compensation method of no homogeneous units presented by Bertoloto (2010).
In the analyzed universe the average performance of private ports is higher than that
of public terminals (Direct Administration and Rented units) and in this latter group,
no specialized terminals presented the highest percentage of DMUs with inefficiencies
lower than 50.00%. The observations suggest that specialization influences terminals
efficiencies, as already stated by Wanke (2009 b).
The results of the study contribute both to the operational management, assisting in
the identification of the determinants of low performance, and to strategic
management, as they provide important information for decision on expansion,
directing port activities and opportunity for specialization of berths.
References
Banker, R. D.; Charnes, A.; Cooper, W.W. (1984) Some models for estimating technical
and scale inefficiencies in data envelopment analysis. Management Science 30 :
1078-1092.#
Bertoloto, R. F. (2010) Eficincia de portos e terminais privativos brasileiros com
caractersticas distintas. 2010. 70 f. Dissertao (Mestrado em Engenharia de Produo)
Programa de Ps-graduao em Engenharia de Produo, Universidade Federal
Fluminense, Niteri.
Proceedings of the 10
th
International Conference on DEA Brazil2012 295
Bloningen, Beferences
. A. e Wilson, W. W. (2008) Port Efficiency and Trade Flows. Review of International
Economic, v. 16, n. 1, p. 21-36.
Brasil (1993) Lei 8.630, de 25 de fevereiro de 1993. Dispe sobre o regime jurdico da
explorao dos portos organizados e das instalaes porturias e d outras
providncias. Available from: <www.planalto.gov.br/ccivil_03/LEIS/l8630.htm>,
retrieved April 22, 2011.
Charnes, A.; Cooper, W. W.; Rhodes, E. (1978) Measuring the efficiency of decision-
making units. European Journal of Operational Research, v. 2, p. 429-444.
Costa, F. M. B. M. (2007) Uma aplicao do Mtodo de Data Envelopment Analysis
DEA para medir a Eficincia Operacional dos Terminais de Contineres. eGesta
Revista Eletrnica de Gesto de Negcios, Brasil e Espanha, v. 3, p. 105-128.
Esprito Santo (2006) Plano de desenvolvimento Esprito Santo 2025: nota tcnica:
desenvolvimento da logstica e dos transportes no Esprito Santo. Macroplan, Esprito
Santo.
Fontes, O. H. P. M.; Soares de Mello, J. C. C. B. S. (2006) Avaliao da eficincia
porturia atravs de uma modelagem DEA. In: IX Simpsio de Pesquisa Operacional e
Logstica da Marinha SPOLM 2006, Rio de Janeiro. Anais do SPOLM 2006.
IPEA. Instituto de Pesquisa Econmica Aplicada (2009) Portos brasileiros 2009:
ranking, rea de influncia, porte e valor agregado mdio dos produtos
movimentados. Rio de Janeiro.
IPEA. Instituto de Pesquisa Econmica Aplicada (2010) Portos brasileiros: diagnstico,
polticas e perspectivas. Rio de Janeiro.
Meza, L. A., Neto, L. B.; Soares de Mello, J. C. C. B.; Gomes, E. G. ISYDS (2005)
Integrated System for Decision Support (SIAD Sistema Integrado de Apoio a
Deciso): a software package for data envelopment analysis model. Pesquisa
Operacional, 2005, v.25, n.3, p. 493-503.
Pires, L. S.; Bertoloto, R. F.; Soares de Mello, J. C. C. B. S. (2009) Anlise da eficincia
de portos de carregamento de minrio de ferro. Rios International Journal on
Sciences of Industrial and Systems Engineering and Management, v.3, p. 094-01.
SEP. Secretaria de Portos da Presidncia da Repblica (2011) Sistema Porturio
Nacional. Available from: <www.portosdobrasil.gov.br/>, retrieved May 20, 2011.
SINDIEX. Sindicato do Comrcio de Exportao e Importao do Estado do Esprito
Santo (2011) Anurio do Comrcio Exterior do Esprito Santo em 2010. Available from:
<http://www.sindiex.org.br/site2008/image_upload/17563413052011anu%E1rio%20201
0%20V1.pdf>, retrieved September 08, 2011.
Proceedings of the 10
th
International Conference on DEA Brazil2012 296
Soares de Mello, J. C. C. B. S.; Meza, L. A.; Gomes, E.G.; Serapio, B. P.; Lins, M. P. E.
(2003) Anlise Envoltria de Dados no estudo da eficincia e dos benchmarks para
companhias areas brasileiras. Pesquisa Operacional, v.23, n.2, p.325-345.
Soares de Mello, J. C. C. B. S.; Meza, L. A.; Gomes, E. G.; Neto, L. B. (2005) Curso de
Anlise Envoltria de Dados. In: XXXVII Simpsio Brasileiro de Pesquisa Operacional
SBPO 2005, Gramado, Anais do SBPO 2005.
Wanke, P. F. (2009 a) Infraestrutura Porturia. In: Wanke, P. F.; Silveira, R. V.; Barros,
F. G.(eds) Introduo ao Planejamento da Infraestrutura e Operaes Porturias:
Aplicaes de Pesquisa Operacional. Atlas, So Paulo.
Wanke, P. F. (2009 b) Avaliando a eficincia dos terminais brasileiros com Anlise
Envoltria de Dados. In: Wanke, P. F.; Silveira, R. V.; Barros, F. G. (eds) Introduo
ao Planejamento da Infraestrutura e Operaes Porturias: Aplicaes de Pesquisa
Operacional. Atlas, So Paulo.
Proceedings of the 10
th
International Conference on DEA Brazil2012 297
37. Benchmarking the Efficiency of
Third Party Logistics in Brazil
Using Data Envelopment
Analysis
Lus Filipe Azevedo de Oliveira
Department of Production Engineering, Federal University of Rio Grande do Norte
luisfilipeao@hotmail.com
Mariana Rodrigues de Almeida
Department of Production Engineering, Federal University of Rio Grande do Norte
almeidamariana@yahoo.com
Abstract
This paper deals with the efficiency evaluation and analysis of 44 Third Party Logistics
(3PLs) providers working in Brazil by means of the Data Envelopment Analysis (DEA).
Since the analyzed companies have diversified sizes, the DEA model considered
variable returns to scales and was oriented to maximize the outputs of 3PLs. The use
of variable selection techniques was critical, in this research, to reach a subset of
variables with greater representativeness. As the main prac- tical result of this study, it
was possible to identify the relative efficiency of 3PLs of national range and their
return to scale. Furthermore, the results are of help in the decision making on
investments based on the enterprises capacity, considering the expected return of
each situation.
Keywords: Thrid Party Logistics, Efficiency, Variable Return to Scale
Introduction
The process of partial or total outsourcing of logistics operations is characterized by
using outside companies to perform typical Third Party Logistics (3PLs) provider
functions, which traditionally used to be performed by the companies themselves
(Sohail and Sohal, 2003; Kayakutlu and Buyukozkan, 2011). This trend is generally
found among the organizations that seek, by outsourcing nonessential activities, the
reduction of internal costs and the service level increase through a raise in the supply
chain management efficiency (Min and Joo, 2006; Seth, et al., 2006).
In accordance with this worldwide phenomenon, a fast growth of logistics activities
has been observed in Brazil, which led companies to reach new competitive levels in
several economies dimensions (Wanke and Affonso, 2011). However, even with an
accelerated growth and with improvements in the national supply chain, Brazil still
has low efficiency when compared to medium or high developed countries (Wanke
and Fleury, 2006).
Proceedings of the 10
th
International Conference on DEA Brazil2012 298
This paper evaluates the efficiency of 44 3PLs working in Brazil, using Data
Envelopment Analysis (DEA). DEA is a non-parametric technique that uses linear
programming to calculate the relative efficiency of a set of similar organizations, which
differ in the inputs and outputs arrangement of their processes. The method results in
a complete ranking, beyond the dichotomized classification of efficient or inefficient,
in order to refine the evaluation of the units. Furthermore, it provides a parameter to
increase the performance of inefficient organizations, showing how they should
evolve to achieve business performance comparable to benchmark enterprises in the
market (Adler, et al., 2002; Cooper et al., 2006; Cook and Seiford, 2009).
This paper presents a framework on three pillars: i) integration of the theme of
logistics and data envelopment analysis, ii) research methods, iii) results of the
analysis.
Efficiency Measurement of Third Party Logistics Providers
The efficiency study of Third Party Logistics (3PLs) providers by means of the Data
Envelopment Analysis (DEA) have had great prominence in current literature (Min and
Joo, 2006; Ding, et al., 2008; Hamdan and Rogers, 2008; Zhou, et al., 2008; Koster, et
al., 2009; Liu and Fu, 2009; Wanke and Affonso, 2011). This fact occurs due to the role
of 3PLs as leverage for the industry development, facing the complex relationships
that exist along the supply chain.
One of the first studies linking efficiency evaluation to 3PLs was conducted by Min
and Joo (2006), developing a meaningful set of financial benchmarks that dictates best
practices, though DEA analysis, applying a non-linear fractional program. The
proposed DEA model helps 3PLs identify potential sources of inefficiency and provide
useful hindsight for the continuous improvement of operational efficiency.
Furthermore, helps 3PLs establish detailed policy guidelines in prioritizing the use of
financial resources and evaluate the effects of financial investment on the profitability
of 3PLs.
Ding, et al. (2008) explain that the traditional DEA model evaluates decision making
units (DMU) from the most beneficial perspective to them, which easily leads to
unreasonable weight assignment. To solve this problem, an improved DEA model
with upper-limit and lower-limit confidence region constraint has been established
based on information entropy. In this context, the paper established an Information
Entropy-DEA model to evaluate 7 logistics suppliers from china.
Hamdan and Rogers (2008) introduce DEA as a tool to evaluate the efficiency of a
group 19 warehouse logistics working in the United States. The relative efficiency
scores for the warehouses used in the study were analyzed before and after the use of
weight restrictions. As a result, it was possible to determine the impact of each input
and output on the efficiency of each warehouse, and also, to examine specific
warehouse characteristics and develop a set of recommendations in the improvement
and design of more efficient operations.
The purpose of Zhou, et al. (2008) was to develop a benchmark for performance
standards for 3PLs in the emerging market. The authors identify the factors that
Proceedings of the 10
th
International Conference on DEA Brazil2012 299
significantly affect the operational efficiency of the Chinese 3PLs, proposing different
ways to improve the competitiveness of them. In particular, it develops both the
Constant Return to Scale (CRS) and the Variable Return to Scale (VRS) model and also
uses step-wise regression analysis to identify factors influencing the performances of
Chinese 3PLs.
In a different perspective, Koster, et al. (2009) applied DEA on primary data of 38
large container terminals in Europe and compared efficiency scores from the
benchmarking exercise with those of previous studies, discussing the reasons behind
diverging results. The authors show that DEA may be appropriate for container
terminal benchmarking, but only if better quality and additional input and output data
can be obtained.
Liu and Fu (2009) compared the results of CRS and VRS input oriented DEA models,
calculating both the technical efficiency and scale efficiency, and also the returns to
scale for each of the 16 logistics public companies from China. By comparing the
efficiency of logistics companies, they could contribute to find the 3PLs weaknesses in
order to take effective measures to improve their level of input and output so as to
improve its logistics efficiency.
Wanke and Affonso (2011) studied the 3PLs industry in Brazil. Its main objective was
to determine the variables that significantly impact on the 3PL scale efficiency by
means of a two-stage DEA. The inputs and outputs required for this analysis were
identified as well as the contextual variables that may impact on the 3PL scale
efficiency. The results corroborate the evidence in the literature on the role of
coordination processes in logistics performance.
Methodology
The research was conducted with data from the special issue Operadores Logsticos
(2011), published annually by the brazilian magazine Tecnologstica, which shows the
main companies from the Third Party Logistics (3PLs) providers market in Brazil. The
analysis contemplates the universe of companies working in Brazil and different sized
businesses.
The analysis took into account four input variables, as follows: i) number of
employees; ii) total area used for storage; iii) number of warehouses owned by the
companies; and iv) number of warehouses within the client facilities. As a result of the
outputs, the following variables were considered: i) number of customers under
contract; ii) companys operating revenue; iii) total managed products volume; and iv)
revenue growth presented by the company as compared with the previous year.
According to Dyson, et al. (2001), the set of Decision Making Units (DMU) must be
homogeneous regarding to the activities they perform, being able to run similar tasks
with the same goals and also presenting the same inputs and outputs, with varying
intensity of these variables. In other words, the DEA methodology requires common
variables among the 3PLs to compare them, making it necessary to obtain a list of
organizations, limited by data availability. In that sense, a sample of 44 3PLs was
Proceedings of the 10
th
International Conference on DEA Brazil2012 300
extracted from the original set of 126 companies, taking as selection criteria those that
provided the information relevant to the chosen input and output variables.
The DEA model was oriented to maximize the outputs of 3PLs and considered
variable returns to scales, since the analyzed companies have different sizes and
thereby have technologies with constant, increasing and decreasing returns to scale.
This model was first proposed by Banker, et al. (1984) and, for such conditions, can
be described as Expression 1.
Min
0
+
0
=1
(1)
Subject to:
=1
= 1
=1
0
=1
0;
= 1, , ,
0; = 1, , ;
= 1, , .
Here the y
rj
and x
ij
, all positive, are the known outputs and inputs from a set of j =
1,, n organizations. The data y
rj
and x
ij
are constants and will usually be
observations from past decisions on inputs and the outputs that resulted therefrom.
The variables v
i
, u
r
0 are weights to be determined by the solution of this problem,
by the data on all of the DMU's which are being used as a reference set. It is worth
noting the presence of variable v
0
, representing the return to scale presented by the
analyzed unit 0, determining whether the operations are conducted with increasing,
constant or decreasing returns to scale. Thus, in case v > 0, returns to scale are
decreasing; if v = 0, returns are constant; and if v < 0, returns will be increasing.
The efficiency measure of any DMU is obtained as the inverse of the minimum of the
weighted inputs and rated relative to the others from the set that accords the most
favorable weighting that the constraints allow. Thus, the weights are objectively
determined to obtain a scalar measure of efficiency in any case. The constraints
provide that ratios of weighted outputs to weighted inputs for every DMU be less than
or equal to unity.
Efficiency of Third Party Logistics in Brazil
The quantitative analysis of this research considers the parameters to evaluate the
efficiency of Third Party Logistics (3PLs) providers in Brazil, with a practical
and measurable focus. The purpose of this analysis is to access and compare the
measurable parameters of 3PLs by the link among the conditions offered by them
such as inputs and/or supplies and their volume and productivity responses as
output and/or products triggered.
Proceedings of the 10
th
International Conference on DEA Brazil2012 301
According to the proposed model, it was obtained an average technical efficiency of
75.74%, a minimum efficiency rate of 13.14% and the standard deviation (SD) for the
sample was 28.05. The results show that 20 3PLs were classified as efficient since their
efficiency score were 100%, which implies the inefficiency status for 24 3PLs, that
represents 55% of the sample (Table 1).
Table 1: Performance evaluation of 3PLs
Performance Evaluation
Average 75.74%,
SD 28.05
Minimum 13.14%
100% 20
99 70% 7
69 40% 9
Low 39% 8
Using the company's size classification adopted by the Banco Nacional do
Desenvolvimento (BNDES, which in a free translation means National Development
Bank), that considers the Annual Gross Revenue, its possible to note that, among the
successful firms, six are small (30%), three are medium-sized companies ( 15%), seven
are medium to large (35%) and four are large companies (20%).
Each return to scale phase presents a specific denotation to support the results, helps
in the decision making on investments based on the enterprises capacity, considering
the expected return of each situation (Table 2). The increasing scale implies that in
raising input variables, a large (disproportional) increase in output variables for the 11
companies that present this kind of return to scale. Therefore, to invest on capacity
growth of these companies is attractive because it would bring a rapid growth in
national performance, provided by economies of scale likely to be achieved for 25%
of 3PLs examined.
Table 2: Evaluation of the scale of 3PLs
Return to
Scale
Scale Evaluation
Total
Efficiency
100% <100%
Increasing 11 5 6
Constant 0 0 0
Decreasing 33 15 18
Proceedings of the 10
th
International Conference on DEA Brazil2012 302
Considering the efficiency score of this group, the result shows that five companies
that are technically efficient presented increasing scale of production, indicating
that despite the absence of inputs in excess, the production volume is below the
optimal range. This means that such units can increase production by raising the
amount of resources required, moving the efficiency frontier away from the analyzed
group.
Noting the inefficient firms that have an increasing return to scale, they represent six
organizations on the sample. These companies operate below optimal scale, and
such units can also increase their production level by raising the amount of resources
to achieve the optimum efficiency.
Running on a decreasing scale, with 33 3PLs from the sample, the resources used
exceed the optimum level. It is not recommended to obtain more resources to
increase the production capacity by 75% of firms, but to reduce the excesses. These
aspects would bring benefits proportionately lower than the rates of resources
increasing.
The 15 units that are efficient and have decreasing returns to scale operate above
the optimal scale, which implies the need to reduce its volume of production,
considering that the increase in production occurs in this situation by adding a larger
quantity of inputs needed, which becomes economically unattractive. Therefore, to
achieve optimum efficiency, it is necessary to reduce the excessive use of
input, maintaining the production level.
Finally, it is observed that 18 organizations are inefficient and have decreasing
returns to scale. To increase their technical efficiency, these companies must eliminate
the use of excess raw materials, while increasing the production level.
Results and Conclusions
The growth of the logistics operations complexity and the demand for efficiency gains
in performing such tasks led to the Third Party Logistics (3PLs) providers appearance.
These are companies that take a strategic role within organizations, since they allow
costs to be reduced and help improve service level.
This paper focused on the efficiency evaluation and analysis of 44 3PLs in operation
in Brazil by means of the Data Envelopment Analysis (DEA). As the main practical
result of this study, it was possible to identify the relative efficiency of 3PLs of national
range and their return to scale. Based on the constructed model results, 20 3PLs were
identified as efficient and representative of national benchmarks by their optimal
relationship between productivity and resource usage.
About the scale in which each 3PLs operates, it was identified that 11 work on an
increasing scale and 33 on a decreasing scale, with no occurrence of constant
performance. These results are helpful in making decisions on investments based on
the enterprises capacity, considering the expected return of each situation.
Proceedings of the 10
th
International Conference on DEA Brazil2012 303
When faced with these results, managers can select the best players concerning the
efficiency of production processes based on a ranking that lists the best 3PLs and
assess the efficiency regarding its strategic objectives, in order to drive companies to
best practices and superior performance.
Therefore, these results help organizations in making decisions regarding resources
allocation in order to optimize and distribute them to the appropriate channels, which
justifies the use of DEA as a tool to contribute to the 3PLs pursuit of excellence.
REFERENCE
Adler, N. et al. (2002), Review of ranking methods in the data envelopment analysis
context, European Journal of Operational Research, Vol. 140, No. 2, pp. 249-265.
Banker, R.D. et al. (1984), Some models for estimating technical and scale
inefficiencies in data envelopment analysis, Management Science, Vol. 30, No. 9, pp.
1078-1092.
Cook, W.D., Seiford, L.M. (2009), Data envelopment analysis (DEA): Thirty years on,
European Journal of Operational Research, Vol. 192, No. 1, pp. 1-17.
Cooper, W. et al. (2006), Introduction to Data Envelopment Analysis and Its Uses: With
DEA-Solver Software and References, Springer, New York, NY.
Ding, B. et al. (2008), Third-party Logistics Provider Efficiency Evaluation Based on
Information Entropy-DEA Model, in International Seminar on Future Information
Technology and Management Engineering, Leicestershire, 2008, IEEE, Washington, pp.
166-170.
Dyson, R.G. et al. (2001), Pitfalls and protocols in DEA, European Journal of
Operational Research, Vol. 132, No. 2, pp. 245-259.
Hamdan, A., Rogers, K.J.J. (2008), Evaluating the efficiency of 3PL logistics
operations, International Journal of Production Economics, Vol. 113, No. 1, pp. 235-
244.
Koster, M.B.M. et al. (2009), On using DEA for benchmarking container terminals,
International Journal of Operations & Production Management, Vol. 29, No. 11, pp.
1140-1155.
Kayakutlu, G., Buyukozkan, G. (2011), Assessing performance factors for a 3PL in a
value chain, International Journal of Production Economics, Vol. 131, No. 2, pp. 441-
452.
Liu, B., Fu, S. (2009), Efficiency Measurement of Logistics Public Companies Basing
on the Modified DEA Model, in International Conference on Computational
Intelligence and Security, Beijing, 2009, IEEE, Washington, pp. 601-605.
Proceedings of the 10
th
International Conference on DEA Brazil2012 304
Min, H., Joo, S.J. (2006), Benchmarking the operational efficiency of third party
logistics providers using data envelopment analysis, Supply Chain Management: An
International Journal, Vol. 11, No. 3, pp. 259-265.
Operadores Logsticos (2011), Tecnologstica, Vol. 184, No. 3, pp. 60-138.
Sohail, M.S., Sohal, A.S. (2003), The use of third party logistics services: a Malaysian
perspective, Technovation, Vol. 23, No. 5, pp. 401-408.
Seth, N. et al. (2006), A conceptual model for quality of service in the supply chain,
International Journal of Physical Distribution & Logistics Management, Vol. 36, No. 7,
pp. 547-575.
Wanke, P.F., Affonso, C.R. (2011), Determinantes da eficincia de escala no setor
brasileiro de operadores logsticos, Produo, Vol. 21, No. 1, p. 53-63.
Wanke, P.F., Fleury, P.F. (2006), Transporte de cargas no Brasil: estudo exploratrio
das principais variveis relacionadas aos diferentes modais e s suas estruturas de
custos, in Negri, J.A., Kubota, L.C. Estrutura e dinmica do setor de servios no Brasil,
IPEA, Braslia, pp 409-464.
Zhou, G. et al. (2008), Evaluating the comparative efficiency of Chinese third-party
logistics providers using data envelopment analysis, International Journal of Physical
Distribution & Logistics Management, Vol. 38, No. 4, pp. 262-279.
Zhu, J. (2008), Quantitative Models for Performance Evaluation and Benchmarking:
Data Envelopment Analysis with Spreadsheets, Springer, Nova York, NY.
Proceedings of the 10
th
International Conference on DEA Brazil2012 305
DATA ENVELOPMENT ANALYSIS: THEORY AND
APPLICATIONS
10
th
International Conference on Data Envelopment Anal ysis
(DEA2012)
ISBN: 978 185449 437 5
International Data Envelopment
Analysis Society