Measurement 65 (2015) 2940

Contents lists available at ScienceDirect

Measurement
journal homepage: www.elsevier.com/locate/measurement

Two alternative approaches for selecting performance measures

in data envelopment analysis
Mehdi Toloo , Tom Tichy
Faculty of Economics, Technical University of Ostrava, Ostrava, Czech Republic

a r t i c l e i n f o a b s t r a c t

Article history: Data envelopment analysis seeks a frontier to envelop all data with data acting in a critical
Received 10 August 2014 role in the process and in such a way measures the relative efciency of each decision mak-
Received in revised form 24 November 2014 ing unit in comparison with other units. There is a statistical and empirical rule that if the
Accepted 23 December 2014
number of performance measures is high in comparison with the number of units, then a
Available online 8 January 2015
large percentage of the units will be determined as efcient, which is obviously a question-
able result. It also implies that the selection of performance measures is very crucial for
Keywords:
successful applications. In this paper, we extend both multiplier and envelopment forms
Decision making
Data envelopment analysis
of data envelopment analysis models and propose two alternative approaches for selecting
Multiplier and envelopment forms performance measures under variable returns to scale. The multiplier form of selecting
The rule of thumb model leads to the maximum efciency scores and the maximum discrimination between
Selective measures efcient units is achieved by applying the envelopment form. Also individual unit and
Banking industry aggregate models are formulated separately to develop the idea of selective measures.
Finally, in order to illustrate the potential of the proposed approaches a case study using
a data from a banking industry in the Czech Republic is utilized.
2014 Elsevier Ltd. All rights reserved.

1. Introduction
Data envelopment analysis (DEA), as a well-known non-
parametric mathematical approach, evaluates the relative
efciency score of a set of homogeneous decision making
units (DMUs), which utilize the multiple inputs to produce
multiple outputs. In managerial applications, DMUs may
include banks, department stores and supermarkets, and
extend to car makers, hospitals, schools, public libraries
and even portfolios. In engineering, DMUs may take such
forms as airplanes or their components such as jet engines.
Basic DEA models measure the efciency of a DMU by
maximizing the ratio of the weighted sum of its outputs
to the weighted sum of its inputs, based on the condition
Corresponding author at: Sokolska trida 33, 701 21 Ostrava 1, Ostrava,
Czech Republic. Tel.: +420 597 322 009.
E-mail addresses: mehdi.toloo@vsb.cz, m_toloo@yahoo.com
(M. Toloo).
that this ratio is less than or equal to one for all DMUs.
The large number of studies has been accomplished in
DEA which show that this is an outstanding and straight-
forward methodology for modeling operational process in
performance evaluations. Nowadays, DEA is becoming a
very important analysis tool and research method in man-
agement science, operational research, system engineer-
ing, decision analysis, etc.
Cooper et al. [14] considered a rough rule of thumb,
which expresses the relation between the number of DMUs
and the number of performance measures to have a reli-
able results. In other words, when the number of perfor-
mance measures is high in comparison with the number
of DMUs, then most DMUs are classied as efcient, which
is not logical. Therefore in this paper we propose an
approach to deal with this issue. In particular, we extend
standard models so that important measures (i.e., inputs
and outputs) can be identied, their total number
decreased and complex result optimized.

http://dx.doi.org/10.1016/j.measurement.2014.12.043
0263-2241/ 2014 Elsevier Ltd. All rights reserved.
30 M. Toloo, T. Tichy / Measurement 65 (2015) 2940

The rest of the paper is organized as follows: Section 2 3. The BCC model
summarizes the rule of thumb in DEA. In Section 3, the
multiplier and envelopment forms of BCC model is pre- Charnes et al. [8] proposed an innovative mathematical
sented. Section 4 introduces a new approach to choose approach for evaluating the relative efciencies among
selective measures. Section 5 illustrates the capabilities DMUs with multi-input and multi-output as a linear pro-
of our proposed approach by using a real data set of bank- gramming model. This approach, which is referred to as
ing industry in the Czech Republic. Conclusions and further CCR (Charnes, Cooper and Rhodes), assumed constant
remarks are provided in the last section. returns to scale (CRS) technology. Banker et al. [3] sug-
gested a new model, which is known as BCC (Banker, Char-
nes and Cooper), to deal with VRS situation.
2. The rule of thumb in DEA
Consider a set of n decision making units (DMUs), each
consuming various amounts of m inputs to produce s out-
Suppose there are n DMUs that consume m inputs to
puts. Let xj = (x1j, . . . , xmj)T and yj = (y1j, . . . , ysj)T represent the
produce s outputs. If a performance measure (input/out-
input and output vectors for DMUj (j = 1, . . . , n), respec-
put) is added or deleted from consideration, it will inu-
tively. There are two BCC forms which are mutually dual:
ence the relative efciencies. Empirically, when the
multiplier and envelopment.
number of performance measures is high in comparison
with the number of DMUs, then most DMUs are evaluated
efcient so that such results cannot be regarded as reliable. Multiplier form of BCC Envelopment form of BCC
m 
A rough rule of thumb expresses the relation between the Ps P x P s
maxz1 ur yro uo maxz2 h  e si syr
number of DMUs and the number of performance mea- r1 i1 r1

sures (see Cooper et al. [14] for more details): s:t: s:t:
P
m P
n
v i xio 1 kj xij sxi hxio 8i
n P maxf3m s; m  sg i1 j1 2
P
s P
m P
n

Table C.1 in Appendix C practically represents the num- ur yrj u0  v i xij 6 0 8j 1 kj yrj  syr yro 8r
r1 i1 j1
ber of DMUs and the number of performance measures P
n

applied in some studies (for more details see [1,47,10 v i P e 8i kj 1

13,1553]).
There are some cases that the number of performance
measures and DMUs does not satisfy the rule of thumb.
To rectify this issue, in many scenarios we select some per-
formance measures in a manner which complies with the
rule of thumb and imposes progressive effect on the ef-
ciency scores. These selected inputs and outputs are
known as selective measures implies
Establishing criteria among a set of suggested standards
is an important issue for the decision maker. For example,
consider the problem of evaluating 50 branches of a bank;
in such a setting, the manager may practically run into
more than 25 inputs and 30 outputs. Apparently, the total
number of measures, i.e. 55, and DMUs do not satisfy the
rule of thumb and subsequently to have an acceptable ef-
ciency scores some performance measures must be omit-
ted. To make the problem easier, suppose that the
manager pre-selected three inputs, e.g. employees,
expenses and space, and three outputs, e.g. loans, prots
and deposits. With this assumptions, if the manager wants
to select 2 out of 22 remaining inputs and 1 out of 27
remaining outputs and also consider all possible combina-
tions of performance measures, then an optimization prob-
lem must be solved at most
    
22 27
196350 50   times, which shows that
2 1
this approach is illogical.
In this paper, we propose an approach to deal with the
selective measures under variable returns to scale (VRS)
assumption. However, in some situations, some special
measures must appear in evaluation model, so that the
managers concern comes true. These measures are named
xed measures and we also consider them in developing
our approach.
j1
ur P e 8r kj P 0 8 j;sxi P 0 8i;syr P 0 8r
where DMUo = (xo, yo) is being evaluated (o e {1, 2, . . . , n}).
In the multiplier form of BCC model (1), u = (u1, . . . , us)
and v = (v1, . . . , vm) are the set of output and input weights
(or multipliers, dual variables or shadow prices), respec-
tively, and u0 is a variable with free in sign, i.e. u0 -
e (1, +1). Charnes et al. [9] added e, which is called
the non-Archimedean epsilon, to the CCR model to have
positive weights. It is important to set a suitable value
for the non-Archimedean epsilon e. Amin and Toloo [2]
designed a polynomial-time algorithm to obtain an appro-
priate value for epsilon. This model maximizes the rela-
tive efciency score of DMUo subject to the condition
that the similar score for all DMUs must be less than or
equal to one. Suppose that model (1) is solved and the
optimal solution (u, v, u0) is at hand, DMUo is efcient
when z1 1 and otherwise is inefcient. As a result,
DEA models can categorize all DMUs into two different
groups: efcient and inefcient. Toloo et al. [47] sug-
gested an efcient approach to nd an initial basic feasi-
ble solution for the multiplier form of BCC model and
practically showed it decreases required computations
by at least one half.
On the other hand, the envelopment form of BCC model
P Pn  
(2) seeks a (virtual) unit n 
ji kj xj ; j1 kj y j that guaran-
tees at least the output level yo of DMUo in all components
P 
n 
j1 kj yrj P yro ; r 1; . . . ; m , while reducing the input
vector xo proportionally to a value as small as possible
P 
n   
j1 kj xij 6 h xio ; i 1; . . . ; m . If z2 1 (or equivalently

ho 1, 8i sxi 0 and 8r syr 0), then DMUo is efcient.

 M. Toloo, T. Tichy / Measurement 65 (2015) 2940 31

Otherwise it is inefcient, which implies that performance measures from the remaining inputs and out-
P Pn  
n 
outperforms (hxo, yo). puts (selective measures). In the case of no preferences
ji kj xj ; j1 kj y j
between different inputs (or outputs), the formulated
The multiplier form of BCC model (1) and the envelop-
model covers all performance measures.
ment form of BCC model (2) differ from similar forms of
In the following subsections, we extend two multiplier
the CCR model by their inclusion of the free variable u0
P and envelopment forms of selecting model. In particular,
and the convexity constraint nj1 kj 1, respectively. On
we develop two individual and aggregate models for each
the other hand, since these two models, multiple and
forms of selecting models. In individual models, the perfor-
envelopment, are mutually dual, from dual optimality con-
mance of each DMU is considered, meanwhile aggregate
ditions for linear programming problems z1 z2 . For a dee-
models deal with overall performance of the collection of
per discussion of multiplier and envelopment forms of BCC
DMUs. To formulate new models we utilize auxiliary bin-
model we refer the reader to Cooper et al. [14]. As will be
ary variable and subsequently proposed models are MIP.
shown, we formulate two multiplier and envelopment
forms of selecting models that are mixed integer linear
4.1. Selective measures and individual DMUs
programs (MIP) and hence their optimal objective values
are not identical.
Let s1 and s2 denote subsets of outputs corresponding to
One should note that to have accurate results, the num-
xed-output and selective-output measures, respectively.
ber of inputs and outputs, and DMUs must support the rule
Similarly, assume that m1 and m2 are the parallel subsets
of thumb, and if this condition will not be met, the results
of inputs. We extended two multiplier and envelopment
are not reliable. In the next section, we propose two forms
forms of BCC models (1) and (2) to obtain two different
of selecting DEA model that can handle the situations, in
approaches for measuring the efciency score of DMUo
which the manager has to select some of the performance
with keeping the rule of thumb.
measures among suggested inputs and outputs.
4.1.1. Multiplier form of selecting model
4. The proposed approach We extend the multiplier form of the BCC model and
propose the following multiplier form of selecting model
Consider n DMUs, each using m inputs to produce s out- to deal with selective measures:
puts, where n < max{3(m + s), m  s}. In this situation, one X
s
approach for having a sharper discrimination among DMUs max z3 ur yro u0
is adopt an assurance region method or the cone ratio r1

model (see Cooper et al. [14]). However, a decision maker s:t:

has to have more information about the value of these Xm

inputs and outputs and subsequently the nal result is v i xio 1

i1
closely related to these facts. In the lack of such informa-
tion, two main methods can be applied to hold property
Xs X
m
ur yrj u0  v i xij 6 0 j 1; . . . ; n
(2): increasing the number of units (n) or decreasing the r1 i1
3
number of performance measures (m + s). In the rst X X nhni po
y x
method, we add some more DMUs for the evaluation, br bi 6 min ; 2 n  jm1 j js1 j
r2s2 i2m2
3
meanwhile in the second method we ignore some inputs x x
and outputs. In practice, it is hardly possible to increase ebi 6 i 6 Mbiv i 2 m2
y y
the number of DMUs and consequently we resort to formu- ebr 6 ur 6 Mbr r 2 s2
late some models which optimally decrease the number of x y
bi ; br 2 f0; 1g i 2 m2 ; r 2 s2
performance measures.
Firstly suppose 3(m + s) = max{3(m + s), m  s}. In this v i ; ur P e i 2 m1 ; r 2 s1
case, m + s  [n/3] performance measures should be where e > 0 is a small number and M is a large positive
ignored in a way that the efciency scores of all DMUs number. In fact, the model (3) is an extended version of
decrease as much as possible. Note that decreasing the ef- the proposed model in [44] to consider VRS assumption.
ciency score of all DMUs leads to maximum discrimination For each selective input and output measure we introduce
among efcient DMUs. If one considers all possible input x y
the binary variables bi and br , respectively. What is left is
and output combinations, then model (2) must be solved to prove that the rule of thumb holds when one uses
  
Pn=31 m s model (3).
n  i1 times. In a similar manner,
n=3  i i
the number of times that model (2) must be solved is Theorem 1. The selecting model (3) will meet the rule of
  
P m s thumb.
n  ni1 when m  s = max{3(m + s), m  s}.
n=i i
Apparently, this method is inappropriate even for not a
very large set of performance measures. In this study, a Proof. See Appendix A. h
new approach is proposed to tackle this issue.
In some cases, the decision maker believes that some It should be mentioned here that if n < 36, then we have
p P y P x
performance measures are more important than others n=3 < 2 n and hence the constraint r2s2 br i2m2 bi 6
p
and we x these measures (as xed measures), and select minfn=3; 2 ng  jm1 j js1 j leads to the constraint
32 M. Toloo, T. Tichy / Measurement 65 (2015) 2940

Ps y Pm x n X
n
r1 br i1 bi
 jm1 j js1 j and otherwise it
6
Ps y
3
Pm x p kj yrj  syr yro r 2 s1
implies the constraint r1 br i1 bi 6 2 n  jm1 j j1
js1 j. X
n
x
Now suppose that a manger is interested in selecting kj xij sxi hxio M1  di i 2 m2
exactly p out of m2 selective inputs and q out of s2 selective j1
p
outputs, where p q 6 minfn=3; 2 ng  jm1 j js1 j. This X
n
y
conditions will be easily established in model (3), when the kj yrj  syr yro  M1  dr r 2 s2
P y P x p j1
constraint r2s2 br i2m2 bi 6 minfn=3; 2 ng  jm1 j X
P 
x
di p
js1 j is replaced with two constraints i2m2 bi p and
P  i2m2
r2s2 br q. Note that to select input and output measures, X y
we rst solve selecting model (3) and obtain a set of optimal dr q
x y
r2s2
bi and br for each DMU. If for the majority of the units
x y
X
n
bi 1br 1, then we select it as an input (output) mea- kj 1
sure. Now, the multiplier form of BCC model (1) can be used j1

to evaluate DMUs with the xed and selective measures. x y

di ; dr 2 f0; 1g i 2 m2 ; r 2 s2
If there is no priority in choosing selective measures
sxi ; syr P0 8i; 8r
(s1 = m1 = /), we propose following selecting model:
kj P 0 8j 5
X
s
max z4 ur yro u0 where M is a very large positive number and also p and q
r1
are two positive parameters such that p q 6
s:t:   p
min n3 ; 2 n  jm1 j js1 j.
Xm
In order to show the contribution of proposed model
v i xio 1
i1
(5), we state and prove a theorem which ensures the cor-
Xs X
m rect efciency score by keeping the rule of thumb:
ur yrj u0  v i xij 6 0 j 1; . . . ; n
r1 i1 Theorem 2. The presented model (5) will meet the rule of
X X nhni po
br
y x
bi 6 min ;2 n 4 thumb.
r2s2 i2m2
3
x x
e 6 vi 6
bi Mbi i 1; . . . ; m Proof. See Appendix A. h
y y
e 6 ur 6
br Mbr r 1; . . . ; s Now, we eliminate the nonlinear terms from the formu-
X
s
y
lated mixed integer nonlinear programming model (5).
br P1 Toward this end, we replace the nonlinear term by a new
r1
x y
variable on which a number of constraints is imposed. First
bi ; br 2 f0; 1g i 1; . . . ; m; r 1; . . . ; s x
for i e m2 let t xi di sxi and clearly what is left is to add some
v i ; ur P e i 1; . . . ; m; r 1; . . . ; s x
suitable constraint to have txi sxi when di 1 and other-
x
P y wise ti 0. This can be done by imposing the following
The constraint sr1 br P 1 is added to have at least one
P x restrictions to the model:
output measure. Note that the condition m i1 bi P 1 can be x
Pm
0 6 txi 6 Mdi
extracted form the rst constraint r1 v i xio 1 . Similar x
to the proof of Theorem 1, one can prove that model (4) sxi  M1  di 6 t xi 6 sxi
fullls the rule of thumb. where M is a very large positive number. If di 0, then the
x

x
constraint sxi  M1  di 6 t xi 6 sxi is redundant and on the
4.1.2. Envelopment form of selecting model other hand from the constraint 0 6 t xi 6 Mdi we have
x

In traditional DEA models there are two forms, i.e. mul-  x x x

ti 0. Otherwise (i.e. di 1) 0 6 ti 6 Mdi is redundant
tiplier and envelopment, which are mutually dual. This x
and the constraint sxi  M1  di 6 txi 6 sxi converts to
section extends the envelopment form of selecting model. x x x x x
si 6 ti 6 si and hence t i si . In a similar manner, we let
As will be shown subsequently, although different forms of y
tyr dr syr for r e s2 and impose the following restrictions
selecting models are not mutually dual, they are related to the model:
together. y
We formulate the following envelopment form of 0 6 tyr 6 Mdr
y
selecting model: syr  M1  dr 6 t yr 6 syr
! !
X X X x
X y As a result, the following mixed integer linear program-
max z5 h  e sxi syr e di sxi dr syr ming is formulated:
i2m1 r2s1 i2m2 r2s2
! !
s:t: X X X X
max z6 h  e sxi syr e t xi t yr
Xn
i2m1 r2s1 i2m2 r2s2
kj xij sxi hxio i 2 m1
j1 s:t:
 M. Toloo, T. Tichy / Measurement 65 (2015) 2940 33

 p
X
n
where p P 1; q P 1; p q 6 min n3; 2 n  jm1 j js1 j.
kj xij sxi hxio i 2 m1 Notice that from p P 1 (q P 1) we have at least one input
j1
(output).
X
n
kj yrj  syr yro r 2 s1 It is worth noting that unlike the multiplier and envel-
j1 opment forms of BCC model which are LP, the proposed
Xn
x
forms of selecting models are MIP. As a result, the optimal
kj xij sxi hxio M1  di i 2 m2 objective value of the multiplier form of selecting model
j1
(3), z4 , is not equal to the optimal objective value of the
X
n
y
kj yrj  syr yro  M1  dr r 2 s2 envelopment form model (6), z6 . The following theorem
j1 expresses the relation between these values:
X x
di p
Theorem 3. 0 < z6 6 z3 6 1.
i2m2
X y
dr q Proof. See Appendix A. h
r2s2
Xn
Indeed, these forms are not mutually dual and, as will
kj 1
j1
be illustrated by a real data set in banking industry, multi-
x plier form of selecting model selects the selective measures
0  t xi  Mdi i 2 m2
x based on the maximum performance whereas the maxi-
sxi  M1  di  t xi  sxi i 2 m2
y
mum discrimination among DMUs is considered in envel-
0  t yr  Mdr i 2 s2 opment form.
y
sr  M1  dr  t yr  syr
y
r 2 s2 In the proposed individual approaches, the manager
x y
di ; dr 2 f0; 1g i 2 m2 ; r 2 s2 solves the proposed multiplier or envelopment forms of
x y selecting models for each DMU and obtains a set of optimal
ti ; tr  0 i 2 m2 ; r 2 s2
x y x y
sxi ; syr  0 8i; 8r bi and br (or di and dr ) specic to that DMU. One criterion
kj  0 8j 6 for selecting measures would then be to base it on the
majority choice among the DMUs. Meanwhile in aggregate
If the manager has no priority over choosing selective approach, the manager makes decision based on a set of
x y x y
measures, in model (6) we can set s1 = m1 = / to achieve obtained optimal bi and br (or di and dr ) specic to all
following envelopment form of selecting model: DMUs. Such a model would be helpful if ties are encountered
! using the proposed selecting models (37) on an individual
X
m X
s
max z7 h  e txi tyr DMU basis. In the following subsection, we adopt this aggre-
j1 r1 gate approach for both multiplier and envelopment forms.
s:t:
Xn 4.2. Selective measures and aggregate efciency
kj xij sxi hxio i 1; . . . ; m
j1 The overall performance of the collection of DMUs can
X
n
be considered as an alternative approach for selecting per-
kj yrj  syr yro r 1; . . . ; s
j1
formance measures. To follow this viewpoint, we examine
Xn the efciency of the aggregate outputs to aggregate inputs
x
kj xij sxi hxio M1  di i 1; . . . ; m with the outlook on selective measures in this part. Specif-
j1 ically, we suggest the following MIPs to obtain the aggre-
Xn
y gate efciency in multiplier and envelopment forms:
kj yrj  syr yro  M1  dr r 1; . . . ; s
j1 Aggregate multiplier form
Xm
x Xs
di p 7 max z8 ~r u0
ur y
i1 r1
Xs
y
dr q s:t:
r1 Xm

Xn v i ~xi 1
kj 1 i1
j1 Xs X
m

txi
0   Mdi
x
i 1; . . . ; m ur yrj u0  v i xij 6 0 8j
x r1 i1
sxi  M1  di  txi  sxi i 1; . . . ; m X y X x nhni po
y y br bi 6 min ;2 n
0  tr  Mdr r 1; . . . ; s r2s
3
2 i2m 2
y
syr  M1  dr  tyr  syr r 1; . . . ; s
x y
ebxi 6 v i 6 Mbxi 8i
di ; dr 2 f0; 1g i 1; . . . ; m; r 1; . . . ; s
ebyr 6 ur 6 Mbyr 8r
txi ; t yr  0 i 1; . . . ; m; r 1; . . . ; s
X
s
sxi ; syr  0 i 1; . . . ; m; r 1; . . . ; s y
br P 1
kj  0 j 1; . . . ; n r1
34 M. Toloo, T. Tichy / Measurement 65 (2015) 2940

x y
bi ; br 2 f0; 1g 8i; 8r nancial institutions from Eurozone) with just a few large
v i ; ur P e 8i; 8r 8 banks, although it can currently be regarded as highly
competitive, especially due to aggressive policy of small
Aggregate envelopment form banks in the retail segment.
! Even if data of some banks were omitted, Table 1 clearly
Xm X
s
max z9 h  e x
ti y
tr shows that just three or four large banks make the market.
i1 r1 Even though, the market can currently be regarded as com-
s:t: petitive and innovative due to the presence of small banks.
Note nally, that the Czech banking sector can be regarded
Xn
kj xij sxi h~xi 8i as very stable and no serious impact of recent subprime
j1 crises was recorded, mainly due to public rescue of large
X
n banks in late 90s and their subsequent acquisition of Euro-
kj yrj  syr y
~r 8r pean nancial institutions.
j1 In order to select the most important bank features, we
X
n
x follow Mostafa [32] who investigated 26 research papers
kj xij sxi h~xi M1  di 8i done on banking industry in different countries. The inputs
j1
and outputs that were applied in these studies are shown
X
n
kj yrj  syr y
~r  M1  dr 8r y in Table D.1 in Appendix D. Reference to the table shows
j1 that in more than 80% of the bank studies employee is con-
X
m sidered as an input and subsequently it is reasonable to
x
di p 9 consider it as a xed input. Considering the managers
i1 interests, availability of information and the most utilized
Xs
y measures in Table D.1, the inputs and outputs measures
dr q are suggested as follows:
r1
Inputs: Employees, Number of branches, Assets, Equity,
Xn
kj 1 Expenses.
j1 Outputs: Deposits, Loans, Non-interest income, Interest
0  txi  Mdi
x
8i income.
x Table 1 exhibits the data set and the BCC-efciency
sxi  M1  di  t xi  sxi 8i score. The last column in the table shows that 93% of banks
y
0  tyr  Mdr 8r (i.e. 13 of 14) are efcient. Such result is questionable and
y is due to an inadequate number of performance measures.
syr  M1  dr  t yr  syr 8r
x y To get an acceptable result, we have to decrease the num-
di ; dr 2 f0; 1g 8i; 8r
ber of performance measure such that n excess 3(m + s).
x y
ti ; tr  0 8i; 8r Toward this end, we rst apply the multiplier form of
sxi ; syr  0 8i; 8r selecting model (3) to the data set in Table 1. Reference
kj  0 8j to the optimal solution shows that Expenses is selected
input and Deposits and Loans are selected outputs.
where Table 2 summarizes all xed and selected data and also
X
n acceptable efciency score which is obtained by the BCC
~x xij i 1; . . . ; m model. As can be extracted from this table, there are 5 ef-
j1 cient banks out of 14 banks which shows the proposed
X
n
multiplier model is succeed in decreasing the percentage
~
y yrj r 1; . . . ; s of efcient DMUs from 93% to 36% with the maximum
j1
number of acceptable efcient DMU. Nevertheless, as will
In the next section, we utilize a real data set of banking be illustrated, the proposed envelopment form can
industry in order to show the applicability of the proposed decrease the number of efcient DMUs even more than
multiplier and envelopment approaches. multiplier form, but with the minimum number of accept-
able efcient DMU.
5. Application Notwithstanding the obtained BCC-efciency scores in
Table 1, these scores in Table 2 illustrate that the proposed
This section illustrates the proposed approach through multiplier form of selecting model is succeed in holding the
a real data set obtained from annual reports of 14 banks rule of thumb and achieving the satisfactory efciency
active in the Czech Republic (see Appendix B for their brief scores.
description). Following the ndings of Mostafa [32] we To decrease the number of efcient DMUs and discrim-
pick up the most important measures to analyze the ef- inate between them we apply the proposed envelopment
ciency of banking sector in the Czech Republic, a member form of selecting model (5) to the data set in Table 1 with
state of EU, which, however, has not joined the Eurozone p = 1, q = 2. The optimal solution implies that Number of
yet. The Czech Republic has relatively stable economy with Branches, Non-interest income and Loans are selective
close tights to Germany and its banking sector is highly measures. Last column of Table 3 demonstrates the
internationalized (majority of banks are owned by acceptable efciencies which are obtained by applying
 M. Toloo, T. Tichy / Measurement 65 (2015) 2940 35

Table 1
Bank data and efciency score.

Banks Inputs Outputs BCC-

efciency
Employees No. of Assets Equity Expenses Deposits Loans Non-interest Interest
branches income income
AIR 400 18 33 600 2596 745 30 696 11 135 14 554 1
CMZRB 217 5 111 706 4958 566 86 967 16 813 634 1700 1
CS 10 760 658 920 403 93 190 18 259 688 624 489 103 15 412 37 717 1
CSOB 7801 322 937 174 73 930 16 087 629 622 479 516 8,747 32 697 1
EQB 296 13 8985 1296 601 7502 5611 19 215 1
ERB 72 1 33 614 464 173 2940 1762 15 131 1
FIO 59 36 18 561 726 347 17 174 6465 211 536 1
GEMB 3346 260 135 474 34 486 5276 97 063 101 898 3943 11 026 1
ING 293 10 128 425 913 1034 92 579 19 216 468 5139 1
JTB 407 3 85 087 7233 1333 62 085 39 330 487 3686 1
KB 8 758 399 786 836 100 577 13 511 579 067 451 547 8834 35 972 1
LBBW 365 18 31 300 2774 1138 20 274 2528 128 1046 0.8625
RB 2927 125 197 628 18 151 57 112 144 143 150 138 2829 8563 1
UCB 2004 98 318 909 38 937 13 804 195 120 192 046 2740 8891 1

Table 2
Selective measures obtained by multiplier form model and BCC-efciency score.

Banks Inputs Outputs Acceptable BCC-efciency

Employees Expenses Deposits Loans
AIR 400 745 30 696 11 135 0.5607
CMZRB 217 566 86 967 16 813 1
CS 10 760 18 259 688 624 489 103 1
CSOB 7801 16 087 629 622 479 516 1
EQB 296 601 7502 5611 0.4551
ERB 72 173 2940 1762 1
FIO 59 347 17 174 6465 1
GEMB 3346 5276 97 063 101 898 0.5875
ING 293 1034 92 579 19 216 1
JTB 407 1333 62 085 39 330 1
KB 8 758 13 511 579 067 451 547 1
LBBW 365 1138 20 274 2528 0.25
RB 2927 57 112 144 143 150 138 0.5346
UCB 2004 13 804 195 120 192 046 1

Table 3
Selective measures by envelopment form and BCC-efciency score.

Banks Inputs Outputs Acceptable BCC-efciency

Employees No. of branches Non-interest income Loans
AIR 400 18 14 11 135 0.3686
CMZRB 217 5 634 16 813 1
CS 10 760 658 15 412 489 103 1
CSOB 7801 322 8,747 479 516 1
EQB 296 13 19 5611 0.3419
ERB 72 1 15 1762 1
FIO 59 36 211 6465 1
GEMB 3346 260 3943 101 898 0.7704
ING 293 10 468 19 216 0.773
JTB 407 3 487 39 330 1
KB 8 758 399 8834 451 547 0.8591
LBBW 365 18 128 2528 0.256
RB 2927 125 2829 150 138 0.7506
UCB 2004 98 2740 192 046 1

the BCC model to the suggested data set. It can be forms of classifying models in the paper,
   
extracted that there are 7 efcient banks, and as a result, 4 4
336 14  optimization problems must be
the formulated selecting model successfully decreased 1 2
the percentage of efcient DMUs from 93% to 50%. solved. In other words, the formulated selecting models
It is worth to point out that to obtain the same results in this paper choice the selective measures with fairly
without utilizing the proposed multiplier or envelopment fewer computations.
36 M. Toloo, T. Tichy / Measurement 65 (2015) 2940

6. Conclusion ebxp 6 v p 6 Mbxp we have v p 0, which means the selective

x
measure p is not selected. If bp 1, then the selective mea-
The efciency discrimination among DMUs is question-
sure p is selected since have v p > 0. In a similar manner,
able when the rule of thumb in the envelopment DEA mod- y
els is violated. Sometimes, the rule does not come across and we obtain uq 0 when bq 0 which means the selective
one way to tackle this issue is selecting some of inputs/out- output q is not selected. We also have uq > 0 when
puts in a manner that the efciency of each DMU or the y
bq 1. As a result, total number of considered inputs
aggregate efciency of all DMUs does not lessen. and output, including xed and selective measures, in
In this paper, we dealt with selecting inputs/outputs P x
selecting model (3) is equal to jm1 j i2m2 bi
and modied both multiplier and envelopment forms of P P P
y y x
BCC model to obtain two alternative selecting models that js1 j r2s2 br . Consider the constraint r2s2 br i2m2 bi
p
could select the performance measures so that the number 6 minfn=3; 2 ng  jm1 j js1 j. If n=3 minfn=3;
of DMUs, inputs and outputs meet the rule of thumb. Indi- p P x
2 ng, then we have n P 3jm1 j i2m2 bi
vidual and aggregate versions of selecting models were P y p p
js1 j r2s2 br ; otherwise, i.e. 2 n minfn=3; 2 ng,
developed for both forms of formulated selecting models. P y P x p
In addition, if there are some xed performance measures, the constraint r2s2 br i2m2 bi 6 2 n  jm1 j js1 j
p P x P y
then selective measures can be chosen through the intro- leads to 2 n P jm1 j i2m2 bi js1 j r2s2 br and
duced selecting models. The multiplier form of selecting consequently an easy calculation shows that
model gives us the maximum efciency scores whereas P x P y
n P jm1 j i2m2 bi  js1 j r2s2 br . As a result, from
the maximum discrimination between efcient DMUs is
the optimal solution of the selecting model (3) we obtain
achieved by applying the envelopment form. The sug- n  P P 
x y
gested approaches have a major contribution to many of n P max 3 jm1 j i2m2 bi js1 j r2s2 br ; jm1 j
P  P 
the evaluating efciency topics. Potential use of the pro- x y
i2m2 bi js1 j r2s2 br g which completes the proof. h
posed approach was illustrated with application to bank-
ing industry. 
Past research has examined the effect of holding the Proof of Theorem 2. Let i e m2 and r e s2. If di 1, then
Pn x x
rule of thumb in DEA on having a reliable and acceptable the constraint j1 kj xij si hxio M1  di changes to
Pn x
efciency scores (see Cooper et al. [14]). Nevertheless, it j1 kj xij si hxio which means input measure i is
is less clear how to select input and output measures. To Pn x
selected. Otherwise, the constraint j1 kj xij si hxio
summarize our contribution, we formulated some DEA x P n x
M1  di converts to j1 kj xij si hxio M which, for M
models which choose some measures in different condi- P
tions to keep the rule of thumb in DEA. sufciently large, implies that nj1 kj xij sxi hxio is redun-
Future research could use the proposed method of this dant and subsequently input measure i is not selected. Note
x
paper, with mirror modications, in various aspects. Deter- that in the objective function, the term di sxi considers slack
mining selective measures in other traditional DEA models, variables of selected inputs. Considering the number of
P x
such as additive, slacks-based measure, hybrids and non- xed-input measures, |m1|, and the constraint i2m2 di p
radial models, can be considered as some further research the total number of input measures is |m1| + p. Similarly, if
topics. Furthermore, the proposed formulated models in y Pn y
dr 1, then the constraint j1 kj yrj  sr yro is active;
this study can be extended to handle selective measures in P y
otherwise it is redundant. The constraint r2s2 dr q
the presence of imprecise, negative data or fuzzy data sets.
implies that the total number of xed-output and selec-
tive-output measures is s1 + q. As a result, the number of
Acknowledgements measures in model (5) is m1 s 1 p q 6
p
minfn=3; 2 ng. This completes the proof. h
The project is a result of European Social Fund project
CZ.1.07/2.3.00/20.0296. The research of the rst author y x
Proof of Theorem 3. Let u ; u0 ; v  ; b ; b and
was supported by the Czech Science Foundation (GACR   x y x y
h ; k ; sx ; sy ; d ; d ; t ; t be the optimal solution of
Project 14-31593S). The second authors research was sup-
models (3) and (6), respectively. Consider model (5) and
ported through the Czech Science Foundation (GACR)  

under project 13-13142S and furthermore by the European let m  fijdxi 1g and s frjdyr 1g. It is easy to verify
Regional Development Fund in the IT4Innovations Centre that the optimal solution of the following envelopment
 
of Excellence Project (CZ.1.05/1.1.00/02.0070) and, an SGS form of BCC is h ; k ; sx ; sy :
Research Project of VSB-TU Ostrava. !
X X
max z10 h  e sxi syr

i2m1 [m r2s1 [s
Appendix A
s:t:
A.1. Proof of theorems Xn
kj xij sxi hxio i 2 m1 [ m

j1
y x
Proof of Theorem 1. Let u ; u0 ; v  ; b ; b be the opti- X
n
mal solution of selecting model (3). Consider a selective kj yrj  syr yro r 2 s1 [ s
x j1
input, say p. If bp 0, then from the constraint
Table C.1
Papers containing practical applications.

No. Title Author(s) Number of

Year DMUs Inputs Outputs
1 Diversication-consistent data envelopment analysis based on directional-distance measures M. Branda 2015 48 10 1
2 A Bounded Data Envelopment Analysis Model in a Fuzzy Environment with an Application to Safety in A. Hatami-Marbini, M. Tavana, K. Gholami, Z. Ghelej Beigi 2015 8 2 2
the Semiconductor Industry
3 Finding an initial basic feasible solution for DEA models with an application on bank industry M. Toloo, A. Masoumzadeh, M. Barat 2015 50 3 3
4 Interval data without sign restrictions in DEA A. Hatami-Marbini, A. Emrouznejad, P. J. Agrell 2014 20 3 2
5 A network DEA assessment of team efciency in the NBA P. Moreno, S. Lozano 2014 30 2 1
6 Incorporating health outcomes in Pennsylvania hospital efciency: an additive super-efciency DEA J. Du, J. Wang, Y. Chen, S.Y. Chou, J. Zhu 2014 119 4 3
approach
7 On relations between DEA-risk models and stochastic dominance efciency tests M. Branda, M. Kopa 2014 48 4 1
8 An Epsilon-free Approach for Finding the Most Efcient Unit in DEA M. Toloo 2014 12 2 2
9 The most cost efcient automotive vendor with price uncertainty: A new DEA approach M. Toloo, T. Ertay 2014 73 7 5
10 Finding the best asset nancing alternative: A DEA-WEO approach M. Toloo, A. Kraska 2014 139 4 1
11 Diversication-consistent data envelopment analysis with general deviation measures M. Branda 2013 25 4 1
12 Chance-constrained DEA models with random fuzzy inputs and outputs M. Tavana, R. Khanjani Shiraz, A. Hatami-Marbini, P.J. 2013 30 2 2

M. Toloo, T. Tichy / Measurement 65 (2015) 2940

Agrell, K. Paryab
13 The most efcient unit without explicit inputs: An extended MILP-DEA model M. Toloo 2013 40 1 6
14 Assessment of the site effect vulnerability within urban regions by data envelopment analysis: A case A. Farzipour Saein, R. Farzipoor Saen 2013 20 3 1
study in Iran
15 Positive and normative use of fuzzy DEA-BCC models: A critical view on NATO enlargement A. Hatami-Marbini, M. Tavana, S. Saati, P.J. Agrell 2012 18 6 5
16 A common set of weight approach using an ideal decision making unit in data envelopment analysis S. Saati, A. Hatami-Marbini, P.J. Agrell, M. Tavana 2012 286 2 4
17 Alternative solutions for classifying inputs and outputs in data envelopment analysis M. Toloo 2012 50 2 3
18 Super-efciency infeasibility and zero data in DEA H.S. Lee, J. Zhu 2012 15 8 1
19 The examination of pseudo-allocative and pseudo-overall efciencies in DEA using shadow prices J.C. Paradi, F.K. Tam 2012 100 5 6
20 Reducing differences between proles of weights: A peer-restricted cross-efciency evaluation N. Ramon, J.L. Ruiz, I. Sirvent 2011 14 3 2
21 A cross-dependence based ranking system for efcient and inefcient units in DEA J.X. Chen, M. Deng 2011 20 2 2
22 Performance and congestion analysis of the Portuguese hospital services P. Simoes, R.C. Marques 2011 68 3 3
23 Employing post-DEA Cross-evaluation and Cluster Analysis in a sample of Greek NHS Hospitals A. Flokou, N. Kontodimopoulos, D. Niakas 2011 27 3 4
24 A neutral DEA model for cross-efciency evaluation and its extension Y.M. Wang, K.S. Chin 2010 14 3 2
25 Some alternative models for DEA cross-efciency evaluation Y.M. Wang, K.S. Chin 2010 7 3 3
26 Malmquist productivity index based on common-weights DEA: The case of Taiwan forests after C. Kao 2010 17 4 3
reorganization
27 A semi-oriented radial measure for measuring the efciency of decision making units with negative A. Emrouznejad, A.L. Anouze, E. Thanassoulis 2010 13 2 3
data, using DEA
28 A distance friction minimization approach in data envelopment analysis: A comparative study on S. Suzuki, P. Nijkamp, P. Rietveld, E. Pels 2010 30 4 2
airport efciency
29 Efciency evaluations with context-dependent and measure-specic data envelopment: An A. Ulucan, K. Bar 2010 81 2 6
application in a World Bank supported project
30 Measuring the performance of nancial holding companies C.M. Chao, M.M. Yu, M.C. Chen 2010 14 3 3
31 DEA model with shared resources and efciency decomposition Y. Chen, J. Dub, H.D. Sherman, J. Zhu 2010 27 3 3
32 DEA-based production planning J. Dua, L. Liang, Y. Chen, G. Bi 2009 20 2 5
33 DEA game cross-efciency approach to Olympic rankings J. Wu, L. Liang, Y. Chen 2009 62 2 3
34 A new method for ranking discovered rules from data mining by DEA M. Toloo, B. Sohrabi, S. Nalchigar 2009 46 1 4
35 A new integrated DEA model for nding most BCC-efcient DMU M. Toloo, S. Nalchigar 2009 19 2 4
36 Slacks-based efciency measures for predicting bank performance S.T. Liu 2009 24 3 3
37 Efciency measurement and ranking of the tutorial system using IDEA H.T. Lin 2009 20 1 2
38 The link between operational efciency and environmental impacts A joint application of Life Cycle S. Lozano, D. Iribarren, M.T. Moreira, G. Feijoo 2009 62 14 1
Assessment and Data Envelopment Analysis

(continued on next page)

37
38 M. Toloo, T. Tichy / Measurement 65 (2015) 2940

X
n
kj 1
Outputs j1
4 sxi P 0 i 2 m1 [ m 
2

1
3
4
5
6

1
4
y
sr P 0 r 2 s1 [ s
Inputs
4
7

4
2
2
3
2

3
4
kj P 0 j 1; . . . ; n 10
Number of
DMUs

Since the optimal objective value of BCC models are

163
85
23

18
19
33
50

37

20
positive and less than or equal to 1 we have 0 < z10 6 1.
Consider the following dual model of (10):
2009
2009

2009
2008
2007
2006
2005

2004
2004
Year

X
max z11 ur yro uo
r2s1 [s
A. Renner, J.M. Kirigia, E.A. Zere, S.P. Barry, D.G. Kirigia, C.

s:t:
X
v i xio 1

i2m1 [m 11
X X
ur yrj u0  v i xij 6 0 j 1; . . . ; n
r2s1 [s 
i2m1 [m

vi P e i 2 m1 [ m 
L. Liang, J. Wu, W.D. Cook, J. Zhu

ur P e r 2 s1 [ s
F.R. Forsund, L. Hjalmarsson

Let u  0 ; v
; u   be the optimal solution of model (11).
Kamara, L. H.K. Muthur

Under the optimality conditions (primaldual relation-

T. Sueyoshi, M. Goto

G.R. Amin, M. Toloo

ships) z10 z11 . Let

W.D. Cook, J. Zhu
P.V. Geymueller

M.M. Mostafa

x

1; i 2 m y 1; r 2 s
b ;b
Author(s)

i r

0; i 2 m2 =m 0; r 2 s2 =s
Y. Chen

A simple computation gives that u  0 ; v

; u  y ; b
 ; b  x is a

feasible solution of model (3) with objective value z9 . Hence,
Technical efciency of peripheral health units in Pujehun district of Sierra Leone: a DEA application

we have z10 z11 6 z3 and what is left is to show that z3 6 1.
Static versus dynamic DEA in electricity regulation: the case of US transmission system operators

To do this, consider the given optimal solution of model (3),

y x x y
u ; u0 ; v  ; b ; b , and x b and b . If we redene
Methodological comparison between DEA and DEADA from the perspective of bankruptcy

 
 fijbxi 1g and s frjbyr 1g, then u ; u0 ; v  is an
m
optimal solution of the multiplier form of BCC model (11)
and hence 0 < z4 6 1 which completes the proof. h
Modeling the efciency of top Arab banks: A DEAneural network approach

Appendix B
Finding the most efcient DMUs in DEA: An improved integrated model

Banks in the Czech Republic

Alternative secondary goals in DEA cross-efciency evaluation

Are all Scales Optimal in DEA? Theory and Empirical Evidence

Bank Full name Description

AIR Air Bank Small and very new
market participant
owned by Czech
Rank order data in DEA: A general framework

conglomerate, which is,

however, registered in
the Netherlands
CMZRB Ceskomoravsk A specialized banking
zrucn a institution managed by
Ranking efcient units in DEA

rozvojov banka the government

representatives in order
to support selected
market segments
CS Cesk sporitelna Large, traditional bank,
Table C.1 (continued)

assessment

owned by Erste Group,

Austria
Title

CSOB Ceskoslovensk Large, traditional bank,

obchodn banka owned by KBC Bank,
No.

39

41
42
43
44
45

46
47
40
 M. Toloo, T. Tichy / Measurement 65 (2015) 2940 39

Appendix B (continued) Appendix B (continued)

Bank Full name Description Bank Full name Description
Belgium RB Raiffeisen Bank Mid-size, owned by
EQB Equa Bank Though existing for Austrian bank of the
more than 20 years, same name
former IC Banka, UCB UniCredit Bank Fourth largest bank after
currently owned by Czech Republic several mergers (HVB
Equa Group, Malta Bank, Bank Austria
ERB Evropsko-rusk Small and relatively Creditanstalt and
banka new, owned by a natural traditional Czech bank
person Zivnostensk banka),
FB Fio Banka Small, though exiting currently owned by
more than 20 years, Austrian bank of the
owned by Fio holding, respective name, which
Czech Republic is owned by Italian
GEMB GE Money Bank Mid-size, owned by GE UniCredit
Holding, USA
ING ING Bank N.V. Midsize branch of
foreign bank
JTB J&T Bank Small to mid-size, Appendix C
owned indirectly by two
natural persons Practical applications in DEA.
KB Komercn banka Large, traditional bank See Table C.1.
owned by Societe
General, France Appendix D
LBBW LBBW Bank CZ Small, owned by
Landesbank Baden- Inputoutput measures in bank studies
Wurttemberg, Germany See Table D.1.

Table D.1
Inputs and outputs in bank studies adapted from Mostafa [32].

Inputs Frequency % Outputs Frequency %

Employees 22 84.62 Loans 12 46.15
Expenses 11 42.31 Number of transactions 8 30.77
Space 5 19.23 Deposits 7 26.92
Terminals 5 19.23 Non- interest income 7 26.92
Capital 5 19.23 Interest income 5 19.23
Deposits 4 15.38 customer response 3 11.54
Assets 4 15.38 Revenues 3 11.54
Number of branches 4 15.38 Prot 3 11.54
Rent 3 11.54 Investment in securities 3 11.54
Number of accounts 3 11.54 Advances 2 7.69
Credit application 2 7.69 Current accounts 2 7.69
ATMs 2 7.69 Error corrections 1 3.85
Location 2 7.69 Liability sales 1 3.85
Costs 2 7.69 Maintenance 1 3.85
Marketing 2 7.69 Marketed balances 1 3.85
Selected nancial ratios 1 3.85 Selected nancial ratios 1 3.85
Supplier 1 3.85 Insurance commission 1 3.85
Acquired equipment 1 3.85 Satisfaction 1 3.85
Funds from customers 1 3.85 ROA 1 3.85
Loanable funds 1 3.85 ROE 1 3.85
Counter transactions 1 3.85
Net worth 1 3.85
Borrowings 1 3.85
Loans 1 3.85
Size 1 3.85
Sale FTE 1 3.85
City employment rate 1 3.85
40 M. Toloo, T. Tichy / Measurement 65 (2015) 2940

References [28] H.T. Lin, Efciency measurement and ranking of the tutorial system
using IDEA, Expert Syst. Appl. 36 (2009) 1123311239.
[29] S.T. Liu, Slacks-based efciency measures for predicting bank
[1] G.R. Amin, M. Toloo, Finding the most efcient DMUs in DEA: an
performance, Expert Syst. Appl. 36 (2009) 28132818.
improved integrated model, Comput. Ind. Eng. 52 (2007) 7177.
[30] S. Lozano, D. Iribarren, M.T. Moreira, G. Feijoo, The link between
[2] G.R. Amin, M. Toloo, A polynomial-time algorithm for nding Epsilon
operational efciency and environmental impacts: a joint
in DEA models, Comput. Oper. Res. 31 (5) (2004) 803805.
application of life cycle assessment and data envelopment
[3] R.D. Banker, A. Charnes, W.W. Cooper, Models for estimation of
analysis, Sci. Total Environ. 407 (2009) 17441754.
technical and scale inefciencies in data envelopment analysis,
[31] P. Moreno, S. Lozano, A network DEA assessment of team efciency
Manage. Sci. 30 (1984) 10781092.
in the NBA, Ann. Oper. Res. 214 (2014) 99124.
[4] M. Branda, Diversication-consistent data envelopment analysis
[32] M.M. Mostafa, Modeling the efciency of top Arab banks: a DEA-
based on directional-distance measures, Omega 52 (2015) 6576.
neural network approach, Expert Syst. Appl. 36 (2009) 309320.
[5] M. Branda, Diversication-consistent data envelopment analysis
[33] J.C. Paradi, F.K. Tam, The examination of Pseudo-allocative and
with general deviation measures, Eur. J. Oper. Res. 226 (2013) 626
pseudo-overall efciencies in DEA using shadow prices, J. Product.
635.
Anal. 37 (2012) 115123.
[6] M. Branda, M. Kopa, On relations between DEA-risk models and
[34] N. Ramon, J. Ruiz, I. Sirvent, Reducing differences between proles of
stochastic dominance efciency tests, Central Eur. J. Oper. Res. 22
weights: a peer-restricted cross-efciency evaluation, Omega 39
(2014) 1315.
(2011) 634641.
[7] C.M. Chao, M.M. Yu, M.C. Chen, Measuring the performance of
[35] A. Renner, J.M. Kirigia, E.A. Zere, S.P. Barry, D.G. Kirigia, C. Kamara,
nancial holding companies, Serv. Ind. J. 30 (2010) 811829.
L.H.K. Muthuri, Technical efciency of peripheral health units in
[8] A. Charnes, W.W. Cooper, E. Rhodes, Measuring the efciency of
Pujehun district of Sierra Leone: a DEA application, BMC Health Serv.
decision making units, Eur. J. Oper. Res. 2 (1978) 429444.
Res. 5 (2005) 77.
[9] A. Charnes, W.W. Cooper, E. Rhodes, Short communication:
[36] S. Saati, A. Hatami-Marbini, P.J. Agrell, M. Tavana, A common set of
measuring the efciency of decision making units, Eur. J. Oper. Res.
weight approach using an ideal decision making unit in data
3 (1979) 339.
envelopment analysis, J. Ind. Manage. Optim. 8 (3) (2012) 623637.
[10] J.X. Chen, M. Deng, A cross-dependence based ranking system for
[37] P. Simoes, R.C. Marques, Performance and congestion analysis of the
efcient and inefcient units in DEA, Expert Syst. Appl. 38 (2011)
Portuguese hospital services, Central Eur. J. Oper. Res. 19 (2011) 39
96489655.
63.
[11] Y. Chen, Ranking efcient units in DEA, Omega 32 (2004) 213219.
[38] T. Sueyoshi, M. Goto, Methodological comparison between DEA
[12] Y. Chen, J. Dub, H.D. Sherman, J. Zhu, DEA model with shared
(data envelopment analysis) and DEA-DA (discriminant analysis)
resources and efciency decomposition, Eur. J. Oper. Res. 207 (2010)
from the perspective of bankruptcy assessment, Eur. J. Oper. Res. 199
339349.
(2009) 561575.
[13] W.D. Cook, J. Zhu, Rank order data in DEA: a general framework, Eur.
[39] S. Suzuki, P. Nijkamp, P. Rietveld, E. Pels, A distance friction
J. Oper. Res. 174 (2006) 10211038.
minimization approach in data envelopment analysis: a
[14] W.W. Cooper, L.M. Seiford, K. Tone, Data Envelopment Analysis: A
comparative study on airport efciency, Eur. J. Oper. Res. 207
Comprehensive Text with Models, Applications, References and
(2010) 11071115.
DEA-Solver Software, 2nd ed., Springer, Berlin, 2007.
[40] M. Tavana, R. Khanjani Shiraz, A. Hatami-Marbini, P.J. Agrell, K.
[15] A. Farzipour Saein, R. Farzipoor Saen, Assessment of the site effect
Paryab, Chance constrained DEA models with random fuzzy inputs
vulnerability within urban regions by data envelopment analysis: a
and outputs, Knowl.-Based Syst. 52 (2013) 3252.
case study in Iran, Comput. Geosci. 48 (2012) 280288.
[41] M. Toloo, Alternative solutions for classifying inputs and outputs in
[16] J. Du, J. Wang, Y. Chen, S.Y. Chou, J. Zhu, Incorporating health
data envelopment analysis, Comput. Math. Appl. 63 (2012) 1104
outcomes in Pennsylvania hospital efciency: an additive super-
1110.
efciency DEA approach, Ann. Oper. Res. 221 (2014) 161172.
[42] M. Toloo, An epsilon-free approach for nding the most efcient unit
[17] J. Dua, L. Liang, Y. Chen, G. Bi, DEA-based production planning,
in DEA, Appl. Math. Model. 38 (13) (2014) 31823192.
Omega 38 (2010) 105112.
[43] M. Toloo, The most efcient unit without explicit inputs: an
[18] A. Emrouznejad, A.L. Anouze, E. Thanassoulis, A semi-oriented radial
extended MILP-DEA model, Measurement 46 (2013) 36283634.
measure for measuring the efciency of decision making units with
[44] M. Toloo, M. Barat, A. Masoumzadeh, Selective measures in data
negative data, using DEA, Eur. J. Oper. Res. 200 (2010) 297304.
envelopment analysis, Ann. Oper. Res. (2015), http://dxdoi.org/
[19] A. Flokou, N. Kontodimopoulos, D. Niakas, Employing post-DEA
10.1007/s10479-014-1714-3.
cross-evaluation and cluster analysis in a sample of greek NHS
[45] M. Toloo, T. Ertay, The most cost efcient automotive vendor with
hospital, J. Med. Syst. 35 (2011) 10011014.
price uncertainty: a new DEA approach, Measurement 52 (2014)
[20] F.R. Forsund, L. Hjalamarsson, Are all scales optimal in DEA? Theory
135144.
and empirical evidence, J. Product. Anal. 21 (2004) 2548.
[46] M. Toloo, A. Kraska, Finding the best asset nancing alternative: a
[21] P.V. Geymueller, Static versus dynamic DEA in electricity regulation:
DEA-WEO approach, Measurement 55 (2014) 288294.
the case of US transmission system operators, Central Eur. J. Oper.
[47] M. Toloo, A. Masoumzadeh, M. Barat, Finding an initial basic feasible
Res. 17 (2009) 397413.
solution for DEA models with an application on bank industry,
[22] A. Hatami-Marbini, A. Emrouznejad, P.J. Agrell, Interval data without
Comput. Econ. (2015), http://dxdoi.org/10.1007/s10614-014-9423-
sign restrictions in DEA, Appl. Math. Model. 38 (2014) 20282036.
1.
[23] A. Hatami-Marbini, M. Tavana, K. Gholami, Z. Ghelej Beigi, A
[48] M. Toloo, S. Nalchigar, A new integrated DEA model for nding most
bounded data envelopment analysis model in a fuzzy environment
BCC-efcient DMU, Appl. Math. Model. 33 (2009) 597604.
with an application to safety in the semiconductor industry, J.
[49] M. Toloo, B. Sohrabi, S. Nalchigar, A new method for ranking
Optim. Theory Appl. 164 (2015) 679701.
discovered rules from data mining by DEA, Expert Syst. Appl. 36
[24] A. Hatami-Marbini, M. Tavana, S. Saati, P.J. Agrell, Positive and
(2009) 85038508.
normative use of fuzzy DEABCC models: a critical view on NATO
[50] A. Ulucan, K. Bar, Efciency evaluations with context-dependent and
enlargement, Int. Trans. Oper. Res. 20 (3) (2012) 411433.
measure-specic data envelopment: an application in a World Bank
[25] A. Kao, Malmquist productivity index based on common-weights
supported project, Omega 38 (2010) 6883.
DEA: the case of Taiwan forests after reorganization, Omega 38
[51] Y.M. Wang, K.S. Chin, Some alternative models for DEA cross-
(2010) 484491.
efciency evaluation, Int. J. Product. Econ. 128 (2010) 332338.
[26] H.S. Lee, J. Zhu, Super-efciency infeasibility and zero data in DEA,
[52] Y.M. Wang, K.S. Chin, A neutral DEA model for cross-efciency
Eur. J. Oper. Res. 216 (2012) 429433.
evaluation and its extension, Exp. Syst. Appl. 37 (2010) 36663675.
[27] L. Liang, J. Wu, W.D. Cook, J. Zhu, Alternative secondary goals in DEA
[53] J. Wu, L. Liang, Y. Chen, DEA game cross-efciency approach to
cross-efciency evaluation, Int. J. Product. Econ. 113 (2008) 1025
Olympic rankings, Omega 37 (2009) 909918.
1030.