Decision Support Systems 11 (1994)545-557
North-Holland
Bankruptcy prediction using
neural networks
R i c k L. W i l s o n a n d R a m e s h Sharda
Oklahoma State University, Stillwater, OK, USA
Prediction of firm bankruptcies have been extensively
studied in accounting, as all stakeholders in a firm have a
vested interest in monitoring its financial performance. This
paper presents an exploratory study which compares the pre-
dictive capabilities for firm bankruptcy of neural networks and
classical multivariate discriminant analysis. The predictive ac-
curacy of the two techniques is presented within a compre-
hensive, statistically sound framework, indicating the value
added to the forecasting problem by each technique. The
study indicates that neural networks perform significantly
better than discriminant analysis at predicting firm bankrupt-
cies. Implications of our results for the accounting profes-
sional, neural networks researcher and decision support sys-
tem builders are highlighted.
Keywords: Neural network applications; Bankruptcy predic-
tion; Discriminant analysis; Classification tech-
niques
Rick L. Wilson is currently an Assis-
tant Professor of Management Sci-
ence and Information Systems at Ok-
lahoma State University. He received
his Ph.D. in MIS from the University
of Nebraska-Lincoln. Dr. Wilson has
published in journals such as Infor-
mation and Management, Interna-
tional Journal of Production Research,
International Journal of Production
and Operations Management, among
others. He is a member of Decision
Sciences Institute, The Institute of
Management Science, and Operations Research Society of
America. His current research interests include neural net-
works, decision support systems and integrated management
science applications.
Correspondence to: Ramesh Sharda, Department of Manage-
ment, College of Business Administration, Oklahoma State
University, Stillwater, OK 74078, (405) 744-8638. Email:
MGMTRSH@OSUCC.bitnett.
0167-9236/94/$07.00 © 1994 - Elsevier Science B.V. All rights reserved
SSDI 01 6 7 - 9 2 3 6 ( 9 2 ) 0 0 0 8 5 - F
545
I. Introduction
The ability to predict firm bankruptcies has
been extensively studied in the accounting litera-
ture. Creditors, auditors, stockholders and senior
management all have a vested interest in utilizing
and developing a methodology or model that will
allow them to monitor the financial performance
of a firm via accounting ratios. This "failure
analysis" can be helpful in identifying internal
problems, firm evaluation by investors, and as a
tool used by auditors to assist them in their job.
Typically, a number of financial ratios are used
in a multivariate discriminant analysis approach
in an attempt to predict firm bankruptcies. Dis-
criminant analysis is a statistical technique used
to construct classification schemes so as to assign
previous unclassified observations to the appro-
priate group [15]. However, it may be a valid
technique only under certain restrictive assump-
tions, including the requirement for the discrimi-
nating variables to be jointly distributed accord-
ing to a multivariate normal distribution. Should
this not be the case, results obtained by the
discriminant analysis procedure may be erro-
neous.
Neural Networks represents a field of study
within the Artificial Intelligence area where re-
Ramesh Sharda is the Conoco/Du-
Pont Professor of Management of
Technology at Oklahoma State Uni-
versity. Professor Sharda received his
Ph.D. from the University of Wiscon-
sin-Madison. His publications have
appeared in major academic journals
such as Management Science, Inter-
faces, Computers and Operations Re-
search, Journal of Intelligent Manufac-
turing and Socio-Economic Planning
Sciences. In addition, he has coedited
two books: Impacts of Recent Com-
puter Advances on Operations Research, and Knowledge-
Based Systems and Neural Networks: Techniques and Applica-
tions. He is a member of The Institute of Management Sci-
ence, Operations Research Society of America and the Deci-
sion Sciences Institute. His current research interests include
decision support systems, forecasting, optimization on micro-
computers, neural networks and expert systems.
546 R.L. Wilson and R. Sharda / Bankruptcy prediction using neural networks
searchers are studying a "biologically inspired"
way of processing information. To this point, neu-
ral networks have proven to be good at solving
some real-world problems, especially in the areas
of forecasting and classification decision prob-
lems.
The exploratory study presented in this paper
contrasts neural network predictive accuracy with
that of discriminant analysis for the decision
problem of firm bankruptcy prediction. Using a
resampling methodological design, a series of ex-
periments was conducted to investigate the effect
of the training and testing (holdout) set composi-
tion on predictive accuracy. These predictive re-
sults were then contrasted with the accuracy ob-
tained by classical discriminant analysis to deter-
mine the conditions where neural network mod-
els are significantly better predictors.
The major objectives of this paper are the
following. First, we report the results of a com-
prehensive, statistically sound comparison of dis-
criminant analysis and a neural network model.
Second, we utilize in this analysis measures of the
validity of any classification technique which have
been used extensively in the psychology litera-
ture. These measures allow researchers to assess
the true value added by a technique. Third, we
conclude with a brief conjecture on how neural
networks may affect decision support systems.
Our objective is not to examine speed of a new
algorithm, or study a new architecture, but rather
to test a neural network's effectiveness in per-
forming classifications as contrasted against the
incumbent techniques. Better algorithms should
only improve the performance. In this sense, our
results should provide a lower bound of the neu-
ral network model's predictive performance in
bankruptcy prediction.
Section 2 briefly reviews bankruptcy prediction
and the neural network literature. Section 3 de-
scribes our comparison procedure. Section 4 pre-
sents the results, while section 5 discusses impli-
cations of our results for researchers from three
areas: bankruptcy prediction, neural networks,
and decision support systems.
2. Brief review of relevant research
2.1. Bankruptcy risk prediction
In the present days of economic turmoil, it is
not surprising that the bankruptcy prediction
problem remains of great interest to researchers
as well as creditors, shareholders and auditors.
Firm insolvency is a problem throughout the in-
dustrialized countries of the world [4]. Creditors
have a vested interest in this decision problem in
that they wish to identify negative developments
of their borrowers. Stockholders hold similar
monetary concerns. Auditors, as a normal re-
sponsibility, must evaluate the financial position
of a client to determine whether or not the firm's
operating ability is endangered [3]. Thus, senior
management of a firm and the board of directors
can attempt to avert the crisis [34]. For all parties,
it is essential that an objective opinion on the risk
of bankruptcy can be formed as early as possible.
Predicting bankruptcy has been studied exten-
sively in the accounting literature. The first stud-
ies were performed to determine whether finan-
cial ratios provide useful information [1,5]. There
have been many different studies since [5] utiliz-
ing financial ratios for bankruptcy prediction, a
majority of which use a multivariate discriminant
analysis approach [1,2,4,7,12,25,32]. The major
evolution in these studies is to identify financial
and economic variables which improve predictive
performance. Two statistical techniques appear
to have been used the most: discriminant analysis
and logistic regression [6]. No technique clearly
provides substantially better results. We have
chosen to study discriminant analysis as a com-
parative classification technique because of its
repeated use in many other problem areas.
Discriminant analysis is a statistical technique
used to classify objects into distinct groups on the
basis of an object' s observed characteristics. Ba-
sically, a linear discriminant function is developed
which will compute a "score" for an object. This
function is a weighted linear combination of the
object's observed values on discriminating charac-
teristics. These weights represent, in essence, the
relative importance and impact of the various
characteristics. On the basis of its discriminant
score, an object is then classified. Often, com-
puter software packages compute the probability
of group membership on the basis of this proce-
dure [39].
Multivariate discriminant analysis is subject to
a number of restrictive assumptions, including the
requirement for the discriminating variables to be
jointly multivariate normal. This multivariate nor-
mality of the variables is critical to the discrimi-
nant analysis procedure; otherwise, results ob-
 R.L. Wilson and R. Sharda / Bankruptcy prediction using neural networks
tained may be erroneous [25]. This theoretical
assumption often cannot be realized in practice
[4].
2.2. Neural network applications
Multi-layer, feed forward neural networks have
been applied to many problem domains in and
outside the business field. For instance, neural
networks have been successfully trained to deter-
mine whether loan applications should be ap-
proved [20]. Similarly, neural networks have been
shown to predict mortgagee applicant solvency
better than mortgage writers [11].
Predicting rating of corporate bonds and at-
tempting to predict their profitability is another
area where neural networks have been applied
successfully [14,21]. Neural networks outper-
formed regression analysis and other mathemati-
cal modeling tools in predicting bond rating and
profitability. The main conclusion reached was
that neural networks provided a more general
framework for connecting financial information
of a firm to the respective bond rating.
Fraud prevention is another area of neural
network applications in business. Credit card
fraud, a costly and difficult problem faced by
banks, was addressed by Chase Manhattan Bank
of New York by neural networks [33]. These
models were shown to be much more successful
than traditional regression analysis. Additionally,
neural networks have been used in the validation
of bank signatures [19,30]. These networks identi-
fied forgeries significantly better than any human
'expert'.
Several people have tested the applicability of
neural networks in financial markets. Collard [10]
states that this neural network model for com-
modity trading would have resulted in significant
profits over other trading strategies. Kamijo and
Tanigawa [24] used a neural network to chart
Tokyo Stock Exchange data. Their finding were
that the results of the model would beat a 'buy
and hold' strategy. Additionally, a neural model
for predicting percentage change in the S & P 500
five days ahead using a variety of economic indi-
cators has been developed [18]. The authors claim
that the model has provided more accurate pre-
diction than alleged experts in the field using the
same indicators.
547
There have been many other applications of
neural networks in non-business related fields
such as speech recognition, robotics, radar detec-
tion and many others. Additionally, other net-
work paradigms have been useful in solving other
types of decision problems. Discussion of these
other applications and approaches is beyond the
scope of this paper.
Most of the studies which have compared neu-
ral networks with statistical techniques report the
results on the basis of either a single experiment
or in an anecdotal form. There is a need for a
thorough comparison using sound statistical pro-
cedures. Our study is based on a resampling
technique to assess the effectiveness of neural
networks on a statistical basis. Further, we bor-
row some measures from the psychology litera-
ture to isolate the value added by a classification
technique. We argue that such measures ought to
be used in determining alleged superiority of any
such model.
3. Method
3.1. Financial ratios and data collection
The basic intent of this study is to compare
and contrast the predictive performance of classi-
cal multivariate discriminant analysis to that of
neural networks for firm bankruptcy. The Altman
study [1] has been used as the standard of com-
parison for subsequent bankruptcy classification
studies using discriminant analysis. Most follow-
up studies have identified several other attributes
to improve prediction performance. In this ex-
ploratory study, we wanted to see if the neural
networks can come close to the traditional tech-
niques. More sophisticated inputs to the neural
network model should not worsen its perfor-
mance. Thus, this could establish a lower bound
on neural network performance in bankruptcy
prediction. For these reasons, we used the same
financial ratios as Altman [1]. These ratios were:
XI: Working C a p i t a l / T o t a l Assets
X2: Retained E a r n i n g s / T o t a l Assets
X3: Earnings before Interest and T a x e s / T o t a l
Assets
X4: Market Value of E q u i t y / T o t a l Debt
Xs: S a l e s / T o t a l Assets
548 R.L. Wilson and R. Sharda / Bankruptcy prediction using neural networks

The sample of firms for which these ratios Table 1

were obtained consisted of firms that either were Number of observations in training and testing sets
in operation or went bankrupt between 1975 and Training set Testing set composition
1982. The sample, obtained from Moody's Indus- composition 50%/50% 80%/20% 90%/10% *
trial Manuals, consisted of a total of 129 firms, 65 50%/50%
of which went bankrupt during the period and 64 TRAIN: 44/44 44/44 44/44
non bankrupt firms matched on industry and test: 20/20 20/5 20/2
year. Data used for the bankrupt firms is from 8o%/2o%
the last financial statements issued before the TRAIN: 44/11 44/11 44/11
firms declared bankruptcy. Thus, the prediction test: 20/20 20/5 20/2
of bankruptcy is to be made about one year in 90%/10% *
advance. TRAIN: 44/5 44/5 44/5
test: 20/20 20/5 20/2
3.2. Data set generation where for each cell:
TRAIN: (nonbankrupt/bankrupt) firms in training set
In assessing the predictive accuracy of discrim- TEST: (nonbankrupt/bankrupt) firms in testing set
inant analysis as compared to neural networks, it * Approximately 90%/10% ratio
is necessary to create two distinct sets of data; a
data set to develop the discriminant function
(similarly, to train the neural network, often re- model based on a certain base rate works across
ferred to as the training set) and a holdout sam- other proportions, it may be possible to build a
ple to validate the derived discriminant function model using a higher proportion of cases of inter-
(in neural network terminology, the testing set). est than actually occur in the population.
Because the decision of splitting the original 129 To study the effects of this proportion on the
firms could affect the results of the comparison, predictive performance of the two techniques, we
this study utilizes the concept of Monte Carlo created three proportions (or base rates) for each
resampling techniques to generate multiple sub- of the training and testing set compositions. The
samples from the original firms in order to gain a first factor level (or base rate) was a 5 0 / 5 0 pro-
better measure of predictive accuracy (see [37], portion of bankruptcy to non bankrupt cases, the
for example). second level was a 80/20 proportion (80% non-
The results of this study could be affected by bankrupt, 20% bankrupt), and the third factor
the proportion of non-bankrupt firms to bankrupt level, an approximate 9 0 / 1 0 proportion (ap-
firms in both the training and testing sets. That is, proximately 90% non-bankrupt; this proportion is
the population of all firms contains a certain not exact due to required round-off to integer
proportion of firms on the verge of bankruptcy. A number of firms). We do not really know the
proportion of interest in any population is some- actual proportion of firms going bankrupt. The
times referred to as the base rate. The base rate 8 0 / 2 0 and 9 0 / 1 0 cases should be close. The
may have an impact on a prediction technique's 5 0 / 5 0 scenario is utilized to investigate the possi-
performance in two ways. First, a technique may bility of a better model by using a high base rate
not work well when the firms of interest (bank- in the training set.
rupt) constitute a very small percentage of the Utilizing a full two-factor design, there were
population (low base rate). This would be due to nine different experimental cells, whose composi-
a technique's inability to identify the features tion is indicated in Table 1. Within each cell, 20
necessary for classification. different training-testing set pairs were generated
A second effect of the base rate is in terms of via Monte Carlo resampling from the original 129
differences in base rates between training sam- firms. Thus, a total of 180 distinct training and
ples and testing samples. If a classification model testing data set pairs were generated from the
is built using a training sample with a certain base original data. In each case, the training set and
rate, does the model still work when the base rate test set pairs contained unique firms; i.e., no
in the test population is different? This issue is overlap was allowed. This restriction provides a
important for one more reason. If a classification stronger test of a technique's performance.
 R.L. Wilson and R. Sharda / Bankruptcy prediction using neural networks
3.3. Implementation of comparative methods
SYSTAT [39], a personal computer-based sta-
tistical package, was used for discriminant analy-
sis. SYSTAT fits the standard multivariate gen-
eral linear model in performing discriminant
analysis and also uses information regarding the
prior probability specification of the training set
in determining the discriminant function. In this
study, the training sets were used to set up initial
discriminating functions, which were, in turn,
evaluated by the corresponding testing sets. All
variables were included in each discriminant
analysis conducted in the study. Tests were not
undertaken to determine whether the discrimi-
nating variables were distributed according to a
joint multivariate normal distribution. However,
previous research has indicated that neural net-
works can also perform well in cases of multivari-
ate normal distributions [13]; thus, the distribu-
tion of the data used in our study is not a relevant
issue in considering robust predictive accuracy.
B R A I N M A K E R [35], a personal computer-
based neural network software package which
implements the aforementioned back propaga-
tion training algorithm, was used to construct and
test trained neural network models. For each
network trained in the study, a structure of 5
input neurons (one for each financial ratio), 10
hidden neurons and 2 output neurons (one indi-
cating bankrupt firm, the other indicating non-
bankrupt firm) was used. Such a network struc-
ture was chosen on the basis of previously es-
poused heuristic guidelines [8,9,36]. Figure 1 pic-
torially illustrates this network.
As training cases are presented to the net-
work, the output neuron values are examined by
the training procedure. For instance, consider a
bankrupt training case that had its bankrupt out-
put node valued at 0.85 and its non-bankrupt
node at 0.17. When training a neural network, a
certain amount of variation away from the de-
sired values of 0 and 1 (indicating bankrupt or
non-bankrupt) is typically allowed at the output
layer when determining whether adjustments
should be made in network weights via back
propagation. This allowable variation is referred
to as the training tolerance. Thus, a training toler-
ance of .2 would allow the training procedure .2
variation of each output node away from the
desired value. Thus, the previous example would
549
BR = 1 NBR = 1
5" q" q" 5"
X1 X2 X3 X4 X5
Fig. 1. A typical NN model for bankruptcy prediction.
satisfy the training tolerance (i.e., 1 - (}.85 < 0.2,
and 0 . 1 7 - 0 < 0.2) and no network weight cor-
rection would take place.
In training the networks, a heuristic back
propagation algorithm was used to ensure conver-
gence (all firms in the training set classified cor-
rectly). A stringent training tolerance was initially
used in training the network (a small value of 0.1)
and gradually relaxed until such a point was
reached when all training cases satisfied the
training tolerance criteria. Then, the training tol-
erance was incrementally lowered (made more
stringent) and the network was trained until con-
vergence occurred at this level. This was repeated
until no further reductions of the training toler-
ance could occur. In all 180 subsamples gener-
ated, the neural network models were able to
obtain 100% classifications of the training set
cases. By using a relaxed tolerance, memorization
or overtraining should have been avoided.
3.4. Dependent variable: correct predictions
The intent of this study was to compare the
predictive capability of discriminant analysis and
neural networks. Thus, the number that each
method correctly predicts in the testing data sets
is the chief measure of predictive success. Other
measures are described as introduced.
SYSTAT, in using the multivariate general lin-
ear model, utilizes Mahalanobis distances [39] to
calculate posterior probabilities for each case,
550 R.L. Wilson and R. Sharda / Bankruptcy prediction using neural networks

indicating the likelihood of group membership Table 2

Training set - correct classifications (%) (grouped by like
for each group. Additionally, prior probabilities
composition)
were incorporated based upon the composition
(base rate) of the training sets. The group with Training Combined Non-bankrupt Bankrupt
set accuracy cases cases
the highest posterior probability, therefore, is
ratio
used as the discriminant analysis prediction for NN DA NN DA NN DA
that case. This is the manner in which correct and 50/50 100 88.65 100 94.54 100 82.76
incorrect classifications were determined for the 80/20 100 90.33 100 97.69 100 60.91
discriminant analysis method. 90/10 100 94.59 100 99.13 100 54.67

When evaluating the predictive capability of NN - Neural network

neural networks, a testing threshoM, similar to
training tolerance, is specified. This testing
threshold identifies how stringent the allowable
variation in output neurons can be when predict-
ing group membership. In this study, a testing
threshold of 0.499 was used; thus, if one output
neuron exceeded 0.5 (and the other neuron value
was < 0.5), the network classified the case as the
corresponding group associated with the first
neuron. Cases where double classifications were
indicated (both neurons > 0.5) were automati-
cally counted as incorrect classifications. It is on
this basis that correct and incorrect classifications
were determined for the neural network models.
Each data set was evaluated by both discrimi-
nant analysis and neural networks. Two different
measures of accuracy could be determined; the
number of correct classifications that the particu-
lar procedure provided on the training set
("learning") and the number of correct classifica-
tions that the specific procedure provided on the
testing set ("generalization").
4. Results
D A - Multivariate discriminant analysis
set are correctly classified. Thus, on the basis of
strictly learning a set of bankrupt and non-bank-
rupt firms, neural networks appear to have
learned more than classical multivariate discrimi-
nant analysis.
4. 2. Testing sets - " G e n e r a l i z a t i o n "
Perhaps a better measure of accuracy compari-
son between the two techniques is their perfor-
mance in classifying cases in the holdout samples,
or the testing sets. Table 3 represents the average
percentage of correct classifications (irrespective
of type of firm) when utilizing the two different
techniques to evaluate the 20 holdout samples for
each combination of base rates. When the train-
ing sets contained an equal number of bankrupt
and non-bankrupt cases, and the testing sets also
contained an equal number of the two cases,
neural networks correctly classified 97.5% of the
holdout cases, while multivariate discriminant
analysis was correct 88.25% of the time. Simi-

Table 3
4.1. Training sets - " L e a r n i n g "
Testing set - correct classification (%) (all cases)

The first results to be presented display the Training set Testing set composition

learning performance of the discriminant func- composition 50/50 80/20 90/10

tion and the neural network model. Table 2 shows NN DA NN DA NN DA
the aggregated percentage of correct classifica- 50/50 97.5 88.25 95.6 91.8 95.68 93.32
tions of training cases by the two approaches
across the three different combinations of train- (p < 0.001) (p = 0.005) (p = 0.046)
ing sets. Table 2 also distinguishes between the 80/20 82.0 75.875 91.0 89.0 95.68 91.59

learning accuracy of non-bankrupt and bankrupt (p = 0.002) (p = 0.126) (p = 0.001)

training cases. 90/10 72.625 72.0 86.25 85.8 94.55 91.81
It is not surprising that the neural network
approach outperforms discriminant analysis, since (p = 0.318) (p = 0.069) (p = 0.008)
the neural network training algorithm employed *-Significant at 0.05 level
will not cease until all members of the training * *-Significant at 0.01 level
 R.L. Wilson and R. Sharda / Bankruptcy prediction using neural networks 551

Table 4 Table 5
Testing set - correct classifications (%) bankrupt cases Testing set - correct classifications (%) non-bankrupt cases

Training set Testing set composition Training set Testing set composition
composition 50/50 80/20 90/10 composition 50/50 80/20 90/10

NN DA NN DA NN DA NN DA NN DA NN DA
50/50 97.0 79.75 92.0 82.0 92.5 90.0 50/50 98.0 96.75 96.5 94.25 96.0 93.5

(p < 0.001) (p = 0.025) (p = 0.282) (p < 0.029) (p = 0.071) (p = 0.038)

80/20 62.25 54.25 62.0 54.0 70.0 45.0 80/20 98.75 97.5 98.25 97.75 98.25 92.5
(p = 0.080) (p = 0.304) (p = 0.061)
(p = 0.002) (p = 0.115) (p = 0.06) 90/10 98.25 97.75 98.0 98.5 97.25 96.5
90/10 47.00 46.25 49.0 35.0 67.5 45.0 (p = 0.240) (p = 0.263) (p = 0.289)

* - Significant at 0.05 level

(p = 0.439) (p = 0.022) (p = 0.036)
** - Significant at 0.01 level

values of this test are given, and those significant

larly, when the training sets contained a balanced noted by asterisks. Again, for predicting bankrupt
number of bankrupt and non-bankrupt firms but cases, note that neural networks predicted better
the testing sets contained 20 percent bankrupt than discriminant analysis at every factor level
firms, neural networks classified at a 95.6% cor- combination. For instance, where training and
rect rate, while discriminant analysis correctly test set composition was equal among the two
classified 91.8%. Table 3 indicates that in every different classes of firms, neural networks cor-
combination of factor levels neural networks per- rectly predicted 97.0% of the bankrupt firms,
formed better at generalization than discriminant while discriminant analysis predicted only 79.75%.
analysis. Similarly, Table 5 presents the results for the
A non-parametric test, the Wilcoxon test for prediction of non-bankrupt cases by the two tech-
paired observations, was undertaken to assess niques. Significance is tested and reported as
whether the different correct classification per- mentioned previously. Both methods appear to
centages for the two different techniques were predict non-bankrupt firms quite well, though the
significantly different. The critical values of this neural network model predicts better than dis-
test are also reported in Table 3. Those experi- criminant analysis in all but a single combination
mental cells that are statistically significant are of factor levels (training set of 90-10, testing set
highlighted by asterisks. In general, neural net- of 80-20, and the difference is negligible and
works were statistically significant better predic- statistically not significant).
tors of firm bankruptcies in the holdout sample Table 6 summarizes the prediction classifica-
than discriminant analysis. tion results for all test sets at each level of the
Tables 4 and 5 provide a more detailed look at training set base rate and also differentiates be-
the classification results, breaking down the cor- tween the different categories of firms. From this
rect percentages in terms of bankrupt firm pre-
dictions and non-bankrupt firm predictions. It is
apparent from Table 4 that it is in the classifica- Table 6
Training composition effect on classification (%)
tion of bankrupt firms where neural networks
significantly out perform discriminant analysis. Training set Bankrupt Non-bankrupt Total
This is important since it is widely accepted in composition cases cases cases
terms of predicting bankrupt firms that it is more NN DA NN DA NN DA
costly to classify a failed firm as non-failing than 50/50 95.74 80.92 96.83 94.83 96.49 90.51
the converse [38]. 80/20 65.00 53.52 98.42 95.92 88.05 82.76
As with the overall aggregate classification 90/10 48.89 44.07 97.83 97.58 82.64 80.97
data, the Wilcoxon paired observation test was Total
used to assess the significance of the differences overall cases 90.56 85.87
of the two prediction techniques. The critical
552 R.L. Wilson and R. Sharda / Bankruptcy prediction using neural networks

and the previous tables, it is apparent that neural Table 7

networks represent a better predictive approach Percent predicted as bankrupt that were bankrupt
than multivariate discriminant analysis. When Training set Testing set composition
considering all 60 cases where a balanced training composition
50/50 80/20 90/10
set was used, the neural network model correctly NN DA NN DA NN DA
predicted 95.74% of the bankrupt firms and
50/50 98.0 94.9 86.8 78.1 69.8 58.1
96.83% of the non-bankrupt firms in the holdout 80/20 98.0 95.6 89.9 85.7 80.0 37.5
samples; as compared to 80.92% of bankrupt 90/10 96.4 95.4 86.0 85.4 71.0 56.3
firms and 94.83% of non-bankrupt firms that
discriminant analysis predicted. While the per-
centage of correct classifications of bankrupt firms
decreased with the increased imbalance of the
training cases, the general trend of neural net- correct 98% of the time, while discriminant anal-
work prediction superiority remained. ysis was correct 94.9%. Note that neural networks
outperformed discriminant analysis irrespective
4.3 Further assessment of predictive capabilities of factor levels (base rates). Also note that, with
the exception of measuring prediction with test
While the results have clearly shown that neu- sets having 90 percent non-bankrupt firms, neural
ral networks outperformed discriminant analysis networks also exceeded the base rate of the most
in predicting firm bankruptcies, our study must frequent class, indicative of good predictive valid-
now address whether the neural network predic- ity.
tion results are better than what can be expected However, this analysis used the base rates of
by pure chance [22,31]. We will employ tests the testing sets, information not available to the
originally proposed in [29] and further clarified in classification technique. Only the base rate of the
[22] for studying discriminant analysis classifica- training set is 'learned' by the classification de-
tion rates. Because our study uses cross-valida- vice. Thus, a pure chance technique, exposed to
tion (i.e., testing sets) for measuring classification 90% non-bankrupt cases in training, would ran-
success and utilizes different base rates for the domly declare 90% of testing cases to be non-
training and testing set, we will further modify bankrupt, irrespective of the testing set composi-
these tests to fit our study. tion. In further investigating the value added by
The underlying concept in comparing a classi- discriminant analysis and neural networks to the
fication or prediction technique to pure chance is classification problem, our standard normal test
to consider what one could do by simply guessing statistics will be based on only information known
at the predictions. For instance, if the base rate to the classification techniques (base rate of the
was 50% for a two group problem, guessing would training sets).
result, in average, 50% correct predictions. Simi- The test statistic utilized will be based upon
larly, if the base rate was skewed (80%-20%), one the proportional chance criterion [22]. This crite-
could blindly predict with 80% accuracy by pre- rion implies that prediction by guessing can
dicting all cases to belong to the more frequent achieve a correct rate for each group involved
class [28,29]. It has been shown that to achieve equal to the proportion of that group (base rate)
significant levels of predictive validity, the pro- in the training set. Thus, for those training sets
portion of correct positive predictions (bankrupt with a balanced number of bankrupt and non-
firms, in our case) to all positive bankrupt predic- bankrupt firms, 50% correct predictions could be
tions must exceed the base rate of the more achieved by chance, while when there are 90%
non-bankrupt firms, 90% correct predictions of
frequent class [17,29].
In Table 7, this proportion is calculated for non-bankrupt firms could be achieved by chance.
both neural networks and discriminant analysis. The following standard normal test statistic is
Thus, when both the training and testing sets calculated as
contained a balanced number of bankrupt and ( 0 - E ) * N t/2
non-bankrupt firms (base rate of 50%), a neural (E .(N_E)),/2 , (1)
network model forecasting a bankrupt firm was
 R.L. Wilson and R. Sharda / Bankruptcy prediction using neural networks 553

Table 8
Predictive validity of classifications

Training set Bankrupt Non-bankrupt Total

composition NN DA NN DA NN DA
50/50 2.88 1.86 4.19 4.00 5.19 4.36
(p = 0.002) (p = 0.031) (p < 0.001) (p < 0.001) (p < 0.001) (p < 0.001)
80/20 3.11 2.31 2.06 1.73 2.57 2.33
(p < 0.001) (p = 0.011) (p = 0.020) (p = 0.042) (p = 0.005) (p = 0.010)
90/10 3.89 3.41 1.17 1.13 1.88 1.70
(p < 0.001) (p < 0.001) (p = 0.121) (p = 0.129) (p = 0.030) (p = 0.045)

where g = groups (bankrupt and non-bankrupt), chance assignment. An index useful in such a
ng = number of test cases in group g , bg = training setting is the improvement-over-chance or reduc-
base rate of group g, og = observed correct pre- tion-in-error index [22,26],
dictions for group g (refer to Table 4), eg = I-Io -- l-Ie
expected correct predictions for group g by
I 1 -H e ' (3)
chance (ng* bg), O = total correct prediction
(~ Og), E = total correct predictions obtainable where H o is the observed rate of correct predic-
by chance (Y~ eg), N = total number of cases tions and He is the correct prediction rate ex-
(E //g) pected by chance. Using the previous notation,
Thus, this statistic will indicate whether pre- H e is defined as (E(bg * n g ) ) / N for the aggregate
dictive results obtained by neural networks and case, and be for each separate group. The index
discriminant analysis differ greatly from those I represents a reduction-in-error statistic in that
that can be obtained by chance. Additionally, one 1 0 0 . 1 % fewer prediction errors result using the
can also calculate a similar statistical measure for classification rule than would be expected by
each separate classification group. Using the same chance.
notation as above, the standard normal test statis- Table 9 provides this calculation for the neural
tics for each group (illustrating whether the pre- network and discriminant analysis predictions ag-
dictive results obtained by a classification tech- gregately across firm type, as well as the improve-
nique significantly differs from chance) is ment-over-chance index for both bankrupt and
non-bankrupt cases. Thus, when the 50-50 train-
( O g - e g ) * l ' l g 1/2 ing set is used to train a neural network, the
(eg * (F/g -- e g ) ) 1/2" (2) network model provides 92.98% fewer classifica-
tion errors than would occur by blind guessing.
As Table 8 indicates, the predictive validity of Also, as another example, the improvement-
neural networks and discriminant analysis is ex- over-chance for the prediction of bankrupt firms
tremely significant. Aggregately, both methods with neural networks trained on a balanced train-
are significantly better than pure chance regard- ing set is 91.48%. Also of interest is that even on
less of the base rate of the training sets. Consid- the 90% base rate where neural networks did not
ering the predictive validity by specific groups, indicate significant differences over chance pre-
the only non-significant result occurs when pre- dictions for non-bankrupt cases, the improve-
dicting non-bankrupt firms when the training set
base rate is 90%, though it is still considerably
better than chance. Not surprisingly, as previous Table 9
results have already shown, neural networks are Reduction-in-error of classifications
judged more statistically significant than chance
Training set Bankrupt Non-bankrupt Total
as compared to discriminant analysis in every
composition NN DA NN DA NN DA
case.
Another approach useful in assessing a predic- 50/50 91.5 61.8 93.7 89.7 93.0 81.0
80/20 56.2 41.9 92.1 79.6 69.1 55.4
tion method is determining how much better a
90/10 43.2 37.8 78.3 75.8 50.2 45.4
classification approach predicts compared to
554 R.L. Wilson and R. Sharda / Bankruptcy prediction using neural networks

Table 10 The moderately significant effect of the testing

A N O V A - Bankrupt cases
set composition is not as easily explained. Con-
Source Sum of DF Mean-square F Ratio P trasting the different factor levels, only the differ-
squares ence between the 80/20 and 90/10 evaluation
Train 49246.942 2 24623.471 49.401 0.000 sets were significant (p = 0.0286). From Table 3,
Test 2663.610 2 1331.805 2.672 0.072 it can be seen that similarly trained networks
Interaction 3397.222 4 349.305 1.704 0.151
were evaluated more favorable when the testing
Error 85233.750 171 498.443
set was composed of 90 percent non-bankrupt
R = 0.627 R 2 = 0.394 cases and 10 percent bankrupt firms. The 80/20
and 50/50 testing sets showed comparable mea-
sures of prediction accuracy across different
training factor levels. This variation can perhaps
ment-over-chance percentage is still a relatively
be explained by the small number of bankrupt
high 78.3%.
cases found in the holdout samples of factor level
90/10 (2 cases). By having such a small number
4.4. Effect of training and testing set composition
of test cases, the random generation of cases may
on generalization
have led to this experimental finding.
Table 11, the ANOVA on the non-bankrupt
In order to further assess the effect on accu-
cases, shows that only the composition of the
racy of classifications that the factor levels of
training set significantly effects neural network
training and testing set composition have on neu-
predictions. Upon closer contrast analysis among
ral network model predictions, two two-factor
the three different factor levels, the only signifi-
ANOVA's were undertaken, one using the per-
cant difference between levels is between the
centage of correct classifications of non-bankrupt
50/50 and 80/20 composition (p = 0.010). In fact,
firms as the dependent variable, another utilizing
predictive accuracy of networks trained by the
correct predictions of bankrupt firms as the de-
80/20 composition sets provided more accurate
pendent variable. The results of these two
results in classifying non-bankrupt cases than the
ANOVA's are presented in Tables 10 and 11.
50/50 training sets.
Similar analysis was not undertaken for the dis-
criminant analysis results since the neural net-
work approach clearly dominates its' perfor-
5. Discussion
mance.
The composition of the training set was signifi-
From the results of this experiment, it is ap-
cant in determining the neural network predic-
parent that for the bankruptcy prediction prob-
tion accuracy of the bankrupt test cases. This
lem, neural networks offer a viable alternative
result further reinforces the intuitive thought that
approach. With simple data (five variables), neu-
to properly train a network (or any model) to
ral networks showed extreme promise by cor-
recognize two different concepts, utilizing an
rectly predicting as high as a 97% accuracy level
equal number of examples of each concept is
(when both the training and testing base rates
desirable [23].
were 50/50). This level is as good as or better
than other studies. In every instance, neural net-
works outperformed discriminant analysis in clas-
Table 11 sification accuracy, especially in the prediction of
A N O V A - Non-bankrupt cases bankrupt finns, the more difficult and, arguably,
Source Sum of DF Mean-square F Ratio P the more important classification problem [38]. It
squares stands to reason that neural networks will per-
Train 76.944 2 38.472 3.492 0.033 form as well or better with the inclusion of more
Test 41.944 2 20.972 1.904 0.152 variables in the analysis. Thus, the results of this
Interaction 15.556 4 3.889 0.353 0.842 exploratory study could be considered to offer a
Error 1883.750 171 11.016
lower bound on the predictive accuracy one can
R = 0.258 R 2 = 0.067 expect with a neural network model for
 R.L. Wilson and R. Sharda / Bankruptcy prediction using neural networks
bankruptcy prediction. Of course, the results of
any study are bound by the limitations of the data
and methodology.
Discriminant analysis classification rules often
incorporate prior probabilities that account for
both the assumed base rate and the costs associ-
ated with misclassification errors if different
[16,27]. In our comparison study, prior probabili-
ties were calculated from the base rates of the
training sets. By using the base rates as the prior
probabilities, the discriminant analysis procedure
in this study actually incorporates significant un-
equal misclassification costs (i.e., misclassifying a
bankrupt firm is a more costly error), since the
true population of bankrupt firms is probably less
than the training set base rate. Even so, neural
networks continually predicted bankrupt firms
more accurately using symmetric costs (testing
threshold of 0.499). The major dilemma in utiliz-
ing the discriminant analysis model is in estimat-
ing the unequal misclassification costs. Future
research investigating performance adjustments
given explicit values for asymmetric misclassifica-
tion costs for both discriminant analysis and neu-
ral networks may be warranted.
The investigation of the effects of different
training and testing set composition on the pre-
dictive results lead to further implications for the
decision maker and neural network researcher.
Results indicated that the composition of the
training set was a significant determinant of neu-
ral network predictive accuracy. Basically, it was
shown that neural networks provide better under-
standing and differentiation between two con-
cepts (bankrupt firms and non-bankrupt firms)
when an equal number of examples of each con-
cept is used in the learning procedure. This result
is not dissimilar to one's intuition and previous
results in discriminant analysis [23].
While all prediction errors are undesirable in a
specific methodology, it is generally accepted that
the incorrect prediction of a bankrupt firm as
non-bankrupt is the most costly error. Results
have indicated that prediction of the bankrupt
firms poses the largest problem to the two differ-
ent techniques. Neural networks were shown to
perform well in predicting both bankrupt firms
and non-bankrupt firms when presented with
equal numbers of examples in the learning phase.
Thus, a more accurate classification model will
result when developed with an equal number of
555
instances of each category. Since in the real-world,
the decision maker may not have control over the
composition of historical data necessary in the
predictive model development, it appears that
"smoothing" the distribution of the training set,
irrespective of the actual distribution, will provide
a better model.
It is true that neural network performance is
less impressive as the proportion of non-bankrupt
to bankrupt firms diverge. However, neural net-
work models continue to outperform discriminant
analysis. If one follows the recommendation of a
50-50 training set, neural network performance
does not deteriorate significantly. Bankrupt firms
are predicted correctly in the 92% to 97% range,
with similar accuracy for non-bankrupt firms,
given a balanced training set.
One caution to this approach in developing the
training set is also indicated in the experimental
results. Significance of testing set composition in
bankrupt firm prediction may have indicated
over-reported accuracy due to the small number
of bankrupt firms in the 90-10 test sets. Thus, this
study indicates that a potential trade-off exists
when creating training and testing sets from the
pool of existing problem data. A better predictive
neural network model can be created by using a
balanced training set; however, if too few of the
hard-to-classify or more important cases exist in
the cross-validation set, the model performance
could be over or under reported. Either way, this
will significantly effect the accuracy of decision
maker confidence in the prediction model.
The results of predicting non-bankrupt cases
improved as the imbalance of bankrupt to non-
bankrupt firms increased in the training sets. This
can be attributed to significant fewer number of
bankrupt firms in the training sets. This phe-
nomenon illustrates that, at the expense of
"learning" about bankrupt firms, the network
"memorizes" and becomes very good at recogniz-
ing (i.e., predicting) non-bankrupt firms. While
overall predictive accuracy may remain high, the
classification accuracy of bankrupt firms is seri-
ously reduced. Thus, one would be significantly
sacrificing the prediction performance of one im-
portant category to marginally increase the pre-
diction performance on the other, easier pre-
dicted category. In firm bankruptcy predictions,
this is obviously not desirable. Thus, great care
must be taken when creating the training and
556 R.L. Wilson and R. Sharda / Bankruptcy prediction using neural networks
cross-validation sets when developing a neural
network prediction model.
From a decision support systems perspective,
this study has illustrated that neural networks are
a viable model that should be included in the
model base of a DSS. Predictive accuracy ob-
tained in this study illustrates the potential of
neural networks from a data reduction stand-
point. With only five simple ratios, neural net-
works predicted at a high rate of classification
accuracy; thus, these models may provide excel-
lent results with less data requirements than other
approaches to the problem.
Discriminant analysis is not the only tool that
has been postulated for use in classification prob-
lems [13]. However, all other models do have
limitations with regard to successful and appro-
priate use. In the case of discriminant analysis,
limitations include the requirement that the vari-
ables should be jointly distributed according to a
multivariate normal distribution, prior probability
specification, and so forth. Neural networks have
no such potential restrictive assumptions or re-
quirements; they are more robust prediction tech-
niques. Thus, neural networks offer additional
benefits in reducing managerial concern over
choosing the appropriate model in the decision
support context.
Much additional research needs to be done
regarding neural networks for bankruptcy predic-
tion. The effect of network architecture, network
training algorithms and learning paradigms need
to be examined to provide more prescriptive re-
sults on implementing a neural network predic-
tion model. As previously mentioned, this ex-
ploratory study uses only a small amount of vari-
ables to achieve its' high level of predictive accu-
racy; other variables should be included in the
neural network model [2]. Notable omissions in-
clude the size of the firm, and time series data
(more than just one years' previous financial data),
among others. Additionally, using matched firms
by industry and year has been postulated to bias
results in predicting bankruptcy [40]. Neural net-
works may or may not be affected by this, but
additional research should study this issue.
6. Conclusion
This paper has compared the predictive capa-
bility of neural networks with that of classical
multivariate discriminant analysis within the con-
text of forecasting firm bankruptcies on the basis
of a small number of financial ratios. In this
study, neural networks clearly outperformed dis-
criminant analysis in prediction accuracy of both
bankrupt and non-bankrupt firms under varying
training and testing conditions. Additionally, it
was shown that neural networks offer a signifi-
cant improvement in prediction over pure chance,
and that their use in prediction can reduce errors
in this problem domain by as much as 93% over
chance.
Neural networks, therefore, represent a classi-
fication technique that is a robust and promising
approach in the prediction of firm stability. While
this study is exploratory in nature and has some
limitations as noted, it has shown the promise of
neural networks through the use of a set of solid
statistical analyses that should be utilized as re-
search continues in this area.
Acknowledgments
The authors wish to sincerely thank Marcus
Odom and Nik Dalai for their help and assistance
in data collection and in their insightful com-
ments on previous drafts of this paper. Also, the
paper has greatly benefitted from comments and
suggestions from the anonymous referees.
References
[1] Altman, E.I., Financial Ratios, Discriminant Analysis
and the Prediction of Corporate Bankruptcy, The Jour-
nal of Finance, (September 1968), 589-609.
[2] Altman, E.I., Haldeman, R.G. and Narayanan, P., Zeta
Analysis, Journal of Banking and Finance, (June 1977),
22-51.
[3] Altman, E.I., Accounting Implications of Failure Predic-
tion Models, Journal of Accounting Auditing and Fi-
nance, (Fall 1982), 4-19.
[4] Baetge, J., Huss, M. and Niehaus, H., The Use of Statis-
tical Analysis To Identify The Financial Strength Of
Corporations In Germany, Studies in Banking and Fi-
nance, Vol. 7 (1988), 183-196.
[5] Beaver, W.H., Financial Ratios as Predictors of Failure,
Empirical Research in Accounting: Selected Studies
(1966), 71-111.
[6] Bell, T., Ribar, G. and Verchio, J., Neural Nets vs.
Logistic Regression: A Comparison of Each Model's
Ability to Predict Commercial Bank Failures, working
paper, Peat Marwick Co., (May 1990).
 R.L. Wilson and R. Sharda / Bankruptcy prediction using neural networks
[7] Blum, M., Failing Company Discriminant Analysis, Jour-
nal of Accounting Research, (Spring 1974), 1-25.
[8] Caudill, M., Neural Network Primer: Part III, AI Expert,
(June 1988) 53-59.
[9] Caudill, M., Neural Network Training Tips and Tech-
niques, AI Expert, (January 1991), 53-59.
[10] Collard, J.E., Commodity Trading with a Neural Net,
Neural Network News, Vol. 2, No. 10 (October, 1990).
[11] Collins, E., Ghosh, S. and Scofield, C., An Application of
a Multiple Neural Network Learning System to Emula-
tion of Mortgage Underwriting Judgments, working pa-
per, Nestor, Inc. (1989).
[12] Deakin, E.B., A Discriminant Analysis of Predictors of
Business Failures, Journal of Accounting Research,
(Spring 1972), 167-179.
[13] Denton, J., Hung, M. and Osyk, B., A Neural Network
Approach to the Classification Problem, Expert Systems
With Applications, Vol. 1, (1990), 417-424.
[14] Dutta, S. and Shekhar, S., Bond-Rating: A Non-Con-
servative Application of Neural Networks, Proceedings of
the IEEE International Conference on Neural Networks,
San Diego, (1988) 443-450.
[15] Eisenbeis, R. and Avery, R., Discriminant Analysis and
Classification Procedures. (Lexington Books, Lexington
MA, 1972).
[16] Eisenbeis, R., Pitfalls in the Application of Discriminant
Analysis in Business, Finance and Accounting, Journal of
Finance, (June 1977) 875-900.
[17] Farrington D.P. and Tarling, R., Prediction in Criminol-
ogy. (State University of New York Press, Albany, NY,
1985).
[18] Fishman, M., Barr, D. and Loick, W., Using Neural
Networks in Market Analysis, Technical Analysis of
Stocks and Commodities, (April 1991) 18-25.
[19] Francett, B., Neural Nets Arrive, Computer Decisions,
(Jan. 1989) 58-62.
[20] Gallant, S.I., Connectionist Expert Systems, Communica-
tions of the ACM, (February 1988), 152-169.
[21] Goodman, R.M., Miller, J.W. and Smyth, P., An Infor-
mation Theoretic Approach to Rule-Based Connectionist
Systems, in Advances in Neural Information Processing
Systems I, D.S. Touretsky ed., (Kaufman Publishing: San
Mateo, CA, 1989) 356-364.
[22] Huberty, C.J., Issues in the Use and Interpretation of
Discriminant Analysis, Psychological Bulletin, Vol 95
(1984), 156-171.
[23] Jain, A. and Chandrasekaran, B., Dimensionality and
Sample Size Considerations in Pattern Recognition Prac-
tice, in Handbook of Statistics, Vol. 2, P. Krishnaiah and
Kanal, L. eds. (North-Holland, 1982), 835-855.
557
[24] Kamijo, K. and Tanigawa, T., Stock Price Pattern Recog-
nition: A Recurrent Neural Network Approach, Interna-
tional Joint Conference on Neural Networks, San Diego,
(June 1990).
[25] Karels, G.V. and Prakash, A., Multivariate Normality
and Forecasting of Business Bankruptcy, Journal of Busi-
ness Finance and Accounting, (Winter 1987), 573-593.
[26] Klecka, W.R., Discriminant Analysis. (Sage Publishing:
Beverly Hills, CA, 1980).
[27] Lachenbruch, P., Discriminant Analysis. (Hafner Press:
NY, NY, 1975).
[28] Meehl, P.E., Clinical versus Statistical Prediction: A The-
oretical Analysis and a Review of the Evidence. (Univer-
sity of Minnesota Press: Minneapolis, 1954).
[29] Meehl, P.E. and Rosen, A., Antecedent Probability and
the Efficiency of Psychometric Signs, Patterns or Cutting
Scores, Psychological Bulletin, 52, (1955) 194-216.
[30] Mighell, D., Back-Propagation and its Application to
Handwritten Signature Verification, in Advances in Neu-
ral Information Processing Systems I, D.S. Touretsky ed.
(Kaufman Publishing: San Mateo, CA, 1989) 340-347.
[31] Morrison, D.G., On the Interpretation of Discriminant
Analysis, Journal of Marketing Research, Vol. 6, (1969),
156-163.
[32] Moyer, R.C., Forecasting Financial Failure: A Reexami-
nation, Financial Management, (Spring 1977), 11-17.
[33] Rochester, J. (ed.) New Business Uses For Neurocomput-
ing, I/S Analyzer, (Feb 1990), 1-17.
[34] Siegel, J.G., Warning Signs of Impending Business Fail-
ure and Means to Counteract such Prospective Failure,
The National Public Accountant, (April 1981), 9-13.
[35] Stanley, J. and Bak, E., Introduction to Neural Networks.
(California Scientific Software, Sierra Madre, CA, 1989).
[36] Surkan, A. and Singleton, J., Neural Networks For Bond
Rating Improved by Multiple Hidden Layers, Interna-
tional Joint Conference on Neural Networks, San Diego,
(June 1990).
[37] Teebagy, N. and Chatterjee, S., Inference in a Binary
Response Model with Applications to Data Analysis,
Decision Sciences, Vol. 20, No. 2, (1989), 393-403.
[38] Watts, R.L. and Zimmerman, J.L., Positive Accounting
Theory. (Prentice-Hall, 1986).
[39] Wilkinson, L., SYSTAT: The System for Statistics. (SYS-
TAT, Inc., Evanston, IL, 1989).
[40] Zmijewski, M.E., Methodological Issues Related to the
Estimation of Financial Distress Prediction Models,
Journal of Accounting Research, Vol. 22 Supplement
(1984), 59-82.