Paper 3A 07 SINOROCK2004 Symposium

Int. J. Rock Mech. Min. Sci. Vol. 41, No. 3, CD-ROM, 2004 Elsevier Ltd.

USING LOGISTIC REGRESSION TO INVESTIGATE

AND IMPROVE AN EMPIRICAL DESIGN METHOD
C.A Mawdesley
) Coffey Geosciences Pty. Ltd.
clare_mawdesley@coffey.com.au

Abstract: Stability graphs are empirical design tools regularly used for estimating the stability of mining
excavations. The principal concept behind stability graphs is that the size of an excavation surface can be
related to rock mass competency for an indication of instability. Stability graphs are a simple, non-rigorous
design approach, based on accepted rock mass classification schemes and past excavation case histories. Past
excavation case histories are plotted on the stability graph and zones of varying stability defined. This paper
documents the use of statistics to investigate the stability database and underlying framework of the
Extended Mathews stability graph method. Logistic regression has been used to improve the objectivity and
accuracy of stability zones, calculate probabilities of stability, and quantify some of the risks and
uncertainties in the method. The statistical investigation of the underlying database has enabled direct
quantification of proposed improvements and comparison of model accuracies. Although this study uses a
mining example, the use of logistic regression to investigate the framework of an empirical model is
applicable to investigating any group of categorical data.
Keywords: stability graph, hydraulic radius, Q, logistic regression, empirical, Mathews.

1. INTRODUCTION
The Extended Mathews stability graph is a
variant of the original Mathews stability graph
method (Mathews et al., 1981). The Mathews
method for predicting stope stability was first
proposed in 1980. A Modified stability graph
variant was subsequently developed by Potvin
(1988).
A large database of new mining case histories
was compiled and this data was used to compare
the accuracy of the original and Modified methods
(Trueman et al., 2000). As no improvement in
accuracy was demonstrated by the newer, Modified
stability graph approach, the original Mathews
stability graph method was selected for subsequent
work. Many new mining cases were added to the
method and the Extended Mathews database now
contains 483 open stoping and caving case histories
over a wide range of geotechnical conditions and
stope dimensions. The format of the Mathews
stability graph has been changed to reflect the
broader range of stope geometries, rock mass
conditions and mining methods now captured
within the expanded database (Mawdesley et al,
2001; Trueman and Mawdesley, 2003).
Logistic regression was applied to statistically
delineate and optimise the placement, and calculate

accuracies and probabilities of the stability

boundaries. Logistic regression also enabled
isoprobability contours to be calculated and drawn
on the stability graph. These contours are a
valuable design tool in quantifying the risks and
uncertainties in the Mathews method.
The Extended Mathews framework and the
large stability database have facilitated the
investigation of a number of concerns about the use
of empirical stability graphs. This paper
summarises how logistic regression was used to
investigate the framework of the Extended
Mathews stability graph method and the underlying
database.
Although this is a mining example, the logistic
investigation applied to develop and improve the
understanding of an empirical design tool has a
wider relevance. With the acknowledged value of
empirical design tools in geotechnical engineering,
logistic regression provides a technique for
objectively investigating the various components
and uncertainties of an empirical model based on
categorical data. A key benefit of logistic
regression is the means of quantifying the accuracy
of a given model. By comparing model fit, there
can be certainty that proposed improvements to an
empirical model have a corresponding increase in
model accuracy.

Paper 3A 07 SINOROCK2004 Symposium

Int. J. Rock Mech. Min. Sci. Vol. 41, No. 3, CD-ROM, 2004 Elsevier Ltd.

2. BACKGROUND
The Mathews method is based on a stability
graph relating two calculated factors: the Mathews
stability number (N) which represents the
competency of the rock mass for a given stress
condition, and the shape factor (S), or hydraulic
radius, which accounts for the geometry of the
surface.
The original Mathews stability graph was divided
into stable, potentially unstable and potential
caving zones according to the scatter of the
stability data (Figure 1). The initial stability zones
and graph devised by Mathews et al. (1981) were
based on 50 case histories.

Figure 1. The original Mathews stability graph

method (after Mathews et al, 1981)
The Mathews method utilizes the Q
classification system (Barton et al., 1974) to
characterise rock mass quality. A modified Q-value
(Q'), is calculated from the results of structural
mapping or geotechnical core logging of the rock
mass according to the Q system. Q' assumes that
the joint water reduction parameter and stress
reduction factor are both equal to one;

Q' =

RQD J r

Jn
Ja

(1)

where RQD is the Rock Quality Designation index

(Bieniawski, 1989); Jn is the joint set number; Jr is
the joint roughness; and Ja is the joint alteration.
The Mathews stability number is determined by
adjusting the Q' value for induced stresses,

discontinuity orientation and the orientation of the

excavation surface;

N = Q' A B C
(2)
where Q' is the modified Q-value (Equation 1); A
is the stress factor; B is the joint orientation factor;
and C is the surface orientation factor.
The stress factor is determined from the ratio of
the intact rock strength (unconfined compressive
strength) to the induced stress at the centre-line of
the stope surface.
A graph relating the strength to stress ratio and
rock stress factor was developed by Mathews et al
(1981) (Figure 2, Factor A). In the original
Mathews stability graph, the joint orientation factor
is a measure of the relative difference in dip
between the stope surface and the critical joint set.
The surface orientation factor considers the
inclination of the excavation surface and its
influence on stability (Figure 2, Factor C).
The geometry of the excavation is considered
by calculating the shape factor or hydraulic radius
of the surface. The shape factor of an excavation
surface is defined as the area of the stope surface
divided by the length of its perimeter.
The initial stability graph was based upon a
limited number of case studies primarily from
North American and Australian steeply-dipping
open stopes in strong rock of medium to good
quality (Mathews et al., 1981). Originally, caving
referred to large stoping failures. However, that
term is considered different to true caving (i.e.
continuous caving) such as that achieved in a block
cave. Accordingly, some of the original failure data
was reclassified to distinguish caving from major
failures (Mawdesley, 2002).
Significantly more stable, failure, major failure
and caving data have been collected (Trueman et
al., 2000; Mawdesley, 2002), which extend over a
much wider range of stope sizes and rock mass
characteristics than had been previously
documented. These were combined with existing
data to produce an extensive database of 483
entries 314 stable, 91 failures, 63 major failures
and 15 continuous caving case histories.
The Extended Mathews stability database
includes rock masses with Q' values ranging
between 0.01 and 90, and stope surfaces with
hydraulic radii from 1 to 55 metres (Figure 3).

Paper 3A 07 SINOROCK2004 Symposium

Int. J. Rock Mech. Min. Sci. Vol. 41, No. 3, CD-ROM, 2004 Elsevier Ltd.

3. LOGISTIC REGRESSION
Optimising the placement of the stability zones
on the stability graph is a critical part of improving
the reliability and minimising the inherent
subjectivity of the Extended Mathews stability
graph method.
Factor A
Stress factor

Factor B
Joint orientation factor

A statistical approach was sought to optimise

the placement and accuracy of the stability zones
on the stability graph to remove the subjectivity
involved in delineating the zones. The stablefailure and failure-caving boundaries on the
Extended Mathews stability graph were determined
using logistic regression. The advantages of
delineating
the
stable-failure
boundary
mathematically, instead of by eye, include
increased objectivity and the ability to quantify
variance in the stable-failure boundary.
A logistic regression was undertaken because
problems with binary or categorical outcomes are
not easily modelled using traditional regression
techniques. Ordinary linear regression does not
consider the discrete nature of the dependent
variable (Liao, 1994). Logistic regression utilises a
non-linear transform to convert a linear
combination of independent variables to a zero to
one (or true/false) probability scale (DeMaris,
1992).
The logistic regression line defining the stability
boundaries is defined by Equations 3 and 4, where
P(z) is the logit value. The logit value is analogous
to the response variable in a linear regression
model and is determined for each data point based
upon the stability number, the hydraulic radius and
the stability.

z = 1 ln( N ) + 2 ln( S ) + 3
(3)

P( z ) =

Factor C
Surface orientation factor

Figure 2. The adjustment factors for determining

the Mathews stability number (after
Mathews et al, 1981).

1
(1 + e z )

(4)

The logit model parameters (1 , 2 and 3 ) are

determined using the maximum likelihood function
contained within the Matlab procedure logitfit
(Holtsberg, 1998).
A new log-log stability graph format was
adopted so that individual cases could be clearly
distinguished on the graph (Figure 3).
It is emphasised that the use of statistical
regression must not be mistaken as adding a greater
level of rigour to the Mathews stability graph
method.
Logistic regression should be viewed as an
objective means of calculating zone boundaries and
isoprobability contours for the available stability
data. Through logistic regression the risks
associated with using the technique can now be
quantified and the statistical significance of the
stability zones understood.

Paper 3A 07 SINOROCK2004 Symposium

Int. J. Rock Mech. Min. Sci. Vol. 41, No. 3, CD-ROM, 2004 Elsevier Ltd.

Extended Mathews Stability Chart

1000.000
Stable Line
57 % Stable
43% Falilure
0% Caving

STABLE
100.000

FAILURE & MAJOR FAILURE

Stability Number, N

10.000

Caving Line
95 % Caving
5% Failure
0% Stable
1.000

CAVING

0.100

Legend

0.010

STABLE
FAILURE
MAJOR FAILURE
CAVING

0.001
1

10

100

Shape Factor, S or Hydraulic Radius (in metres)

Figure 3. Extended Mathews stability graph, showing stable and caving boundaries calculated using
logistic regression.

Investigating logit model accuracy

The accuracy of a binary diagnostic test (in this
case, a boundary on the Extended Mathews
stability graph) can be summarised in terms of the
proportions of the correctly and incorrectly
classified cases. It is possible to determine the
proportion of misclassified cases for any given
stability boundary by comparing the true stability
status with the fitted logit model stability
boundaries (which represent the predicted stability
outcome).
The classification approach was used to
evaluate the accuracy or goodness of fit of the logit
calculated stability boundaries (Parker and Davis,
1999). For a single stability class, such as stable,
this involves considering both the proportion of
stable points correctly identified as stable and the
proportion of unstable points correctly classified as
unstable.
Parker and Davis (1999) refer to the proportions
measured in the classification approach as
sensitivity and specificity. Sensitivity is defined as
the probability that a true case will be correctly

classified. For the stability data, this refers to the

probability that a stable case will be correctly
classified as stable. Specificity is the probability
that a false case will be correctly classified as false
(i.e. an unstable case is classified as either a failure,
major failure or caving case).
The sum of the sensitivity and specificity is
defined by Parker and Davis (1999) as the accuracy
of the test classification, giving equal weighting to
sensitivity and specificity, and has a maximum
value of two for a perfect classification result.
Sensitivity, specificity and accuracy are
measures of misclassification and they can be used
to evaluate the goodness of fit of any logistic
regression model. Determining the sensitivity,
specificity and accuracy of individual stability
boundaries is beneficial in understanding the
significance of the placement of the boundaries
with respect to the distribution of the stability data.
The key benefit of an objective measure of
model fit is in comparing models and assessing
which model best fits the underlying data. The
accuracy of the calculated stability boundaries can
be determined for range of boundary positions.

Paper 3A 07 SINOROCK2004 Symposium

Int. J. Rock Mech. Min. Sci. Vol. 41, No. 3, CD-ROM, 2004 Elsevier Ltd.

Figure 4 shows the accuracy of the various stability

boundaries for variations in boundary position (as
defined logit value). This allows the significance of
moving the boundaries to be investigated.
Accuracy plot
2.0

1.9

Accuracy = sensitivity + specificity

1.8

1.7

1.6

1.5

1.4

1.3

1.2

Stable-failure line
Failure-major failure line

1.1

Caving line
1.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Logit value of the stability boundary

Figure 4. Plot of accuracy with changing logit

value for the boundaries on the
Extended Mathews stability graph.
Database subsets or new data can be compared
with current logit model coefficients using
confidence intervals. Asymptotically, each
estimated parameter of a fitted model will have a
normal distribution, and therefore each parameter
can be tested with a t-test (Bergerud, 1996). The
data was found to be approximately normally
distributed, which justified the use of the t-test. The
standard deviations of the regression parameters
were calculated from boot strapping which also
suits the t-test. Figure 5 shows the comparison of
logit coefficients for three data groups of stability
data.
95 % Confidence intervals for the regression coefficients
Regression coefficient

Magnitude of coefficient

Regression coefficient 2
Regression coefficient 1

All Data
-1

JKMRC Data Non-JKMRC

Data

All Data

JK Data

Non-JKMRC
Data

All Data

JKMRC Data Non-JKMRC

Data

-2

-3

-4

Database subset and regression coefficient

Figure 5. Confidence intervals on regression

coefficients from the fitted boundaries
on the Extended Mathews graph.

Testing stability graph parameters

Statistical investigations were undertaken to
improve understanding of the strengths and
weaknesses of the framework of the Extended
Mathews method (Mawdesley, 2002). By analysing
logit model statistics it was possible to investigate
the significance of each of the six Mathews
stability parameters (S, N, Q', A, B, C).
Shape factor was the strongest determinant of
stability. Stability number proved to have a poorer
correlation with excavation stability than shape
factor, but was significantly better than Q', A, B or
C. The sensitivity analysis indicated that the
stability number is a stronger determinant than any
of its components.
In addition to evaluating parameters
individually, S, N, Q', A, B and C were considered
in various combinations and formulations. No
better determinant than the original stability
number was found, either individually or by
considering combination logit models. It was
concluded from the sensitivity analysis that the
current formulation or framework of the Mathews
method could not be improved upon with the
existing parameters (Mawdesley, 2002). However,
this does not exclude the fact that the addition of
new parameters or the development of an
alternative model framework in the future could
improve model accuracy.

4. CONCLUSIONS
The Extended Mathews method is essentially
self-validating. However, statistical investigations
of the method and the contained stability data have
been undertaken to improve the understanding of
the strengths and weaknesses of the model
framework. Applying regression techniques to a
sufficiently large stability database is currently the
best option for minimising the influence of
subjective data.
The placement and accuracy of the stability
zone boundaries on the stability graph can be
objectively determined using the modified logistic
regression. The removal of subjectivity in
delineating the stability zones has important
implications in comparing logit models and
stability boundaries.
Misclassification can be objectively assessed by
comparing the predicted logit stability with the true
stability outcomes. The proportion of misclassified
points provides a means of assessing and
comparing the accuracy of boundaries on the
Extended Mathews stability graph.

Paper 3A 07 SINOROCK2004 Symposium

Int. J. Rock Mech. Min. Sci. Vol. 41, No. 3, CD-ROM, 2004 Elsevier Ltd.

Logistic regression has allowed the statistical

comparison of logit models and stability
boundaries, which in turn has enabled investigation
of the underlying stability database to a previously
unachievable depth. The effect of the removal of
data on boundary position can now be
comprehensively and objectively evaluated through
analysis of the logit model and specifically by
comparing the predicted logit stability outcomes
with the true stability status.

5. ACKNOWLEDGEMENTS
The work reported in this paper is a component
of research funded by the Australian Research
Council and Stage 1 of the International Caving
Study. The author gratefully acknowledges the
support and assistance provided by Dr. W. Whiten
and Dr. R. Trueman of the Julius Krutschnitt
Mineral Research Centre at the University of
Queensland.

6. REFERENCES
Barton, N., Lien, R & Lunde, J., 1974. Engineering
classification of rock masses for the design of
tunnel support. Rock mechanics 6(4):189-236.

spans for mining at depths below 1000 metres

in hard rock. CANMET, Vancouver, Canada.
Mawdesley, C., 2002. PhD Thesis. Predicting rock
mass cavability in block caving mines. Julius
Kruttschnitt Mineral Research Centre. The
University of Queensland, Brisbane, Australia.
Mawdesley, C., Trueman, R. & Whiten, W., 2001.
Extending the Mathews stability graph for
open-stope design. Trans. Instn Min and Metall.
(Section A: Mining Industry) 110:A27-39.
Parker, R. A., & Davis, R. B., 1999. Evaluating
whether a binary decision rule operates better
than chance. Biometrical Journal 41(1):25-31.
Potvin, Y., 1988. PhD thesis. Empirical stope
design in Canada. Department of Mining and
Minerals Processing. University of British
Columbia, Canada.
Trueman, R., & Mawdesley, C., 2003. Predicting
cave initiation and propagation. CIM Bulletin
96(1071):54-59.
Trueman, R. T., Mikula, P., Mawdesley, C. A., &
Harries, N., 2000. Experience in Australia with
the application of the Mathews method for open
stope design. CIM Bulletin 93 (1036):162-167.

Bergerud, W.A., 1996. Biometrics Information

Handbook No. 7. Introduction to Logistic
regression models with worked forestry
examples. Ministry of Forests Research
Program. Working Paper 26/1996. 146p
Bieniawski, Z.T., 1989. Engineering rock mass
classifications. John Wiley, New York.
DeMaris, A., 1992. Logit modelling: Practical
applications. Sage University Paper series on
Quantitative Applications in the Social
Sciences. California; Sage: 87p.
Holtsberg, A. 1998. Logitfit.m, Lodds.m and
Loddsinv.m.
Available
online
at:
http://www.maths.lth.se/matstat/stixbox/content
s.html Internet: (sighted 1/02/00).
Liao, T.F., 1994. Interpreting Probability Models:
Logit, probit, and other generalised linear
models. Sage University Paper series on
Quantitative Applications in the Social
Sciences. California; Sage: 88p.
Mathews, K.E., Hoek, E., Wyllie, D.C. & Stewart.,
S.B.V., 1981. Prediction of stable excavation