The ROC Curve

Maria Hafeez
Theoretical Representation
ROC is Receiver Operating characteristics curve.
This figure presents the raw data obtained from
SDT(signal detection theory) experiment.
SDT (Signal Detection Theory) realizes that there
usually is no perfect scenario where you only have hits
and correct rejections. So, one usually analyzes the
probability of such occurrences.
As a result, we have 4 classes of joint events: HITS,
MISSES, FALSE ALARMS, and CORRECT
REJECTIONS.
Two curves are critical, normally P(H) and P(FA).
Theoretical Representation
This figure shoes the theoretical representation of the
neural mechanism within the brain that generated the
2X2 matrix. As the criterion is set at different
locations along x-axis, different set of values will b
generated in matrix.

Theoretical Representation
Next figure shows the ROC curve, plotting P(H) against P(FA) for different
settings of the response criterion.
Each signal detection condition generates one point on ROC.
IF signal strength and the observers sensitivity remains constant, changing
beta from one condition to another will produce a curved set of points.
Theoretical Representation
Points in the lower left represent conservative responding
Points in the upper right represent risky responding.
When connecting they form ROC curve.
Figure shows relationship between Raw data, ROC curve, and theoretical
distributions, collected for three different beta values.
P
[
X
|
(
N

o
r

S
)
]

Noise
Signal
Hit Correct
rejection
Miss False alarm
Theoretical Representation
It is difficult to carry out same signal detection experiment several times,
each time changing just by different pay off or probability of signal. Instead
of this if three confidence levels are employed, the data may b analyzed
twice in different ways. (e.g. 1= confidant no signal was present.2= uncertain,
3= confidant that signal was present)
Example of two types of analysis criteria is given in table:


Theoretical Representation
Formally the value of beta at any given point along the
ROC curve is equal to slope of the tangent drawn to
curve at that point.
If a Z transformation is applied to the ROC curve, the
curve will be transformed into a straight line. This z-
transformation is based on a normal distribution with a
mean of zero and a standard deviation of one.
Z-Scores tell us whether a particular score is equal to the
mean, below the mean or above the mean of a bunch of
scores.

Empirical Data
It is important to realize the distinction between the theoretical, idealized curves in the previous
diagrams and actual data collected in experiment or field investigation or detection performance. The
most obvious contrast is that the representation data in previous figures are continuous, smooth curves
whereas empirical data would consist of discrete points.
More important, empirical results have data that do not fall precisely along a line of constant
bowedness, often slope is slightly shallower. This is due to distributions of noise signal plus noise
energy is not precisely normal and of equal variance.
This tilting of ROC curve away from the ideal presents some difficulties for the use of d as a measure
of sensitivity.
Empirical Data
Discriminability index (d'): Thus, the discriminability of a signal depends both on
the separation and the spread of the noise-alone and signal-plus-noise curves. To
write down a complete description of how discriminable the signal is from no-
signal, we want a formula that captures both the separation and the spread. The
most widely used measure is called d-prime (d' ), and its formula is simply:
d' = separation / spread
This number, d', is an estimate of the strength of the signal. Its primary virtue, and
the reason that it is so widely used, is that its value does not depend upon the
criterion the subject is adopting, but instead it is a true measure of the internal
response.

Empirical Data
If there are only one or two points in the ROC space and there is evidence for strong
risky or conservative bias, another measure of sensitivity should be used.
In this situation measure P(A)or the area under ROC curve is an alternative measure
of sensitivity.
Advantage of this measure is that it is parameter free, i.e. its value does not depend
on any assumption concerning the shape or form of the underlying signal and noise
distributions.