Escolar Documentos
Profissional Documentos
Cultura Documentos
Contents
Introduction
Discriminant functions(DF)
Linear discriminant
Fishers linear discriminant
Perceptron
Linear regression with indicator variables
Probabilistic approach
Discriminative model(DM) : Logistic regression
Generative model(GM) : LDA
Logistic regression vs LDA
Multiple classes
one-versus-the rest classifier
one-versus-one classifier
Spring, 2016
Introduction
Spring, 2016
Spring, 2016
Spring, 2016
Spring, 2016
Spring, 2016
Spring, 2016
Marvin Minsky
1927-
perceptron
Spring, 2016
4
10
Spring, 2016
Discriminant Function:
Linear regression with indicator variables
Linear regression with indicator variables
In the binary case, target label
, then perform linear regression
(with least squares error criterion) as before.
Class 1
There are strategies for extending this to multiple classes, but none of
them work very well.
Well get the same vector this way as from LDA, in the two-class
problem, as long as our class priors
. If the priors are
different, the direction will be the same, but the intercept will change.
EE531 Statistical Learning Theory
Spring, 2016
11
0.8
0.6
0.4
0.2
0
-10
-5
10
12
Spring, 2016
13
Spring, 2016
14
Spring, 2016
number of
mice
59
60
62
56
63
59
62
60
number
killed
6
13
18
28
52
53
61
60
P(killed|Dosage)
0.10
0.22
0.29
0.50
0.83
0.90
0.98
1.00
15
Spring, 2016
16
Spring, 2016
17
Spring, 2016
18
Spring, 2016
19
Spring, 2016
to class
if
Decision boundary
20
Spring, 2016
Summary
Introduction
Discriminant functions(DF)
Linear discriminant
Fishers linear discriminant
Perceptron
Linear regression with indicator variables
Probabilistic approach
Discriminative model (DM): Logistic regression
Generative model (GM): LDA
Logistic regression vs LDA
Multiple classes
one-versus-the rest classifier
one-versus-one classifier
21
Spring, 2016
22
Spring, 2016