Escolar Documentos
Profissional Documentos
Cultura Documentos
f
−1 intruder
>
: UNKNOWN
TRAINING EXAMPLES INPUT
( x1 , y1 ), ... , ( xN , yN ) DISTRIBUTION
- In-sample: N
P (x)
1 X
e h(xn), f (xn)
E (h) =
in
N n=1 - (x1, y1), · · · , (xN , yN ) generated by
Sponsored by Calte
h's Provost O
e, E&AS Division, and IST • Tuesday, April 17, 2012
Outline
• Illustrative examples
• Puzzle
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 2/20
The nal exam
Testing:
P [ |E − E | > ǫ ] ≤ 2 e
in out
−2ǫ2N
Training:
P [ |E − E | > ǫ ] ≤ 2M e
in out
−2ǫ2N
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 3/20
Where did the M
ome from?
P[B1 or B2 or · · · or BM ]
≤P [B 1 ] + P[B2] + · · · + P[BM ] B3
M terms
| {z }
no overlaps:
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 4/20
Can we improve on M?
up
+1
down
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 5/20
What
an we repla
e M with?
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 6/20
Di
hotomies: mini-hypotheses
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 7/20
The growth fun
tion
The growth fun tion ounts the most di hotomies on any N points
mH(N ) ≤ 2N
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 8/20
ements PSfrag repla
ements
N =3 N =3 N =4
mH(3) = 8 mH(4) = 14
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 9/20
Outline
• Illustrative examples
• Puzzle
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 10/20
0.04
Example 1: positive rays
0.06
h(x) = sign(x − a)
mH(N ) = N + 1
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 11/20
0.06 Example 2: positive intervals
N +1
mH(N ) = 2 +1 = 12 N 2 + 12 N + 1
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 12/20
Example 3:
onvex sets
2 up +
H is set of h : R → {−1, +1} −
+
h(x) = +1 is
onvex +
mH(N ) = 2N −
−
The N points are `shattered' by
onvex sets
−
+
− +
bottom
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 13/20
The 3 growth fun
tions
•H is positive rays:
mH(N ) = N + 1
•H is positive intervals:
mH(N ) = 12 N 2 + 12 N + 1
•H is onvex sets:
mH(N ) = 2N
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 14/20
Ba
k to the big pi
ture
P [ |E − E | > ǫ ] ≤ 2M e
in out
−2ǫ2N
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 15/20
Outline
• Illustrative examples
• Puzzle
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 16/20
1.5
2
Break point of H
2.5
Denition: 3.5
4
If no data set of size k
an be shattered by H,
0.5
then k is a break point for H
1
1.5
mH(k) < 2k
2
2.5
For 2D per
eptrons, k=4
3
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 17/20
Break point - the 3 examples
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 18/20
Main result
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 19/20
Puzzle
x1 x2 x3
◦ ◦ ◦
◦ ◦ •
◦ • ◦
• ◦ ◦
• ◦ •
AM
L Creator: Yaser Abu-Mostafa - LFD Le
ture 5 20/20