ROBOT VISION

Lecture Notes











Course No. 15678
Credits Hours 3





Instructor:
Professor
Choi, Tae-Sun

Course outline:

The principles of the machine/robot vision are introduced. It covers image formation,
pattern classification, motion and optical effect for object recognition. In addition, the
design technology of the robot vision system with optical device is studied.

Prerequisite: Digital Signal Processing, Image Processing.

Textbook and References: 1. Robot Vision, B. K. P. Horn, MIT Press.
2. Computer Vision, Dana Ballard and Christopher
Brown, Prentice Hall.
Course Schedule
1. Image Formation & Image Sensing
2. Binary Images: Geometrical Properties
3. Binary Images: Topological Properties
4. Regions & Image Segmentation
5. Image Processing: Continuous Images
6. Image Processing: Discrete Images
7. Edge & Edge Finding
8. Lightness & Color
9. Reflectance Map: Photometric Stereo
10. Reflectance: Shape from Shading
11. Motion Field & Optical Flow
12. Photogrammetry & Stereo
13. Pattern Classification
14. Polyhedral Objects
15. Extended Gaussian Images
16. Passive Navigation & Structure from Motion

1
1. Introduction
I. Human vision system

Figure 11
Rod : more sensitive to light
Scotopic vision
Slender receptor

Cone : shorter & thicker in structure
Photopic vision : R.G.B
6.5 million

Fovea : Density of cones is greatest
The region of sharpest photopic vision Camera visual system

2

Figure 12
i. Photometric information
brightness : (ex: bright green, dark green)
color (ex, R. G. B)
ii. Geometric information
shape

Figure 13

2
NTSC : US standard for TV picture and audio coding and transmission
(timing calculation)
For one frame : 512512 = 256k Byte
For RGB : 3
For real time : 30 frames/sec
256k 3 30 = 22.5M Byte/sec
180 M bit/sec
180 Mbps T1 1.544 Mbps
Compression ratio :
180
120
1.544

Wavelet compression
MPEG I, II, IV
Compression JPEG
Fractal
H.261
II. Robot vision
RV
Computer graphics
Image processing
compressio
3

Figure 14


Figure 15

4
Scene description
Under constrained problems : Inverse problem
for example 2 equations
3 unknown variables are given
1. Image analysis :a detailed but undigested description
2. Scene analysis : more parsimonious, structured descriptions suitable for
decision marking


The apparent brightness of a surface depends on three factors
1. Microstructure
2. Distribution of incident light
3. Orientation of the surface w. r. t light source & observer


5

Figure 16
Lambertian Surface
One that appears equally bright from all viewing directions and reflects all incident light,
absorbing none

6
2. Image Formation & Image sensing
What determines where(Shape-Geometric) the image of some point will appear?
What determines how(Brightness, Color Photonetiz) bright the image of some surface
will be?

I. Two Aspects of Image Formation
i. Perspective projection

Figure 21



7
ii. Orthographic Projection
2 2
2 2
0
' ' '
(magnification)
x y f
m
z
x y


+
= =
+

In case the depth range of a scene is small relative to the distance z
0

Figure 22

\
|
=
=
my y
mx x
'
'
if m=1

\
|
=
=
y y
x x
'
'

=> This orthographic projection can be modeled by rays parallel to the optical axis
(rather than one passing through the origin)

(a) (b)
Figure 23


I
p
E

= [watt/m]
Where P is the power of the radiant energy falling on the
8

II. Brightness
1. Image Brightness : Irradiance
2
[ / ]
P
E watt m
I

=
2. Scene Brightness :
L=
2
2
[ / ]
P
L watt m
I W

= where
2
P is the power emitted by the infinitesimal surface patch
of area (I) into an infinitesimal solid angle (w)
2
, where is area, and is Distance
A
SolidAngle A D
D

Ex 1 Hemisphere
w
R
2
2
4 1
Solid angle = = 2
2
Whole sphere = 4
R
R

Figure 24
Ex 2 Small patch
9

Figure 25
Lens

Figure 26



cos

( / cos )2
Image Irradiance
P
E
I
I
Solid angle
f

=
=

0
2

=
'
cos

( / cos )
Scene Radiance
P
L
O w
Solid angle
z

=

Transfer Function: ( )
L
f
E

= i
10
f() : BRDF(Bidirectional Reflectance Distribution Function) in Ch 10
Ex3) Sphere/Radius 2m

Figure 27
III. Lenses


Figure 28

i. Lens formula
1 1 1
' f z z
= + 21
' '
2
s z z
R D

= 22
Solve (2-1) & (2-2) to calculate R blur circle radius
11
1 1 1
( )
2
DS
R
f s z
=
R = 0 at focused point P
ii. Depth of field (DOF)
The range of distances over which objects are focused sufficiently well in the sense
that thediameter of the blur circle is less than the resolution of the imaging device.
The larger the lens aperture => the less the DOF

Figure 29

iii. Vignetting
In a simple lens, all the rays that enter the front surface of the lens end up being focused in the image

Figure 210
12
In a compound lens, some of the rays that pass through the first lens may be occluded
by portions of the second lens, and so on Vignetting is a reduction in light-gathering
power with increasing inclination of light rays with respect to the optical axis




Figure 211
iv. Aberration
Points on the optical axis may be quite well focused, which those in a corner of the
image are smeared out
13
IV. Image sensing
i. Color sensing
a) Human eye

Figure 212
b) CCD sense
R, G, B filter
R() : 1 ( at Red color)
14
0 (otherwise)
G() : 1 (at Green color)
0 (otherwise)
B() : 1 (at Blue color)
0 (otherwise)
UV filter filtered out UV ray
ii. Randomness and Noise
a) Random variable [R.V]
Consider an experiment H with sample description space the elements on points of
,, are the random out comes of H
If to every , we assign a real number X() The real establish a correspondence rule
between and R the real time, such a rule subject to certain constraints is called a
random variable

15
Figure 213
R.V.
Probability density function (p.d.f) p(x)
Where p(x) 0 for all x, therefore: ( ) 1 p x dx

mean :
( ) xp x dx

: first moment
variance :
2 2 2
( ) ( ) ( ) : second moment x p x dx x p x dx


= =



Cumulative probability distribution (CDF) P(x)
( ) ( )
x
P x p x dt

Two R.V.

S V R
X X X
. .
2 1
+ =

P(X
1
) & P(X
2
) what P(x) = ?
Solution

16
Figure 214

Given x
2
, x
1
: x-x
2

1 1 1 1 2
( ) ( ) P x x P x x x =

Now, x
2
can take on a range of values P
2
(x
2
).(x
2
)
To find the prob. that x lies between x and x+x,
1 2 2 2 2
1 2 2 2 2
1 2
1 2
( ) ( ) ( )
( ) ( ) ( )
( ) ( )

P x x P x x xP x dx
P x P x x P x dx
P x t P t dt
P P

=
=
=
=

For multiple
RV's
:

=
=
N
i
i
x
N
x
1
1

Mean value :

=
N
N

Variance :
2 2
2
N
N N

=
Standard Deviation :
N

b) Gaussian (Normal)
p.d.f :
2
1
2
1
( )
2
x
p x e

| |

|
\
=

17


c) Poisson (m>0)
p.d.f : ( )
!
n
m
m
p k e
n

=
where n arrivals in a time interval T for some m
mean = m variance =
2


Ex Let X and Y be independent r.v.'s with ( ) ( )
x
x
p x e u x

= and
1
( ) [ ( 1) ( 1)]
2
y
p y u y u y = + and Let Z X Y + , what is the p.d.f of Z?

(a) (b)
Figure 215

18
( )
1
( ) ( ) * ( ) ( ) ( )
(a) 1 ; ( ) 0
1
(b) 1 1 ; ( )
2
(c) z 1 ; (
z x y x y
z
z
z y
z
z
p z p x p y p z y p y dy
z p z
z p z e dy
p z

= =
< =
< =

1
( )
1
1
)
2
z y
e dy

Figure 216
19
3. Binary images : Geometric properties
I. Binary Images

Figure 31
1. Area of the object 1 sin cos 0
sin cos
x y
x y


+ = + =

( , )
I
A b x y dxdy =

; zero-th Moment
2. Position of the object : center of area
1
( , ) x xb x y dxdy
A
=

: First moment
1
( , ) y b x y dxdy
A
=


3. Orientation of the object
(,)
2
( , )
I
E r b x y dxdy =

: second moment

20
II. Simple Geometrical Properties

Figure 32
2 2 2
0 0
2 2
2 2 2 2
( ) ( )
( ( sin cos )) ( ( cos sin ))
( ) 2 ( sin cos ) 2 ( cos sin )
r x x y y
x s y s
x y x y s x y s


= +
= + + +
= + + + + +

To find the shortest distance from (x, y) to (x
0
, y
0
) differentiate with respect to S and set
equal to zero
2
( )
0
d r
ds
=
sin cos y x S + =
2 2
0
2
(sin cos ) sin ( cos sin ) cos
sin sin sin cos
sin (sin cos )
x x x x y
x y
y



= + + +
= +
= +

) cos sin ( cos
0
+ = y x y y
2
0
2
0
2
) ( ) ( y y x x r + =

2
) cos sin ( + = y x

21

If, r=0 , ; 0 cos sin = + y x
; 0 cos sin ), , ( = + y x y x At

Finally,
2
( , ) E r b x y dxdy =



2
( sin cos ) ( , )
I
x y b x y dxdy = +


Differentiating w.r.t and setting the result to zero lead to
2( sin cos ) ( , ) 0
I
dE
x y b x y dxdy
d

= + =


0 ) cos sin ( = + y x A
Let ' ' x x x x x x = =
y y y = '


cos ' sin ' cos sin
0 ) cos ) ' ( sin ) ' (
y x y x
y y x x
= +
= +

and so

2
2 2
( ' sin ' cos ) ( , )
' sin ( , ) ( 2 ' ') cos sin
I
I I
x y b x y dxdy
x b x y dxdy x y

= +



2 2
2 2
( , ) ' cos ( , )
sin sin cos cos
I
b x y dxdy y b x y dxdy
a b c

+
= +


where,
22

2
2
( ') ( , ) ' '
2 ( ' ') ( , ) ' '
( ') ( , ) ' '
I
I
I
a x b x y dx dy
b x y b x y dx dy
c y b x y dx dy
=
=
=




Now
2 sin
2
1
2 cos ) (
2
1
) (
2
1
b c a c a E + =

Differentiating w.r.t and setting the result to zero, we have
c a
b

=
2
tan
unless 0 ; b a c = =

\
|
+ =

cos sin
tan
2
1
1
y x
c a
b


III. Projections

Figure 33
23

= = dx y x b x V dy y x b x h ) , ( ) ( , ) , ( ) (
Area :
( , ) ( )
( )
A b x y dxdy h x dx
v y dy
= =
=

Position :

=
= =
dy y yv
A
y
dx x xh
A
dxdy y x xb
A
x
) (
1
) (
1
) , (
1

Orientation :
c a
b

=
1
tan
2
1

( , ) cos sin y x + =
2 2
( , ) ( )
I
x b x y dxdy x h x dx =


2 2
( , ) ( )
I
y b x y dxdy y v x dy =



= dy y h y x v x dt t d t dxdy y x xyb
I
) (
2
1
) (
2
1
) ( ) , (
2 2 2


Diagonal projection (=45)

Figure 34
24
Suppose that is 45

+ = = ds s t s t b dxdy y x b t d ) (
2
1
), (
2
1
( ) , ( ) (

Now consider that
2 2
2 2
1
( ) ( , ) ( )
2
1 1
( ) ( , )
2 2
I I
I
x y b x y dxdy t d t dt
x xy y b x y dxdy
+ =
= + +



so,
2 2 2
1 1
( , ) ( ) ( ) ( )
2 2
I
xyb x y dxdy t d t dt x h x dx y v y dy =



IV. Discrete Binary Images
(reference of book figure 3-8)

area

= =
=
N
i
M
j
ij
b A
1 1

position ) , ( j i

=
=
i j
ij
i j
ij
jb
A
j
ib
A
i
1
1


orientation ( , )

i j
ij
i j
ij
i j
ij
ijb
b j
b i
2
2


25



V. Run-Length coding

Figure 35
Where r
ik
is the k-th run of the i-th line and the first run in each row is a run of zeros

2
.......... ,
6
1
2
1
4
1
2 2
i
i i
n
i
m
k
i
n
i
i k i
m
r r r r r A
i
+ + + = =

= = =

position (center of area) : ) , ( j i
26

= =
=
= =
=
n
i
j
n
i
i
k
m
k
i i
jv
A
j h i
A
i
r h
i
1 1
2
2
1
1
..........
1
,

j
v


1. Find the first horizontal differences of the image data


Figure 36
2. The first difference of the vertical projection can be computed from the
projection of the first
27
horizontal differences
count the number of circles subtracted by the number of triangles.
3. The vertical projection v
j
can be found by summing the result from left to right




Orientation ( , )
j
j i
j t
t i j
ij
j
j
i
i
v j h i d t ijb
v j
h i

=
2 2 2
2
2
2
1
2
1

Where, ) (
2
1
j i t + =
d
t
can be obtained in a way similar to that used to obtain the vertical projection
28
4. Binary Image : Topological properties
Jordan curve theorem
A simple closed curve separate the image into two simply connected regions

Figure 41
4 Connectedness only edge-adjacent cells are considered neighbors.

Figure 42


29
Ex 1

Figure 43
4 objects
` 2 backgrounds
No closed curve contradiction(by Jordan curve theorem)

8 connectedness Corner-adjacent cells are considered neighbors, too.

Figure 44
6 connectedness

30
Figure 45


Figure 46

Figure 47
i. A sequential Labeling Algorithm
Recursive Labeling
1. Choose a point where b
ij
=1 and assign a label to this point and into neighbors
31

Figure 48

2. Next, label all the neighbors of these neighbors one component will have
been labeled completely

3. Find new places to start a labeling operation whenever an unlabeled point is
found where
b
ij
=1
4. Try every cell in this scan

Sequential Labeling

32

Figure 49

1. if A is zero, there is nothing to do.

Figure 410
2. if A is one, if D has been labeled, simply copy that label and move on if not [if
one B orc is labeled, copy that label else, choose a new label for A]
(go to step 2)
33
II. Local counting and iterative modification
i. Local counting

Figure 411

Horizontal N
Vertical N Total Length = 2N
34

Figure 412
The overestimated rate : 1 : 2 2 : 2 = N N
Considering all slopes overestimated average ratio :
4

Overestimated average rate :

4


-Euler number-
1. No. of Bodies No. of Holes
2. No. of upstream convexities No. of upstream concavities

convexities(+1) concavities(-1)
Figure 413
35


Figure 414
Body hole
1. body - hole : E = 1- 2 = -1 2 - 0 = 2 1 - 0 = 1 1 - 1 = 0 1 - 0 = 1 1 - 0 = 1
2. convexity - concavity : 1- 2 = -1 2 0 = 2 1 0 = 1 1 1 = 0 1 0 = 1 1 - 0 = 1




ii. The additive set property

Figure 415
36

This permits to split an image up into smaller pieces and obtain an overall answer by
combining the
results of operations performed on the pieces

Figure 416
1 2 2 2 1
) ( ) ( ) ( ) ( ) ( ) ( e E d E c E b E a E e d c b a E + + + + =
[ ( ) ( ) ( ) ( )] E a b E b c E c d E d e + + +
1-0 = 1 2-0 =2 3-0 =3 2-0 =2
(1 2 2 2 1) (1 2 3 2) = 0 = + + + + + + +
right (body = 1, hole 1) 1-1 =0
convexities =2
1(body)
37
concavities =2

Ex1) hand

Figure 417
Body : 1 Hole : 0 1-0 =1
Convexities : 5 , Concavities : 4 5-4 =1
Euler number = 1


iii. Iterative modification
Instead of adding up the outputs of Local operators in the previous section, the new
binary image determined by the iterative method can be used as input to another cycle
of computation this process, called iterative modification is useful because it allows us
convexit
concavit
38
to incrementally changes an image that is difficult to process into one that might
succumb to the methods already discussed

Euler Differential E*
2. E* = - ( 1 1 = 0 ) + ( 1- 0 ) = 1
Part pattern Current pattern
2


3
39

4

5

* *
E E
new
=
40
5. Regions & image segmentation

I. 5.1 Thresholding Methods

Image segmentation : An image segmentation is the partition of an image into a set of
non-overlapping regions whose union is the entire image


Figure 51
Detection of discontinuities : points, lines, and edges

Figure 52
41

Figure 53

=
= + + + =
q
i
i i q q
z w z w z w z w R
1
2 2 1 1


A mark used for detecting isolated points

Figure 54
i. Point detection
The detection of isolated points in an image is straightforward
If |R| > T when T is threshold, its an isolated point
ii. Line detection
The next level of complexity involves the detection of lines in an image. Consider
the four masks, as shown below


42

Figure 55
If at a certain point in the image, |R
i
|>|R
j
| for all ji that point is said to be
more likely associated with a line in the direction of mask i, for example, if
|R
1
|>|R
j
| , for j=2,3,4 the line is more likely associated with a horizontal line.

iii. Edge detection
The idea underlying most edge detection technologies is the computation of a local
derivative operator

Figure 56
43
Gradient operator
(
(
(
(

=
y
f
x
f
f

2
2
|
|

\
|

+ |

\
|

=
y
f
x
f
f

Figure 57
Laplacian Operator
2
2
2
2
2
y
f
x
f
f

=
) ( 4
8 6 4 2 5
2
z z z z z f + + + = in discrete form

Figure 58

44
2
2
5 2 4 6 8
Cross Section of f
f = 4 (z ) z z z z

+ + +


II. Thresholding
Thresholding is one of the most important approaches to image segmentation

Figure 59
1 if ( , )
( , )
0 otherwise
f x y T
g x y
<
=

III. Region-oriented segmentation
i. Basic Formulation

Figure 510

45
Let R represent the entire image region and R
i
the partitioned region for n=1,2,n
(a) R Ri
n
i
=
=

1
(entire image region)
(b)
i
R is a connected region, i=1,2,n
(c)
i j
R R = for all i & j , i j

\
|

=
=

1 1
2 1
5 1
R R
R R
R R

(d) ) (
i
R P = True for i=1,2,.n

Figure 511
ii. Region Growing by Pixel Aggregation
Region Growing is a procedure that groups pixels or sub-regions into larger regions.
The simplest of these approaches is pixel aggregation, which starts with a set of seed
points and from these grows regions by appending to each seed
Point those neighboring pixels that have similar properties



46
Ex 1

Figure 512
iii. Region splitting and Merging
Spitting
Let R represent the entire image region and select a predicate P as discussed in the
previous section For a square image, one approach for segmenting R is to subdivide it
successively into smaller and smaller quadrant regions so that, for any region R
i
,
P(R
i
) = TRUE that is, if P(R
i
or R) = FALSE, divide the image into quadrants

Figure 513
Merging
If only splitting were used, the final partition likely would contain adjacent regions
With identical properties, this draw back may be remedied by allowing merging as well
as splitting two adjacent regions R
j
&R
k
are merged only if P(R
j
R
k
)=TRUE

Figure 514
47
The preceding discussion may be summarized by the following procedure in
which, at any step, we split into four disjointed quadrants any region R
i
where
P(R
i
)= FALSE

Figure 515
a) merge any adjacent regions R
j
& R
k
for which P(R
j
R
k
) =TRUE ;
b) stop when no further merging and splitting is possible
Ex 2

Figure 516
Merging FALSE R P
TRUE R R R P FALSE R P
FALSE R P FALSE R P
FALSE R P TRUE R P
TRUE R P FALSE R P
+
+ +
= +
= =
= =
) (
) ( ) (
) ( ) (
) ( ) (
) ( ) (
4
41 32 23 3
43 2
34 1
21



Figure 517
48
6. Image Processing : Continuous images
I. Linear, Shift-Invariant Systems
Spatial domain Point Spread Function (PSF)
Frequency domain Modulation Transfer Function (MTF)

Figure 61
Defocused image ( g ) is a processed version of the focused image ( f )
Double the brightness of ideal image and the defocused image. If the image system
moves, f & g moves the same amount.

Figure 62

Linearity : 1 2 1 2 ( , ) ( , ) ( , ) ( , ) f x y f x y g x y g x y + +
Shift : ( , ) ( , ) f x a y b g x a y b
49
II. Convolution
( , ) ( , ) ( , ) f x y h x y g x y
( , ) ( , ) ( , ) g x y f x y h d d
g f h



=
=


Linear
( ) ( ) 1( , ) 2 ( , ) ? f x y f h g x y + =
[ ] 1 2 ( , ) ( , ) ( , ) ( , ) g x y f x y f x y h d d


= +


1 2 ( , ) ( , ) g x y g x y = +
Shift invariant
( , ) ( , ) ( , ) f x a y b h x y g x y
( , ) ( , ) ( , ) g x y f x a y b h d d


=


Impulse response
( , ) ( , ) ( , ) f x y h x y h x y
( , ) ( , ) ( , ) g x y f x y h d d


=


and ( , ) ( , ) g x y h x y =

Only possible if
Origin
( , )
0 otherwise
f x y

=

This function is called unit impulse ( , ) x y or Dirac delta function
( , ) 1 x y dxdy


=



50
Shifting Property : ( , ) ( , ) (0, 0) x y h x y dxdy h


=


and ( , ) ( , ) ( , ) x y h d d h x y


=


=> ( , ) h x y : Impulse Response of the system

Figure 63
( , ) ( , ) k x y : scaled delta function

( , ) ( , ) ( , ) f x y f x y d d


=


( , ) ( , ) ( , ) g x y f x y h x y d d


=



b a = ( , ) ( , ) a x y b x y d d




<= Let , x = y =
= ( , ) ( , ) a b x y d d b a


=



Similarly, ( ) ( ) a b c a b c =


51
Cascade
1 1 2 ( )
1 2
f h f h h
f h h


=> 1 2 ( ) f h h

1 2 f h h g
III. MTF
An Eigen function of a system is a function that is reproduced with at most a change in
amplitude.
( )
jwt jwt
e A w e

2 ( )
( , )
i ux vy
f x y e
+
=

{ } 2 ( ) ( )
( , ) ( , )
i u x v y
g x y e h d d



+

=


=
2 ( ) 2 ( )
( , )
i ux vy i u v
e e h d d



+ +

=
2 ( )
( , )
i ux vy
e A u v
+


IV. Fourier Transform
Represent the signal as an infinite weighted sum of an infinite number of sinusoids (u: angular frequency)
( ) ( )
iux
F u f x e dx

Note:
cos sin 1
ik
e k i k i = + =
52
Arbitrary function => Single Analytic Expression
Spatial Domain(x)=> Frequency Domain(u) (Frequency Spectrum F(u))
Inverse Fourier Transform (IFT)
( ) ( )
1
2
iux
f x F u e du

( )
2
1
( , ) ( , )
4
i ux vy
f x y F u v e dudv

+ +

=



V. The Fourier Transform of Convolution
Let g f h =
Now,

( ) ( )
( ) ( )
( ) ( )
( )
( ) ( )
( ) ( )
2
2
2 2
2 2 '


' '

i ux
i ux
i u x i u
i u i ux
G u g x e dx
f h x e d dx
f e d h x e dx
f e d h x e dx
F u H u

=
=
(
( =

( ( =

=





Convolution in spatial domain Multiplication in frequency domain
Spatial Domain (x) Frequency Domain(u)
g f h G FH
g fh G F H
= =
= =


53
So, we can find g(x) by Fourier transform

Figure 64

Spatial Domain (x) Frequency Domain (u)
Linearity
( ) ( )
1 2
c f x c g x + ( ) ( )
1 2
c F u c G u +
Scaling
( ) f ax 1 u
F
a a
| |
|
\

Shifting
( )
0
f x x ( )
0
2 i ux
e F u


Symmetry
( ) F x ( ) f u
Conjugation
( ) f x

( ) F u

Convolution
( ) ( ) f x g x ( ) ( ) F u G u
Differentiation
( )
n
n
d f x
dx

( ) ( ) 2
n
i u F u



54
VI. Generalized Functions and Unit Impulses
The integral of the one-dimensional unit impulse is the unit step function,
( ) ( )
x
t dt u x

, where ( ) u x = 1, for 0; x
1
2
= , for 0; x =
0 = , for 0 x .
What is the Fourier transform of the unit impulse?
We have
( )
( , ) 1
i ux vy
x y e dxdy

+

=

, as can be seen by substituting 0 x = and 0 y =
into
( ) i ux vy
e
+
, using the sifting property of the unit impulse. Alternatively, we can use
( )
0
lim ( , )
i ux vy
x y e dxdy

, or
0
1 1
lim
2 2
iux ivy
e dx e dy

,
That is
0
sin sin
lim 1
u v
u v

= .
VII. Convergence Factors and the Unit Impulse
We want a smoothed function of f(x)
( ) ( ) ( ) g x f x h x =
Let us use a Gaussian kernel

Figure 65
55
( )
2
2
1 1
exp
2 2
x
h x

(
=
(


Then
( ) ( )
2
2
1
exp 2
2
H u u
(
=
(


( ) ( ) ( ) G u F u H u =
H(u) attenuates high frequencies in F(u) (Low-pass Filter)!
VIII. Partial Derivatives
What is the F.T. of
( , ) x y f
x

and
f
x

?
2 ( ) i ux vy
f
e dxdy
x

=
2 2 ux vy
f
e dx e dy
x




(

2 ux
f
e dx
x

=
2 2
( , ) ( 2 ) ( , )
iux iux
f x y e iu f x y e dx

=
2 2
( , ) (2 ) ( , )
iux iux
f x y e iu f x y e dx

We cant proceed unless, lim ( , ) 0
x
f x y

=
In that case,
2 ( )
2 ( , )
i ux vy
iu f x y e dxdy

+


=(2 ) ( , ) iu F u v

( , )
( , )
2 ( , )
2 ( , )
x y
x y
f
F iu F u v
x
f
F iv F u v
x

=
`

)

=
`

)

56
IX. Rotational Symmetry and Isotropic operators
Rotationally symmetric operators are particularly attractive because the treat
image features in the same way, no matter what their orientation is.
Circular symmetry

Figure 66
Let us introduce polar coordinates in both spatial and frequency domain

Figure 67
cos x r = and sin y r = cos( ) ux vy r + =
cos u = and sin v =

57
{ }
2 ( )
( , ) ( , )
i ux vy
F f x y f x y e dxdy

+

=


2 cos( )
0 0
( )
i r
r f r e rdrd

=


Zeroth order Vessel function
(Hankel Transform) => 0 ( ) 2 ( ) (2 ) r F f r J r rdr

0
0
( ) 2 ( ) (2 ) fr r f J r d

where
2
cos( )
0
0
1
( )
2
i x w
J x e d

X. Blurring

2 2
2
1
2
2
1
( , )
2
x y
h x y e

=
. . F T
2 2
2
1
2 ( ) 2
2
1
( , )
2
x y
i ux vy
H u v e e dxdy

+

=


2 2
1 1
2 2 2 2
1 1
2 2
x y
iux ivy
e e dx e e dy


| | | |

| |

\ \

=



2
1
2 2
1
2
x
iux
e e dx

| |

|

\

2 2
1 1
2 2
cos(2 ) sin(2 )
x x
e ux dx i e ux dx


| | | |

| |
\ \

=



58
The second integral is odd so, integral over symmetric region is zero.

And,
2
2 2
1
2
2
1
cos(2 ) ,
2
x u
a
e ux dx e a
a

| |
|
\

= =

2
2 2 2
1
1
(2 )
2
2
cos(2 ) 2
x
u
e ux dx e

| |

|
\

2 2 2 2 2 2
1 1
(2 ) (2 )
2 2
1 1
( , ) 2 2
2 2
u v
H u v e e




=



2 2 2 2
1
(2 ) ( )
2
u v
e
+
=
Which is also rotationally symmetric. { } F G a u s s i a n G a u s s i a n =

Figure 68: Low Pass Filtering
XI. Restoration and Enhancement
( , ) f x y ( , ) '( , ) ( , ) h x y h x y f x y
The cascade of the two systems is the identity system.
1
'( , ) min ,
( , )
H u v A
H u v
| |
=
|
|
\
,
Where, A is the maximum gain. Or more elegantly, we can use something like
59
2 2
( , )
'( , )
( , )
H u v
H u v
H u v B
=
+
,
where
1
(2 ) B
is the maximum gain, if ( , ) H u v is real.

XII. Correlation

Figure 69
Cross correlation of ( , ) a x y and ( , ) b x y is defined as
( , ) ( , )* ( , )
( , ) ( , )
ab x y a x y b x y
a x y b d d

=
=


If ( , ) a x y = ( , ) b x y , the result is called autocorrelation
( , ) .( ( , ) ( , )) ab ab ab x y Symmetric x y x y =
(0, 0) ( , ) aa aa x y for all ( , ) x y
2
(0, 0) ( , ) ( , ) ( , ) aa a a d d a d d


= =




60
XIII. 6.13 Optimal Filtering

Figure 610
E =
{ }
2
( , ) ( , ) o x y d x y dxdy

( , ) ( , ) ( , ) i x y b x y n x y = +
( , ) ( , ) ( , ) o x y i x y h x y =

So,
2 2
( , ) 2 ( , ) ( , ) ( , ) E o x y o x y d x y d x y dxdy


= +


{ }
2
2
( , ) ( , ) ( , ) o x y i x y h x y =
( , ) ( , ) ( , ) ( , ) i x y h d d i x y h d d


=



Therefore,
2
( , ) ( , ) ( , ) ( , ) o dxdy h h d d d d i x y i x y dxdy


=



x A x A
y B y B


= = +
= = +

61
( ( ), ( ) ( , ) i A B i A B dAdB


=


( , ) ii = (Autocorrelation)
62
7. Image Processing Discrete Image
I. Sampling Theorem
To convert a continuous distribution of brightness to a digital image, it necessary to
extract the brightness at each point arranged at a period. Thus is called sampling.

Assume an image as a one-dimensional function for simplicity.
Let the brightness at position x i.e. pixel value of pixel at x, be fix.

Figure 71

Figure 72
63
Suppose a function that is composed by infinite numbers of Diracs delta functions
assigned at an interval T. This is called comb function defined as follows.
A simple digital image from ( ) f x , denoted ( ) T f x , is expressed as ( ) f x multiplied by
the comb function ( ) T comb x , i.e.

( ) ( ) ( ) T T f x f x comb x =
{ } { } { } ( ) ( ) ( ) ( ) ( ) ( ) T T FT f x v FT f x v FT comb x v =
and, { }
1
1
( ) ( ) ( ) T
T
FT comb x v comb V
T
=
{ } { }
1
( ) ( ) ( ) ( ) T FT f x v FT f x v
T
=
1
( )
T
comb V

Thats a convolution of comb function?
We start from convolution with delta function
( ) ( ) ( ) ( ) ( ) ( ) ( ) f t t f y t y dy f t t t dy f t


= = =


i.e., convolution of a function and delta function is equal to original function itself.
Since a comb function is a sequence of the delta function arranged at constant period,
convolution of a function and comb function is an arrangement of duplication of the
original function at a constant period.

64
The F.T. of ( ) T f x , which is the sampled version of ( ) f x at period T is an infinite
sequence of duplicating of { } ( ) T FT f x which is the F.T. of the original function ( ) f x
at period
1
T
.

Figure 73
If the period of comb function in the frequency domain is sufficiently large, adjacent
{ } ( ) FT f x s do not overlap. In this case, the F.T. of the original function { } ( ) FT f x ,
can be separated and extracted i.e. no information of the brightness distribution of the
original image is lost by sampling. However, if the internal of the comb function is the
frequency domain is small, adjacent { } ( ) FT f x (v)s overlap. In this case, the original
{ } ( ) FT f x cant be separated and a faulty function will be extracted. This effect is
called aliasing.
65
{ } { } { } ( ) ( ) ( ) ( ) ( ) ( ) T T FT f x v FT f x v FT comb x v =

Figure 74
Since the support of { } ( ) FT f x is in the range between ~ c c V V , the period has to be at
least 2 c V to avoid overlapping of { } ( ) FT f x s since, T is the sampling period.
1
T

denote the number of sampling per unit length, i.e. sampling rate consequently, the
original brightness distribution can be reconstructed by a sampled digital image of the
sampling rate is more than twice the maximum frequency contained in the original
distribution. This theorem is called sampling theorem.



66
Discrete Fourier Transform (DFT)

1 1
2 ( )
2 ( )
0 0
0 0
1
( , ) ( , ) ( , )
m n
M N
i u v
i ux vy
M N
m n
F u v f x y e dxdy f m n e
MN

+
+
= =
= =

(u=0~M-1, v=0~N-1)
Inv. D.F.T.
1 1
2 ( )
0 0
( , )
m n
M N
i u v
M N
m n
F u v e

+ +
= =
=

(m=0~M-1, n=0~N-1)

f(m,n) Input Signal Amplitude (real or complex) at sample (m, n)
T Sampling interval
s f Sampling rate
1
( )
T
= , samples/second
M,N = Numbers of Spatial samples = Number of frequency samples (integer)

Magnitude
1
2 2
2
( , ) ( , ) ( , ) F u v R u v I u v ( = +


Phase Angle
1
( , )
( , ) tan
( , )
I u v
u v
R u v

=
Power Spectrum
2
2 2
( , ) ( , ) ( , ) ( , ) P u v F u v R u v I u v = = +


1 1
0 0
1
(0, 0) ( , )
M N
m n
F f m n
MN

= =
=

Average of ( , ) f m n or D.C. component.
If ( , ) f x y is real, its F.T. is conjugate complex
( , ) ( , )
( , ) ( , )
F u v F u v
F u v F u v

=
=
Spectrum of D.F.T. is symmetric.

67
Relationship between samples in the spatial and frequency domains.

1
u
M x
=


1
v
N y
=



Translation
0 0
0 0
2 ( )
( , ) ( , )
u v
i x y
M N
f x y e F u u v v
+

0 0
0 0
2 ( )
( , ) ( , )
ux vy
i
M N
f x x y y F u v e
+


Case : when 0 0 ,
2 2
M N
u v = =
{ }
2 ( )
( )
2 2
cos ( ) ( 1)
x y
i
i x y x y
e e x y

+
+ +
= = + =
( , )( 1) ( , )
2 2
x y
M N
f x y F u v
+


And similarly,
( , ) ( , )( 1)
2 2
u v
M N
f x y F u v
+


Rotation
cos
sin
x r
y r

=
=

cos
sin
u w
v w

=
=


Then, ( , ) f x y and ( , ) F u v become ( , ) f r and ( , ) F w
68
0 0 ( , ) ( , ) f r F w + +

Rotating ( , ) f x y by an angle 0 rotates ( , ) F u v by the same angle, and vice versa.

Periodicity ( , ) ( , ) ( , ) ( , ) F u v F u M v F u v N F u M v N = + = + = + +
The inverse transform is also periodic.
( , ) ( , ) ( , ) ( , ) f x y f x M y f x y N f x M y N = + = + = + +

Separability
1 1
2 2
0 0
1 1
( , )
y x
M M
j v j u
N M
x y
F u v e e
M N



= =
=


1
2
0
1
( , )
x
M
j u
M
x
F u v e
M

=
=

, (where
1
2
0
1
( , )
y
M
j v
N
y
F u v e
N

=
=

)
( , ) ( , ) ( , ) f x y F x v F u v


Convolution
1 1
0 0
1
( , ) ( , ) ( , ) ( , )
M N
m n
f x y h x y f m n h x m y n
MN

= =
=

f h F H
f h F H


Pr ( . .) ove H W
1-D
Row transform
1-D
column transform
69
8. Edge & Edge detection
I. Edges in Images
An Edge is a boundary between two figures with relatively distinct grey-level
properties.
Edges are curves in the image where rapid changes occur in brightness or in the
spatial derivatives of brightness.
Change of brightness can be occurred at
Change of surface orientation
Different surfaces
One object occludes another
A boundary between light and shadow falling on a single surface
II. Differential Operators

Figure 81
70

Figure 82
sin cos 0 x y + =
( ) ( )
1 2 1
E , B B -B u( sin cos ) x y x y = + +
If t > 0 B
1
t < 0 B
2

t = 0 (B
1
+ B
2
)
( ) ( )
( ) 1
t
u t x dx
x dx

=
=

Figure 83
) cos sin ( ) ( cos
) cos sin ( ) ( sin
2 1
2 1


+ =

+ =

y x B B
y
E
y x B B
x
E


71
squared gradient
2
2 1
2
2
)) cos sin ( ) (( + =
|
|

\
|

+ |

\
|

y x B B
y
E
x
E

Laplacian of E(x,y)
2 2
2
2 2
2
2
1 2 2
2
2
1 2 2
2 '
1 2
sin ( ) '( sin cos )
cos ( ) '( sin cos )
( ) ( sin cos )
xx
yy
E E
E
x y
E
E B B x y
x
E
E B B a y
y
E B B x y




= +

= = +

= = +

= +


III. Discrete Approximations
Discrete form of Laplacian
E
x
= E( i+1 , j ) E( i , j )
E
y
= E( i , j+1 ) E( i , j )

E
xx
= E( i 1 , j ) 2E( i , j ) + E( i + 1 , j )
E
xx
+ E
yy
= (E
i-1 j
+ E
ij-1
+ E
i-1 j
+ E
i j+1
) 4E
ij



72
IV. 8.4 Local operators and noise
bb
nn
H

+
=
1
1
2 '
-------------

2
= u
2
+ v
2


suppose that
2
2
2
& N
S
nn bb
= =

Then
2 2 2
2 2
'
N S
S
H

+
= -
at low frequency -
2
: Laplacian operation
at high frequency -
2
2
N
S
<< small gain Noise can be reduced !
Derivation of

Figure 84
''
' '' 2

1 1
( )
1
( )
i bb nb
ii bb bn nb nn
bb
bb nn
nn
bb
H
SNR
H H

+
= =
+ + +
=
+
= =
+
=

73
For example, if the optimal filter is a gaussian

Figure 85
2
2 2
2
2
2
1
) , (

y x
e y x h
+

=
) (
2
1
2 2 2
) 0 , (
v u
e u H
+
=

74
9. Lightness & color

Figure 91
b(x,y) = e(x,y) - r(x,y)
Note that r(x,y) is constant within a patch and sharp discontinuities at edges
between patches and e(x,y) varies smoothly

Figure 92
Task : separate r(x,y) & e(x,y)
Take the logarithm of image brightness b(x,y)
log b(x,y) = log r(x,y) +log e(x,y)
b(x,y) = r(x,y) +e(x,y)

2
b(x,y) =
2
r(x,y) +
2
e(x,y)

Figure 93
75
cut slow slope by a threshold

Figure 94
T(
2
b(x,y) =
2
r(x,y)
|
2
b(x,y)| > T then t(x,y) =
2
b(x,y) else t(x,y) = 0
recover r(x,y)

Figure 95

=
=
dxdy y x r y x l
y x t y x l
) , ( ) , ( (
) , ( ) , (
2
2

Fourier Transform :
) (
1
) (
) ( ) (
)] , ( [ )] , ( [
2
2
2
e T L
T L
y x t y x l

=
=
=

2
1
) ( ..
) ( ) (

=
=
p G where
T G

+ = ) log(
2
1
) ( r r g
IFT : I(x,y) = g(r) * t(x,y)
76

Figure 96
I(x,y) = r(x,y) = reflectance
e(x,y) = b(x,y) I(x,y)


Figure 97



77
. . .
) , ( ) , ( ) , (
) , ( ) , ( ) , (
5 5 5
4 4 4
3 3 3
2 2 2
1 1 1
e r b
e r b
e r b
y x e y x r y x b
y x e y x r y x b
=
=
=
=
=

) (
74
37
:
2 1
2 2
1 1
2
1
1
e e
e r
e r
b
b
R = = - equation 9.1

52
26
2
1
=
r
r
- equation 9.2
3
2
3
2
5
2
5
2
2
95
38
112
28
:
r
r
b
b
r
r
b
b
R
= =
= =


Least square error 11 variables 16 equations
0 reflectance 1
16
equations
78
10. Reflectance Map : photometric stereo
(using one camera)

I. Radiometry
irradiance
A
p
E

= geometric stereo (using one camera)

radiance
W A
p
L

2
=

solid angle = Area of a spherical surface patch / (radius of the sphere)
2


Figure 101
2
2
1
4
2
Solid angle 2
R
R

= =

Figure 102
79

Figure 103

2 2
1
) 1 , , (

q p
q p
n
+ +

=

II. Image Formation

Figure 104
80
solid angle =
2
) cos / (
cos

z
O

solid angle =
2
) cos / (
cos

S
SI


2
cos
cos
|

\
|
=
S
z
I
O

--------- equation 10.2.1

The solid angle subtended by the lens

3
2
2
2
cos
4 ) cos / (
cos
2
|

\
|
=
|

\
|
=
z z


Thus the power of the light originating on the patch and passing through the lens is
cos = O L P
where L is the radiance of the surface in the direction toward the Lens

cos cos
4
cos cos
4
3
2
3
2
|

\
|
= =
|

\
|
=
z I
O
L
I
P
E
z
O L p

where E is the irradiance of the image at the patch under consideration

substituting for I O , we finally obtain
81

4
2
4
2
cos
4
cos
4
|

\
|
=

|

\
|
=
s
L E
L E
s
L E


III. Bidirectional reflectance distribution function (BRDF)

Figure 105

Figure 106
82
) , (
i i
E ; irradiance
The amount of light falling on the surface from the direction ) , (
i i

) , (
e e L
: radiance

The brightness of the surface as seen from the direction
) , (
e e

BRDF :
1
cos sin ) , ; , (
) , (
) , (
) , ; , (
2
0
=
=

e e e e e e i i
i i
e e
e e i i
d d
E
L
f


IV. Extended Light sources

Figure 107
solid angle w
2
) tan (
.
ce dis
area The
= The area =
i i
sin
83
(proof :
2
0
sin 2
i i i
sw

= =


) , (
i i
E : Radiance per unit solid angle coming from the direction ) , (
i i

The radiance from the patch under consideration

i i i i i i i
E E sin ) , ( ) , ( =
Total irradiance of the surface is
2
0
0
( , ) sin cos
i i i i i i
E E d d

=


where
i
cos accounts for the foreshortening of the surface as seen from the direction
) , (
i i

The radiance of the surface,

= ) , ( ) , , ( ) , (
i i e e i i e e
E j f L

i i i i
d d cos sin

V. Surface Reflectance properties
Ideal Lambertian surface
One that appears equally bright from all viewing directions and reflects all
incident light, absorbing none
84
BRDF ) , , (
e e i i
j f constant

2
0
2
0
( , , , ) sin cos 1
2 sin cos 1
1
1
i i
e e e e e e
e e e
f j d d
f d
f
f

=
=
=
=

sine BRDF is constant for a Lambertian surface
)
1
(
1
cos sin ) , (


= = =
=

f E E f
d d E f L
o o
i i i i i i

where the irradiance is E
o

Ideal specular Reflector

Figure 108

85
2
0
( , ; , ) sin cos 1
sin cos 1
1
sin cos
i i e e e e e e
i i
i i
f d d
k
k

=
=
=



In this case,
i i
i e i e e e i i
j f


cos sin
) ( ) ( ) , , ( =


Determine the radiance of a specular reflecting surface under an extended source where
the irradiance is E
o

2
0
( ) ( )
( , )
sin cos
( , ) sin cos
( , )
( , )
e i e i
e e
i i
i i i i i i
e e
i i
L
E d d
E
E

=

=
=



VI. Surface Brightness

Figure 109
86
i
E L

cos
1
=

Light source : a sky of uniform radiance E
2
0
2
0
2
0
1
sin cos
sin cos
cos 2
[1 1]
2 2
i i i i
i i i i
i
L E d d
E
d d
E
E E

=
=
(
= = + =
(




The radiance of the patch The radiance of the source

VII. Surface orientation

Figure 1010

87
T
y y y
T
x x x
y x
q r
p r
q p r r n
) , , 0 (
) , 0 , (
) 1 , , (


=
=
= =


The unit surface normal n
2 2
1
) 1 , , (

q p
q p
n
n
n
T
+ +

= =

Figure 1011

T
v ) 1 , 0 , 0 (
assuming is very small

2 2 2 2
2 2
1
1
) 1 , 0 , 0 (
1
) 1 , , (
cos
1
) 1 , , (

q p q p
q p
v n
q p
q p
n
T
+ +
=
+ +

= =
+ +

=

88
VIII. The reflectance Map

Figure 1012
) , (
) , (
2 2
1 1
q p R s
q p R s

i
E L

cos
1
= for 0
i


2 2 2 2 2 2 2 2
1 1
1
1
) 1 , , (
1
) 1 , , (
cos
s x
s s
s s
T
s s
i
q p q p
q q p p
q p
q p
q p
q p
s n
+ + + +
+ +
=
+ +

+ +

= =
= R(p,q) ; Reflectance Map
c
q p q p
q q p p
q p R
s s
s s
=
+ + + +
+ +
=
2 2 2 2
1 1
1
) , ( - equation 10.9.1

Figure 1013
89
2 2 2
1
2
1
2 2
2
2 2 2
1
2
1
1 1
1
1 1
1
) , (
1 1
1
) , (
s s
s s
s s
s s
q p q p
q q p p
q p R
q p q p
q q p p
q p R
+ + + +
+ +
=
+ + + +
+ +
=


IX. Shading in image

Consider a smoothly curved object the image of such an object will have spatial
variations in brightness due to the fact that surface patches with different orientations
appear with different brightness the variation of brightness is called shading

) , ( cos
cos
4
4
2
q p R L E
f
d
L E
i

|
|

\
|
=

after normalizing E(x,y) = R(p,q)
: reflectance factor where 1 0

n s y x E where
n s p y x E
) , ( , 1
) ( ) , (
= =
=

Ex1) Consider a sphere with a Lambertian surface illuminated by a point source at
essentially the same place as the viewer in the case of cylinder
90

Figure 1014

In this case,
2 2
1
) 1 , , (

) 1 , 0 , 0 (
) 1 , 0 , 0 (
q p
q p
n
v
s
+ +

=
=
=

From equation 10.9.1 - s n q p R ) , ( =
2 2
2 2 2 2
1
1
1 1
1
q p
q P q p
q q p p
s s
s s
+ +
=
+ + + +
+ +
=

0
2
0
0
2
1
2 2 2
) (
) 2 ( )) ( (
2
1
z z
z z y
d
d
q
z z
x
x y x r
d
d
p
y
z
x
z

= =

= + = =

Finally, E(x,y) = R(p,q)
91
2
2 2
2
0
2 2
2 2
1
) (
1
1
1
1
r
y x
z z
y x
q p
+
=

+
+
=
+ +
=

( x , y ) = ( 0 , 0 )
E( x, y) maximum = 0
x
2
+y
2
=r
2

E(x,y) minimum = 0

X. Photometric stereo

Figure 1015

92
2
2
2
2
2 2
2 2
2 2
2
1
2
1
2 2
1 1
1 1
1 1
1
) , (
1 1
1
) , (
s s
s s
s s
s s
q P q p
q q p p
q p R E
q P q p
q q p p
q p R E
+ + + +
+ +
= =
+ + + +
+ +
= =


Ex2) R(p, q) : linear and independent
2
2 2
2
1
1 1
1
1
) , (
1
) , (
r
q q p p
q p R
r
q q p p
q p R
+ +
=
+ +
=

then
2 1 2 1
1 2
2
2 2 1
2
1
) 1 ( ) 1 (
p q q p
q r E q r E
p

=

2 1 2 1
2 1
2
1 1 1
2
2
) 1 ( ) 1 (
p q q p
p r E p r E
q

=

Ex3) R(p,q) : nonlinear

Figure 1016
93
XI. Albedo : reflectance factor :
) ( n s E
i i
=
where
2 2
1
) 1 , , (

i i
T
i i
i
q p
q p
s
+ +

= for i=1, 2, 3,

2 2
1
) 1 , , (

q p
q p
n
T
+ +

=

Figure 1017

Unknown variables p, q,
At least 3 equations



94
11. Reflectance Map : shape from shading
I. Recovering Shape from Shading

Figure 111
i. Picture
One more information => strong constraint smoothness neighboring patches of the
surface => similar orientation assumption of smoothness : strong constraint

ii. Linear reflectance map

Figure 112
95
2 2
0
2 2
0 0
sin cos tan
b a
b
b a
a
a
b
+
=
+
= =

Figure 113

: sin cos ) ( q p m + = directional derivative
)) , ( (
1
sin cos ) (
1
2 2 2 2
0 0 0
y x E f
b a b a
bq ap
q m

+
=
+
+
=
+ =


S can be recovered from E(x,y) using above curve

Figure 114
96
m z = (small change a long the characteristic curve)
)) , ( (
1
1
2 2
y x E f
b a
z
m

+
= =

sin ) (
cos ) (
0
0
+ =
+ =
y y
x x where

0
0
1
0
2 2
0
( ) ( )
1
( ( , ))
z z m d
z f E x y d
a b

= +
= +
+

iii. Rotationally symmetric reflectance maps

Figure 115


97
2 2
2 2
sin
cos
tan
q p
q
q p
p
p
q
s
s
s
+
=
+
=
=

The slope
s s s
q p m sin cos ) ( + =

2 2
q p + =
)) , ( (
1
y x E f

=
The change in Z



2 2
2 2
1 2 2
sin
cos
)) , ( (
q p
q
y
q p
p
x
y x E f q p m z
s
s
+
= =
+
= =
+ = =


To simplify z , we could take a step of length
2 2
1
q p +
rather than

)) , ( (
) (
1
2 2
2 2 2 2
y x E f
q p
q p q p z

=
+ =
+ + =

x = p - equation 11.1.1
y = q

98
We need to determine p&q at the new point in order to continue the solution
y x t r w y x E ate Differenti
q p f y x E
& , , ) , (
) ( ) , (
2 2
+ =

2 2
'
) (
q p S where
x
S
S f
x
S
S
E
x
E
E
x
+ =

=

y
z
q
x
z
p
x
q
q
x
p
p
x
s

, , 2 2

y x
z
q
x
z
p

=
2
2
2
2 2

2 2
2
2( ) '( )
2( ) '( )
x
y
E pr qs f s
E ps qt f s
p p z z
p x x y
q y x x y

= + |

= +
\

= + = +

- equation 11.1.2
y s x r + =
y t x s q + =

in our case q y p x = &


+ =
+ =
) (
) (
qt ps q
qs pr p

From (2)


' 2
' 2
f
E
q
f
E
p
y
x
=
=


As 0 , we obtain the differential equations
99

\
|
= =
= + = = =

' 2
,
' 2
)) , ( ( , ,
1 2 2
f
E
q
f
E
p
y x E f q p z q y p x
y
x




(x, y, z, p & q) solvable
5 Differential Eqs
one more time by differentiating
)) , ( (
' 2
,
' 2
. . , ,
1
y x E f z
f
E
q y
f
E
p x
t r w q y p x
y
x

=
= = = =
= =

iv. General Case

R(p,q) = f(ap + bq)
R(p,q) = f(P
2
+ q
2
)
R(p,q) = arbitarary


y s x r
y
y x
z
x
x
z
p
y q x p z



+ =

=
+ =
2
2
2
- equation 11.1.3

y t x s
y
y
z
x
y x
z
q


+ =

=
2
2 2


100
|
|

\
|
=
|
|

\
|
|
|

\
|
=
|
|

\
|
y
x
H
y
x
t s
s r
q
p

where H is the Hessian Matrix
2
2
2
2
y
z
x
z
t r

= + : Laplacian
Differentiating E(x,y) w.r.t x & y

x
q
q
E
x
p
p
E
x
E
E
x

=

S R r R
S
y x
z
R
x
z
R
q p
q p
+ =

=
2
2
2

t R S R E
q p y
+ =
|
|

\
|
=
|
|

\
|
q
p
y
x
R
R
H
E
E
- equation 11.1.4
from equation 11.1.3 & equation 11.1.4
Let

|
|

\
|
=
|
|

\
|
q
p
R
R
y
x


q
p
R y
R x
=
=

Then

|
|

\
|
=
|
|

\
|
=
|
|

\
|
=
|
|

\
|
y
x
q
p
E
E
R
R
H
y
x
H
q
p

101

\
|
= =
+ = + = + =
+ = =
=
y x
q p
q p q
p
E q E p
qR pR y q x p z
y
q
x
p
z
qR pR z R y
R x

,
,

5unknown variables : x,y,z, p & q

Figure 116

II. 11.3 Singular points

Figure 117
) , ( ) , (
0 0
q p R q p R < for all ) , ( ) , (
0 0
q p q p

Figure 118
102
0 , 0
0 , 0
= = = =
= =
q p
q p
R y R x
R R


) (
2
1
) , (
2 2
q p q p R + = - equation 11.3.1
(p,q) = (0,0) singular point
) 2 (
2
1
2 2
0
cy bxy ax z z + + + =

Thus by ax
dx
dz
p + = =
& cy bx
dy
dz
q + = =
substituting p & q in equation 11.3.1
] ) ( ) [(
2
1
) (
2
1
) , (
2 2 2 2
cy bx by ac q p q p R + + + = + =

2 2 2 2 2 2
) (
2
1
) ( ) (
2
1
y c b bxy c a x b a + + + + + =

The brightness gradient is
y c b bx c a E
by c a x b a E
y
x
) ( ) (
) ( ) (
2 2
2 2
+ + + =
+ + + =


Differentiating again,
(
(
(
(

+ =
+ =
+ =
2 2
2 2
) (
c b E
b c a E
b a E
yy
xy
xx
3 equations, 3 unknowns
103
III. Stereo graphic projection

Figure 119

= +
+
=
+
=
+
2 2 2
2 2
2 2
) (
2
R b a
R
q p
a
b
R
q f
a R
b

solve the above equations

2 2 2 2
1 1
2
,
1 1
2
q p
q
g
q p
p
f
+ + +
=
+ + +
=
conversely
2 2 2 2
4
4
,
4
4
g f
g
q
g f
f
p

=

=


104
IV. 11.7 Relaxation methods
characteristic strip expansion


E(x , y) = R(p , q)
= R
s
(f,g)
Minimize


+ + + = dxdy gy gx fy f e
s
) ( ) (
2 2 2 2

A low, Minimize


= dxdy g f R y x E e
s i
2
)) , ( ) , ( (
overall, minimize
i s
e e +
if brightness measurement are very accurate,
is large
if brightness measurement are very noisy,
is small

105
minimize

dxdy g g f f g f F
y x y x I
) , , , , , (
= Its a problem in the calculus of variations.(see appendix)
The corresponding Euler equations are


0
0
=

y x
y x
g g g
f f f
F
y
F
x
F
F
y
F
x
F

where F
f
is the partial derivative of F with respect to F
2 2 2 2 2
)) , ( ) , ( ( ) ( ) ( g f R y x E g g f f F
s y x y x
+ + + + =
The Euler equations for this problem yield

yy f
xx x f
s
s f
f F
y
f f
x
F
x
f
R
g f R y x E F
y
x
2
2 ) 2 (
)) , ( ) , ( ( 2
=

=


f f f yy xx
F F
y
F
x
f f
y x
=

= + ) ( 2

f
R
g f R y x E
s
s

= )) , ( ) , ( (

g
R
g f R y x E g
s
s

= )) , ( ) , ( (
2
(2 equations, 2 unknowns(f,g))
Application to photometric stereo consider n images,
106




=
+ + + =
n
i
i i I i y x y I
dxdy g f R y x E dxdy g g f fx e
1
2 2 2 2 2
)) , ( ) , ( ( )) ( ) ((
where E
i
(x,y) is the brightness measured in the i th image and R
i
is the corresponding
reflectance map
The Euler equations for this problem yield

=
=

=
n
i
i
i i i
i
i i
n
i
i
g
R
g f R y x E g
f
R
g f R y x E f
1
2
1
2
)) , ( ) , ( (
)) , ( ) , ( (

V. 11.8 Recovering Depth from a needle diagram

Figure 1110



107

Figure 1111

0 0
( , )
0 0
( , )
( , ) ( , ) ( )
x y
x y
z x y z x y pdx qdy = + +

Minimize the error
dxdy p z p z
I
p x
) ) ( ) ((
2 2
+


where p and q are the given estimates of the components of the gradient while
y x
z z &
are the partial derivatives of the best-fit surface
Minimize

dxdy z z z F
y x I
) , , (

The Euler equations is
0 =

y x
z z z
F
y
F
x
F

108

so that from
2 2
) ( ) ( q z p z F
y x
+ =
0
) ( 2 ) ( 2
=

=
q z
y
p z
x
F
y
F
x
F
y x
z z z
y x

) (
0 ) ( 2 ) ( 2
2
y x
y x yy xx
y yy x xx
q p z
q p z z
q z p z
+ =
+ = +
= +


109
12. Motion field & Optical flow

Figure 121

Motion field : a purely geometric concept
Optical flow : Motion of brightness patterns
observed when a camera is moving relative to the objects being imaged

I. Motion Field

Figure 122
110

0
0
0
0
, &
'
i
r r
i
t t
i
v v
r r
f r z


= =
=

- equation 12.1.1

Differentiate - equation 12.1.1
2
0
0 0 0 0
2
0
0
0 0
0
) (
) ( ) (
) (
) ( ) (
'
1
z r
r z v v z r
z r
r z
dt
dr
dt
dr
z r
dt
r d
f
i

=

2
0
0 0
) (
) (
'
1
z r
z v r
v
f
i

= - equation 12.1.2

II. Optical flow

Figure 123
111
Ex1)

Figure 124

E(x, y, t) E(x + ut , y + vt , t + t) where u(x, y) & v(x, y) are the x & y components
of the optical flow vector

For a small time interval t , if brightness varies smoothly with x, y, &t,
) , , ( ) , , ( ) , , ( t y x E
t
E
t
y
E
y
x
E
x t y x E t t t v y t u x E + + + + = + + +

where contains second and higher order terms in x , y , and t

0 =

t
t
t
E
t
y
y
E
t
x
x
E

0 = + +
t y x
E v E u E (Optical flow constraint equation)
u, v : two variables
one equation :
one more constraint needed

III. Smoothness of the optical flow
Usually, the motion field varies smoothly minimize a measure of departure from
smoothness
112

+ + + = dxdy vy vx uy ux e
s
)) ( ) ((
2 2 2 2

Also, minimize the error in the optical flow constraint eq.

+ + = =
+ +
y dxd E v E u E dxdy t y x E t y x E e
t y x att t at c
2 2
) ( ] ) , , ( ) , , ( [

Hence, minimize
c s
e e + ,
Minimize
c s
e e +
Where

+ + + = dxdy vy vx uy ux e
s
)) ( ) ((
2 2 2 2



+ + = dxdy E v E u E e
t y x c
2
) (
: a parameter that weights the error in the image motion equation relative to the
departure
from smoothness
=> Large
=> small
Minimize

dxdy v v u u v u F
y x y x
) , , , , , (

2 2 2 2 2
) ( ) ( ) (
t y x y x Y X
E v E u E v v u u F + + + + + + =
Euler equations are

0
0
=

y x
y x
v v v
u u u
F
y
F
x
F
F
y
F
x
F


113
yy y u
xx x u
x t y x u
u u
x
F
y
u u
x
F
x
E E v E u E F
y
x
2 ) 2 (
2 ) 2 (
) ( 2
=

+ + =


u u u E E v E u E
yy xx x t y x
2
) ( = + = + +
Similarly,
0
) (
2
= + +
= + +
t y x
y t y x
E v E u E
v E E v E u E


IV. Boundary conditions
*Natural boundary conditions (see appendix)
ds
dx
F
ds
dy
F
ds
dx
F
ds
dy
F
y x
y x
v v
u u
=
=

where S denotes arc length along the boundary curve
T
ds
dx
ds
dy
n |

\
|
= , is a unit vector
perpendicular to the boundary
Rewriting the above conditions.
0 ) , (
0 ) , (
=
=
n F F
n F F
y x
y x
v v
u u

114
in our case,
2 2 2 2 2
) ( ) ( ) (
t y x y x y x
E v E u E v v u u F + + + + + + =
0 ) , (
0 ) , (
=
=
n v v
n u u
y x
y x


V. Discrete case
From
c s
e e e + =
dxdy E v E u E dxdy v v u u
t y x y x y x
2 2 2 2 2
) ( )) ( ) (( + + + + + + =


) (
ij
i j
ij
c s e + =



) ) ( ) ( ) ( ) ((
4
1
2
, 1 ,
2
, , 1
2
, 1 ,
2
, , 1 , j i j i j i j i j i j i j i j i j i
v v v v u u u u s + + + =
+ + + +

2
) (
t ij y ij x ij
E v E u E c + + =
Differentiating w.r.t
kl kl
v u & yields

,
) ( 2 ) ( 2
, ) ( 2 ) ( 2
y t kl y kl x kl kl
kl
x t kl y kl x kl kl
kl
E E v E u E v v
v
e
E E v E u E u u
u
e
+ + + =

+ + + =

where v u & are average of u & v

115
0
0
=

kl
kl
v
e
u
e
solve this

t y kl x kl x y kl y x
t x kl y x kl y kl y x
E E v E u E E v E E
E E v E E u E u E E


+ + = + +
+ + = + +
) 1 ( )) ( 1 (
, ) 1 ( )) ( 1 (
2 2 2
2 2 2



Iterative scheme is
.
) ( 1
,
) ( 1
2 2
1
2 2
1
y
y x
t
n
kl y
n
kl x n
kl
n
kl
x
y x
t
n
kl y
n
kl x n
kl
n
kl
E
E E
E v E u E
v v
E
E E
E v E u E
u u
+ +
+ +
=
+ +
+ +
=
+
+

Figure 125
116
13. Photogrammetry & Stereo
I. Disparity between the two images.

Figure 131
Disparity : D x x
r l
=
' '

) 3 (
) 2 ( ) (
2
) 1 ( ) (
2
' '
'
'
= =

=

+
=
z
y
f
y
f
y
line red
z
b
x
f
x
line blue
z
b
x
f
x
r l
r
l

solve (1) (2) & (3)
f
z
b
x x
r l
= ) (
' '
: disparity
' ' ' '
' '
' '
' '
,
2 / ) (
,
2 / ) (
r l r l
r l
r l
r l
x x
bf
z
x x
y y
b y
x x
x x
b x

=
117
D x x
z
r l
1 1
' '
=

II. Photogrammetry

Figure 132

absolute orientation
) , , ( known r r
r l


(
(
(

r
r
r
z
y
x

0
r r R r
l r
+ =

(
(
(

33 32 31
23 22 21
13 12 11
r r r
r r r
r r r
component
rotaional

(
(
(

34
24
14
r
r
r
component
nal translatio


x
r
118
where R is a 3*3 orthonormal matrix
I R R
T
=

r l l l
r l l l
r l l l
z r z r y r x r
y r z r y r x r
x r z r y r x r
= + + +
= + + +
= + + +
34 33 32 31
24 23 22 21
14 13 12 11


3 equations => 12 unknowns

Orthonormality
I R R
T
=
1
1
1
2
33
2
32
2
31
2
23
2
22
2
21
2
13
2
12
2
11
= + +
= + +
= + +
r r r
r r r
r r r


0
0
0
13 33 12 32 11 31
33 23 32 22 31 21
23 13 22 12 21 11
= + +
= + +
= + +
r r r r r r
r r r r r r
r r r r r r



Figure 133

119
1 2 1 2 r r l l
r r r r =
3+7 = 10 equations
12 unknown variables

P&Q : 10+3 = 13 equations

Figure 134

0
r r R r
l r
+ = - equation 13.3.1

(
(
(

=
(
(
(

=
34
24
14
0
33 32 31
23 22 21
13 12 11
r
r
r
r
r r r
r r r
r r r
R

III. Relative orientation
r l
r r , ; not known
r l
z z , unknowns
The projection points ) , ( & ) , (
' ' ' '
r r l l
y x y x known determine the ratios of x and y to z
using
120
r
r r
r
r r
l
l l
l
l l
z
y
f
y
z
x
f
x
z
y
f
y
z
x
f
x
= =
= =
' '
' '
&
&
- - equation 13.4.1

12 + 2*(n)
substitute B into A

l
r
r l
l l
l
l
r l l l
z
f
f
z
x r z r
f
z
y r
f
z
x r
z
f
x r z r y r x r
) ( ) ' (
'
14 13 12
'
11
14 13 12 11
= + + +
= + + +



= = + + +
= = + + +
= = + + +
l
r
l
l l
r
l
r
r
l
l l
r
l
r
r
l
l l
z
z
f
z
f
r f r y r x r
s
y
z
z
y
z
f
r f r y r x r
s
x
z
z
x
z
f
r f r y r x r
34 33
'
32
'
31
'
24 23
'
22
'
21
'
14 13
'
12
'
11
- equation 13.3.2

12 + 2n ; # of unknowns for n points
7 + 3n ; # of equations
7 from orthonormality & distance length

121
I R R
T
=
(
(
(

1 0 0
0 1 0
0 0 1

j i if r r
r
j i
i
=
=
0
1
2

12 + 2n = 7 + 3n
n = 5
n 5

IV. 13.5 Using a known relative orientation
s
f
z
l
= ; constant s x
f
z
x x
l
l
l l
= =
/ '
s y y
l l
=
'
s f z
l
=
where s is a scaling factor
from equation 13.3.2
w cs r s f r y r x r z
v bs r s f r y r x r y
u as r s f r y r x r x
l l r
l l r
l l r
+ = + + + =
+ = + + + =
+ = + + + =
34 33
'
32
'
31
24 23
'
22
'
21
14 13
'
12
'
11
) (
) (
) (

w cs
v bs
z
y
f
y
w cs
u as
z
x
f
x
r
r r
r
r r
+
+
= =
+
+
= =
'
'





122
V. Computing depth
Once IR , ) , ( ), , ( ,
' ' ' '
0 r r l l
y x y x r known, The depth ) , (
r l
z z can be computed
as follows

Figure 135

From equation 13.3.2, multiply by
f
z
l

r l
l l
r
r
l
l l
r
r
l
l l
z r z r
f
y
r
f
x
r
z
f
y
r z r
f
y
r
f
x
r
z
f
x
r z r
f
y
r
f
x
r
= + + +
= + + +
= + + +
34 33
'
32
'
31
'
24 23
'
22
'
21
'
14 13
'
12
'
11
) (
) (
) (


From any two equations
Compute
r l
z z &
Then compute
r
r r T
r r r l
l
l l T
l l l l
z
f
y
f
x
z y x r
z
f
y
f
x
z y x r
|
|

\
|
= =
|
|

\
|
= =
1 , , ) , , (
1 , , ) , , (
' '
' '


123
VI. 13.7 exterior orientation

Figure 136
0
r r R r
a c
+ =

c a a a
c a a a
c a a a
z r z r y r x r
y r z r y r x r
x r z r y r x r
= + + +
= + + +
= + + +
34 33 32 31
24 23 22 21
14 13 12 11


I R R
T
= (orthonormality)

c
c
c
c
z
y
f
y
z
x
f
x
= =
'
&
'

34 33 32 31
24 23 22 21
34 33 32 31
14 13 12 11
'
'
r z r y r x r
r z r y r x r
z
y
f
y
r z r y r x r
r z r y r x r
z
x
f
x
a a a
a a a
c
c
a a a
a a a
c
c
+ + +
+ + +
= =
+ + +
+ + +
= =
- equation 13.7.1

# of equations = 2n +6
# of unknowns = 12
2n+6 = 12 => n=3 (at least we need three equations)
124
VII. Interior Orientation
scaling error
translation error
skewing error
shearing error

An affine transformation


23 22 21
13 12 11
) ( ) ( '
) ( ) ( '
a
z
y
a
z
x
a y
a
z
y
a
z
x
a x
c
c
c
c
c
c
c
c
+ + =
+ + =


from equation 13.7.1

34 33 32 31
24 23 22 21
34 33 32 31
14 13 12 11
'
'
s z s y s x s
s z s y s x s
f
y
s z s y s x s
s z s y s x s
f
x
a a a
a a a
a a a
a a a
+ + +
+ + +
=
+ + +
+ + +
=


ij
s
R : not orthonormal
I R R
ij ij
s
T
s
12 unknowns
#of equations = 2n
2n=12 ; n=6
125
VIII. Finding conjugate points
i. Gray-Level Matching
(reference figure 3-1)
z
b
x
f
x
z
b
x
f
x
r l 2
&
2
' '
=
+
=
at a matched point
) , ( ) , (
' ' ' '
r r r l l l
y x E y x E =
) (
) ' ,
2
( ) ' ,
2
( ~
' '
y y y
y
z
b
x
f E y
z
b
x
f E E
r l
r l
=

=
+

Let
z
bf
y x d
z
x
f
x
= ) ' , ' ( &
'


) ' ), ' , ' (
2
1
' ( ) ' ), ' , ' (
2
1
' ( y y x d x E y y x d x E
r l
= +
criterion : minimize
i s
e e e + =
where

= ' ' ) (
2
dy dx d e
s



= ' ' ) (
2
dy dx E E e
r l i

(solution) using Euler equation
0
' '
' '
=

y x
d d
F
y
F
x
Fd
where
126
( )
]
' '
[
2
1
()] () [ 2
]
'
()
)
2
1
( )
2
1
(
'
()
()][ () [ 2
)] ' ), ' , ' (
2
1
' ( ) ' ), ' , ' (
2
1
' ( [ ) (
2 2
x
E
x
E
E E
x
E
x
E
E E E Fd
y y x d x E y y x d x E d F
r l
r l
r l
r l
r l

=
+ + =

]
'
()
'
()
[
2
1
()] () [ ) (
) ( 2
' '
2 2
2 2
' '
x
E
x
E
E E d
d F
y
F
x
r l
r l
dy dx

=
=

127
14. Pattern classification
Reference Pattern Classification & Scene Analysis
Dudda & Hart (Wiley Interscience pub. C)

1. Bayes Decision Theory
Fundamental statistical approach to the problem of pattern classification
Maximum likelihood estimation : parameters fixed but unknown
Baysian estimation : parameters, random variables having some known a prior
distribution

(A) Maximum likelihood estimation

Figure 141

Given ; 1. Model of system
2. pata : outcome of some type of probabilistic experiment.
Goal : Estimate unknown model parameters from data
Let n i
i
. ,......... 2 , 1 , = be n I.I.D. observations on a random variable X drawn from
) : ( x f
x

128
The joint lpdf of
n
X X X ,........ ,
2 1
is
1 x
f

Where ) ; .. ,......... (
1

n
x x L is called the likelihood function (LF)
The maximum likelihood estimation (MLE)

is that value that maximizes the LF, that

is
) ; .. ,......... ( )

; .. ,......... (
1 1

n n
x x L x x L

Ex1) Assume ) , ( ;
2
N X where is known. Compute the MLE of the mean
<solution> The LF for h realizations of X is on a
|

\
|

|
|

\
|
=

=
h
i
i
n
x X L
0
2
2
2
) (
2
1
exp
2
1
) ; (

since the log function is monotonic, the maximum of L(x ; ) is also that of log L(x ; )

Hence

=
=
n
i
i
x
n
x L
1
2
2
2
) (
2
1
) 2 log(
2
) ; ( log

Set 0
) ; ( log
=

u
x L

This yields

=
=

n
i
i
x
0
2
0 ) (
1

=
=
n
i
i
x
n
1
1


Ex2) Consider the normal p.d.f.,
129

=
=
n
i
x
x x f
1
2
2
2
2
) ) (
2
1
exp(
2
1
) , , (

The log likelihood function is

=
=
n
i
i
x n
n
j x x L
1
2
2
2
2 1
) (
2
1
log 2 log
2
) , ,....... ( log

Now set 0 , 0 =

L L

Obtain the simultaneous equations

=
=
= +
=
n
i
i
n
i
i
x
n
x
1
2
3
1
0 ) (
1
0 ) (

Hence

=
=
n
i
i
X
n
1
1

=
=
n
i
i
x
n
1
2 2
) (
1


(B) Baysian Estimation
Suppose that an classifier to separate two kinds of lumber, ash and birch


Figure 142

130
A prior : probability p(w
1
) : the next piece is ash
probability p(w
2
) : the next piece is birch

Decision Rule : Decide w
1
if p(w
1
) > p(w
2
)
Decide w
2
otherwise
Bays Rule ;


) (
) ( ) | (
) | (
x p
w p w x p
X w p
j j
j

=
where

=
=
r
j
j j
w p w x p x p
1
) ( ) | ( ) (

Bayes Decision Rule for minimizing the probability of error ;
Decide w
1
if p(w
1
|x) > p(w
2
|x)
Decide w
2
otherwise

Decide w
1
if P(x| w
1
)p(w
1
) > p(x| w
2
)p(w
2
)
Decide w
2
otherwise

If equally likely, p(w
1
)=p(w
2
), then
131
Decide w
1
if p(x| w
1
) > p(x| w
2
)
Decide w
2
if otherwise

Figure 143

Probability error : p(error)
( ) ( , )
( | ) ( )
p error p error x dx
p error x p x dx

=
=

where p(error|x)

\
|
1 2
2 1
) | (
) | (
w decide we if x w p
w decide we if x w p


Figure 144
132
(B-1) Generalized Form (for Bayes classifier)
Let } .......... {
1 s
w w = be the finite set of S states of nature and ) ........ {
1 a
A = be the
finite set of possible actions.


Let ) | (
j i
w be the loss incurred for taking action
i
when the state of nature is
j
w
Let the feature vector X be a d-component vector-valued r.v
Then
) (
) ( ) | (
) | (
x p
w p w x p
x w p
j j
j
=
Where

=
=
s
j
j j
w p w x p X p
1
) ( ) | ( ) (
The expected loss associated with taking action
i
is
) | ( ) | ( ) | (
1

=
=
s
j
j j i i
X w p w X R

Target: to find a Bayes decision rule against p(w
j
) that minimizes the overall risk
The overall risk is given by

= x d x p x x R R ) ( ) | ) ( (
clearly, if ) (x is chosen so that ) | ) ( ( x x R is as small as possible, for every X , then
the overall risk will be minimized Bayes decision rule for the general form; to minimize
the overall risk.
133
1.Compute the conditional risk

=
=
s
j
j j i i
X w p w X R
1
) | ( ) | ( ) | (
for i=1,..a


2.Select the action
i
for which ) | ( X R
i
is minimum
3.Obtain the resulting minimum overall risk called the Bayes risk
B.2 Two category classification
Let ) | (
j i ij
w = the loss incurred for deciding w
i
when the true state of nature is w
j
.
) | ( ) | ( ) | (
) | ( ) | ( ) | (
2 22 1 21 2
2 12 1 11 1
X w p X w p X R
X w p X w p X R


+ =
+ =
- equation 14.1

The decision rule :
Decide w
1
if ) | ( ) | (
2 1
X R X R < - equation 14.2
Decide w
2
otherwise


If we substitute 1 into 2
We have
) | ( ) ( ) | ( ) (
) | ( ) | ( ) | ( ) | (
1 11 21 2 22 12
2 22 1 21 2 12 1 11
X w p X w p
X w p X w p X w p X w p


<
+ < +

Hence likelyhood ratio
134

Figure 145

<Minimum error rate classification>
Symmetrical on zero-one loss function
The loss ) | (
j i
w = 0 i=j for i,j=1,c
1 ij
all errors are equally costly
The conditional risk is

=
=
c
j
j j i i
X w p w X R
1
) | ( ) | ( ) | (

) | ( 1
) | (
X w p
X w p
i
c
i j
j
=
=

To minimize the average prob of error, we should select the i that maximizes the a
posterior prob
) | ( X w p
i
In other words, for minimum error rate j
Decide
i
w if ) | ( ) | ( X w p X w p
j i
> for all , i j
B-3 The multicategory classification
135
A pattern classifier

Figure 146

), ( X g
i
for i=1,c ; discriminant function
The classification is said to assign a feature vector X to class
j
w
If ) ( ) ( X g X g
j i
> for all i j
) | ( ) ( X w p X g
i i
= ; a posterior prob

=
=
c
j
j j
j i
w p w X p
w p w X p
1
) ( ) | (
) ( ) | (
by Bayes rule

) ( log ) | ( log
) ( ) | (
i i
i i
w p w X p
w p w X p
+

Decision Rule
1. Divide the feature space into C division regions, R
1
,..R
c

2. If ) ( ) ( X g X g
j i
> for all ji then X is in R
I

136
In other words, assign X to w
I


Ex) three category classification

Probability error calculation
P(error X
o
) ) ( ) (
0 0
X P X error P =
) ( ) ( ) ( ) (
3 3 0 2 2 0
w P w x P w P w X P + =
Total error
P(error)

= dx x error P ) , (