EE4H, M.

Sc 0407191
Computer Vision
Dr. Mike Spann
m.spann@bham.ac.uk
http://www.eee.bham.ac.uk/spannm
Introduction
Images may suffer from the following degradations:
Poor contrast due to poor illumination or finite
sensitivity of the imaging device
Electronic sensor noise or atmospheric disturbances
leading to broad band noise
Aliasing effects due to inadequate sampling
Finite aperture effects or motion leading to spatial

Introduction
We will consider simple algorithms for image
enhancement based on lookup tables
Contrast enhancement
We will also consider simple linear filtering
algorithms
Noise removal
Histogram equalisation
In an image of low contrast, the image has grey
levels concentrated in a narrow band
Define the grey level histogram of an image h(i)
where :
h(i)=number of pixels with grey level =I
For a low contrast image, the histogram will be
concentrated in a narrow band
The full greylevel dynamic range is not used

Histogram equalisation
h i ( )
i
Histogram equalisation
Can use a sigmoid lookup to map input to output grey
levels
A sigmoid function g(i) controls the mapping from
input to output pixel
Can easily be implemented in hardware for maximum
efficiency
Histogram equalisation
h i ( )
g i ( )
h i ' ( )
h i h g i ' ( ) ( ( )) =
1
( )
( )
g i
i
( )
exp
=
+
1
1 u
Histogram equalisation
controls the position of maximum slope
controls the slope
Problem - we need to determine the optimum
sigmoid parameters and for each image
A better method would be to determine the best
mapping function from the image data
Histogram equalisation
A general histogram stretching algorithm is defined in
terms of a transormation g(i)
We require a transformation g(i) such that from any
histogram h(i) :

constant ) ( ) ( '
) ( :
= =

= j g i j
j h i h
Histogram equalisation
Constraints (N x N x 8 bit image)
No crossover in grey levels after transformation


2
) ( ' N i h
i
=

i i g i g i
1 2 1 2
( ) ( )
Histogram equalisation
An adaptive histogram equalisation algorithm can be
defined in terms of the cumulative histogram H(i) :
H i ( ) = number of pixels with grey levels i s

=
=
i
j
j h i H
0
) ( ) (
Histogram equalisation
Since the required h(i) is flat, the required H(i) is a
ramp:
h(i) H(i)
Histogram equalisation
Let the actual histogram and cumulative
histogram be h(i) and H(i)
Let the desired histogram and desired cumulative
histogram be h(i) and H(i)
Let the transformation be g(i)


H g i
N g i
' ( ( ))
( )
~
2
255
( ' ( ) , ' ( ) ) H N H 255 0 0
2
= =
Histogram equalisation
Since g(i) is an ordered transformation
i i g i g i
1 2 1 2
< < ( ) ( )
H g i H i
N g i
' ( ( )) ( )
( )
= =
2
255
g i
H i
N
( )
( )
=
255
2
Histogram equalisation
Worked example, 32 x 32 bit image with grey levels
quantised to 3 bits
g i
H i
( )
( )
=
7
1024
h i h j
j i g j
' ( ) ( )
: ( )
=
=
Histogram equalisation
i h i ( )
H i ( ) g i ( )
0

197

197

1.351

-

1

256

453

3.103

197

2

212

665

4.555

-

3

164

829

5.676

256

4

82

911

6.236

-

5

62

993

6.657

212

6

31

1004

6.867

246

7

20

1024

7.07

113

h i ' ( )
Histogram equalisation
0 1 2 3 4 5 6 7
0
50
100
150
200
250
300
0 1 2 3 4 5 6 7
Original histogram
Stretched histogram
Histogram equalisation
0.00
500.00
1000.00
1500.00
2000.00
0.00 50.00 100.00 150.00 200.00 250.00
i
h(i)
0.00
500.00
1000.00
1500.00
2000.00
0.00 50.00 100.00 150.00 200.00 250.00
i
h(i)
Histogram equalisation
0.00
500.00
1000.00
1500.00
2000.00
2500.00
3000.00
0.00 50.00 100.00 150.00 200.00 250.00
i
h(i)
0.00
500.00
1000.00
1500.00
2000.00
2500.00
3000.00
0.00 50.00 100.00 150.00 200.00 250.00
i
h(i)
Histogram equalisation
ImageJ demonstration
http://rsb.info.nih.gov/ij/signed-applet

Image Filtering
Simple image operators can be classified as 'pointwise'
or 'neighbourhood' (filtering) operators
Histogram equalisation is a pointwise operation
More general filtering operations use neighbourhoods
of pixels
Image Filtering
(x,y) (x,y)
Input image Output image
(x,y) (x,y)
pointwise
transformation
neighbourhood
transformation
Input image Output image
Image Filtering
The output g(x,y) can be a linear or non-linear
function of the set of input pixel grey levels {f(x-M,y-
M)f(x+M,y+M}.
(x,y) (x,y)
Input image f(x,y) Output image g(x,y)
(x-1,y-1)
(x+1,y+1)
Image Filtering
Examples of filters:
g x y h f x y h f x y
h f x y
( , ) ( , ) ( , )
..... ( , )
= +
+ + +
1 2
9
1 1 1
1 1
g x y median
f x y f x y
f x y
( , )
( , ), ( , )
..... ( , )
=

+ +
|
\

|
.
|
1 1 1
1 1
Linear filtering and convolution
Example
3x3 arithmetic mean of an input image (ignoring
floating point byte rounding)
(x,y) (x,y)
Input image f(x,y) Output image g(x,y)
(x-1,y-1)
(x+1,y+1)
Linear filtering and convolution
Convolution involves overlap multiply add with
convolution mask
H =
|
\

|
.
|
|
|
|
|
|
1
9
1
9
1
9
1
9
1
9
1
9
1
9
1
9
1
9
Linear filtering and convolution
(x,y) (x,y)
Input image f(x,y) Output image g(x,y)
Image point
Filter mask point
E
Linear filtering and convolution
We can define the convolution operator
mathematically
Defines a 2D convolution of an image f(x,y) with a filter
h(x,y)
g x y h x y f x x y y
f x x y y
y x
y x
( , ) ( ' , ' ) ( ' , ' )
( ' , ' )
' '
' '
=
=
= =
= =
1
1
1
1
1
1
1
1
1
9

Linear filtering and convolution
Example convolution with a Gaussian filter kernel
determines the width of the filter and hence the
amount of smoothing
g x y
x y
g x g y
g x
x
( , ) exp(
( )
)
( ) ( )
( ) exp( )
=
+
=
=
2 2
2
2
2
2
2
o
o

Linear filtering and convolution
0.00
0.20
0.40
0.60
0.80
1.00
-6 -4 -2 0 2 4 x
g(x)

Linear filtering and convolution
Original
Noisy
Filtered
=1.5
Filtered
=3.0
Linear filtering and convolution
ImageJ demonstration
http://rsb.info.nih.gov/ij/signed-applet
Linear filtering and convolution
We can also convolution to be a frequency domain
operation
Based on the discrete Fourier transform F(u,v) of the
image f(x,y)
F u v f x y
j
N
ux vy
y
N
x
N
( , ) ( , )exp( ( )) = +
=

2
0
1
0
1
t
u v N , .. = 0 1
Linear filtering and convolution
The inverse DFT is defined by
f x y
N
F u v
j
N
ux vy
y
N
x
N
( , ) ( , )exp( ( )) = +
=

1 2
2
0
1
0
1

t
x y N , .. = 0 1
Linear filtering and convolution
x
y
f(x,y)
(0.0)
(N-1,N-1)
v
u
(0,0)
(N-1,N-1)
F(u,v)
DFT IDFT
Linear filtering and convolution
log( ( , ) ) 1+ F u v
Linear filtering and convolution
F(u,v) is the frequency content of the image at spatial
frequency position (u,v)
Smooth regions of the image contribute low frequency
components to F(u,v)
Abrupt transitions in grey level (lines and edges)
contribute high frequency components to F(u,v)
Linear filtering and convolution
We can compute the DFT directly using the formula
An N point DFT would require N
2
floating point
multiplications per output point
Since there are N
2
output points , the computational
complexity of the DFT is N
4
N
4
=4x10
9
for N=256

Bad news! Many hours on a workstation


Linear filtering and convolution
The FFT algorithm was developed in the 60s for
seismic exploration
Reduced the DFT complexity to 2N
2
log
2
N
2N
2
log
2
N~10
6
for N=256

A few seconds on a workstation

Linear filtering and convolution
The filtering interpretation of convolution can be
understood in terms of the convolution theorem
The convolution of an image f(x,y) with a filter h(x,y) is
defined as:
g x y h x y f x x y y
f x y h x y
y
M
x
M
( , ) ( ' , ' ) ( ' , ' )
( , )* ( , )
' '
=

0
1
0
1
Linear filtering and convolution
(x,y)
Input image f(x,y) Output image g(x,y)
(x,y)
E
Filter mask h(x,y)
Linear filtering and convolution
Note that the filter mask is shifted and inverted prior
to the overlap multiply and add stage of the
convolution
Define the DFTs of f(x,y),h(x,y), and g(x,y) as
F(u,v),H(u,v) and G(u,v)
The convolution theorem states simply that :
G u v H u v F u v ( , ) ( , ) ( , )
Linear filtering and convolution
As an example, suppose h(x,y) corresponds to a linear
filter with frequency response defined as follows:



Removes low frequency components of the image
H u v u v R ( , ) = + <
=
0
1
2 2
for
otherwise
Linear filtering and convolution
DFT
IDFT
Linear filtering and convolution
Frequency domain implementation of
convolution
Image f(x,y) N x N pixels
Filter h(x,y) M x M filter mask points
Usually M<<N
In this case the filter mask is 'zero-padded' out to N
x N
The output image g(x,y) is of size N+M-1 x N+M-1
pixels. The filter mask wraps around truncating
g(x,y) to an N x N image


Linear filtering and convolution
Filter mask h(x,y)
Input image f(x,y) zero padding
x x
x x
x x
DFT DFT
H(u,v) F(u,v)
H(u,v)F(u,v)
IDFT
f(x,y) * h(x,y)
x x x x x
x x x x x
Linear filtering and convolution
Input image f(x,y) Output image g(x,y)
(x,y)
Filter mask h(x,y)
(x',y')
x' = x modulo N
y' = y modulo N
Linear filtering and convolution
We can evaluate the computational complexity of
implementing convolution in the spatial and
spatial frequency domains
N x N image is to be convolved with an M x M
filter
Spatial domain convolution requires M
2
floating point
multiplications per output point or N
2
M
2
in total
Frequency domain implementation requires 3x(2N
2
log
2

N) + N
2
floating point multiplications ( 2 DFTs +
1 IDFT + N
2
multiplications of the DFTs)
Linear filtering and convolution
Example 1, N=512, M=7
Spatial domain implementation requires 1.3 x 10
7

floating point multiplications
Frequency domain implementation requires 1.4 x 10
7
floating point multiplications
Example 2, N=512, M=32
Spatial domain implementation requires 2.7 x 10
8

floating point multiplications
Frequency domain implementation requires 1.4 x 10
7
floating point multiplications
Linear filtering and convolution
For smaller mask sizes, spatial and frequency
domain implementations have about the same
computational complexity
However, we can speed up frequency domain
interpretations by tessellating the image into sub-
blocks and filtering these independently
Not quite that simple we need to overlap the filtered
sub-blocks to remove blocking artefacts
Overlap and add algorithm
Linear filtering and convolution
We can look at some examples of linear filters
commonly used in image processing and their
frequency responses
In particular we will look at a smoothing filter and a
filter to perform edge detection
Linear filtering and convolution
Smoothing (low pass) filter
Simple arithmetic averaging
Useful for smoothing images corrupted by additive
broad band noise
H H
3 5
1
9
1 1 1
1 1 1
1 1 1
1
25
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
=
|
\

|
.
|
|
|
=
|
\

|
.
|
|
|
|
|
|
etc
Linear filtering and convolution
h x ( )
H u ( )
Spatial domain Spatial frequency domain
u
x
Linear filtering and convolution
Edge detection filter
Simple differencing filter used for enhancing edged
Has a bandpass frequency response
H =

|
\

|
.
|
|
|
1 0 1
1 0 1
1 0 1
Linear filtering and convolution
ImageJ demonstration
http://rsb.info.nih.gov/ij/signed-applet
Linear filtering and convolution
f x ( )
p
x
f x ( )*( ) 1 0 1
p
x
Linear filtering and convolution
We can evaluate the (1D) frequency response of
the filter h(x)={1,0,-1 } from the DFT definition
H u h x
jux
N
ju
N
ju
N
ju
N
ju
N
j
ju
N
u
N
x
N
( ) ( )exp( )
exp( )
exp( ) exp( ) exp( )
exp( )sin( )
=
=
=
|
\

|
.
|
=
=

2
1
4
2 2 2
2
2 2
0
1
t
t
t t t
t t



Linear filtering and convolution
The magnitude of the response is therefore:



This has a bandpass characteristic
H u
u
N
( ) sin( ) =
2t
Linear filtering and convolution
H u ( )
u
Conclusion
We have looked at basic (low level) image processing
operations
Enhancement
Filtering
These are usually important pre-processing steps
carried out in computer vision systems (often in
hardware)