Merged

!"#$"!
%
!
E4830 DIGITAL IMAGE
PROCESSING
SPRING 2014
THURSDAYS 1:10 3:40PM
Lecture 1
January 24, 2014
1/23/14
1
1/23/14 Christine P. Hendon 2
My background
Christine P. Hendon, PhD
Assistant Professor
Electrical Engineering Department

Research interests: biomedical optics, signal and
image processing
What is a Digital Image?
1/23/14 Christine P. Hendon
4
! A digital image is a representation of a two-
dimensional image as a finite set of digital values
called pixels
Gonzalez and Woods. Digital Image Processing. 3
rd
edition
What is Digital Image Processing
5
Low Level Process
Input: Image
Output: Image
Examples: Noise
removal, image
sharpening
Mid Level Process
Input: Image
Output: Attributes
Examples: Object
recognition,
segmentation
High Level Process
Input: Attributes
Output: Understanding
Examples: Scene
understanding,
autonomous navigation
Geographical
Information Systems
Industrial Inspection
Medical Imaging
rd
edition
E4830 Digital Image Processing
6
! Prerequisites: Signals and systems or equivalent required;
Familiarity with probability and linear algebra. Intended for seniors,
juniors, or beginning graduate students.
! Description: Introduction to theories, algorithms, and practical
solutions of digital image/video perception, acquisition, color
representation, quantization, transform, enhancement, filtering, multi-
spectral processing, restoration, analysis, feature extraction,
segmentation, morphological transform, and compression.
! Students will gain understanding of algorithm design, mathematical
tools, and practical implementations of various digital image
applications. Considerations of practical system requirements (e.g.,
medical, satellite, consumer) will be discussed.
!"#$"!%
#
Key Stages in Digital Image Processing
7
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation &
Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
Key Stages in Digital Image Processing:
Image Aquisition
8
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation &
Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
rd
edition
Image Enhancement
9
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation &
Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
rd
edition
Image Restoration
10
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation &
Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
rd
edition
Morphological Processing
11
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation &
Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
rd
edition
Segmentation
12
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation &
Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
rd
edition
!"#$"!%
$
Object Recognition
13
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation &
Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
rd
edition
Representation & Description
14
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation &
Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
rd
edition
Image Compression
15
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation &
Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
rd
edition
Colour Image Processing
16
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation &
Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
rd
edition
Syllabus
17
!"#$"!%
%
Evaluation
19
Digital image fundamentals
1/23/14
20
Christine P. Hendon
Elements of Visual Perception
21
! The human eye
!The lens focuses light from objects
onto the retina
!The retina is covered with
light receptors called
cones (6-7 million) and
rods (75-150 million)
!Cones are concentrated
around the fovea and are
very sensitive to colour
!Rods are more spread out
and are sensitive to low levels of
illumination
rd
edition
Image Formation in the Eye
22
!Muscles within the eye can be used to change the shape
of the lens allowing us focus on objects that are near or
far away
!An image is focused onto the retina causing rods and
cones to become excited which ultimately send signals to
the brain
rd
edition
Brightness, Adaptation, and
Discrimination
23
! Because digital images are
displayed as a discrete set of
intensities, the eyes ability to
discriminate between different
intensity levels is an important
consideration in image
processing.
! Subjective brightness intensity
perceived by the human visual
system
! Logarithmic function of the light
intensity incident on the eye
Range of subjective brightness
sensations showing a particular
adaptation
rd
edition
Human
perception
phenomena
Mach band, perceived
brightness pattern strongly
scalloped near the
boundaries
rd
edition
!"#$"!%
'
Human perception phenomena
25
Simultaneous contrast regions perceived brightness does not depend simply
on its intensity
rd
edition
Optical Illusions
26
! Optical illusions another
example of human
perception phenomena
! Eye fills nonexisting
information or wrongly
perceives geometric
properties of objects
rd
edition
Light and the Electromagnetic Spectrum
! Color that humans perceive in an object are determined
by the nature of he light reflected from the object
! Images can also be generated from other sources:
sound, electron beams, synthetic, etc
! =
c
"
E = h"
rd
edition
28
! Monochromatic light
! Described by intensity or gray level
! Chromatic (color) light
! Described by radiance, luminance, and
brightness
" Radiance total amount of energy from
the light source [W]
" Luminance amount of energy observer
perceives from the light source [lm]
" Brightness subjective descriptor of light
perception
Image Sensing and Acquisition
29
Single Element
Linear Sensor
Array Sensor
Single sensor with motion to
generate 2D image
Linear Strip
Circular Strip
rd
edition
Digital Image Acquisition Process
30
! Model of image formation, image f(x,y)
! Characterized by
! Amount of source illumination incident on the scene being
viewed, i(x,y)
! Amount of illumination reflected by objects in scene, r(x,y)
0 < f x, y ( ) < !
f x, y ( ) = i x, y ( )r x, y ( )
0 < i x, y ( ) < !
0 < r x, y ( ) <1
0 < t x, y ( ) <1
rd
edition
!"#$"!%
(
Image Sampling and Quantization
31
! Sampling: digitizing the coordinates
! Quantization: digitizing the amplitude
Continuous image projected onto
sensor array results of sampling and
quantization
rd
edition
1 pixel
Representing Digital Images
33
Numerical array for processing and
algorithm development
MxN array
As a surface
Visual intensity
array
Numerical array
rd
edition
Image Intensity Levels
34
! L=2
k
Discrete Intensity levels
! Discrete levels are equally spaced and are integers in
the interval [0, L-1]
! b=MxNxk
! bits requires to store a digitized image
! If M=N, b=N
2
k
rd
edition
Dynamic Range
35
! Dynamic range of imaging
system
! Upper limit determined by
saturation
! Lower limit determined by noise
! Contrast
! Difference in intensity in between
the highest and lowest intensity
levels in an image
rd
edition
Number of Storage Bits for
Various N and K
36
! b=N
2
k (bits)
! L=2
k
(intensity level)
rd
edition
!"#$"!%
)
Spatial and Intensity Resolution
37
! Spatial resolution: line pairs
per unit distance, dots (or
pixels) per unit distance
! Intensity resolution: number of
bits used to quantize intensity.
ie 8bit, 16 bit
rd
edition
1250 dpi 300 dpi
72 dpi 150 dpi
Spatial Resolution
Gonzalez and Woods. Digital Image Processing. 2nd edition
G
r
a
y
-
l
e
v
e
l

R
e
s
o
l
u
t
i
o
n

256 level image
2 level image
rd
edition
Artifact: False contouring insufficient number of intensity
levels in smooth areas of digital image
Image Interpolation
1/23/14
! Image interpolation or Image resampling methods
! Applied to image resizing (shrinking and zooming)
! Interpolation process of using known data to estimate values at
unknown locations
! Nearest neighbor interpolation assigns new location intensity of its
nearest neighbor in the original image
! Bilinear uses four nearest neighbors to estimate intensity at given
location
! Bicubic uses sixteen nearest neighbors to estimate intensity at given
location
v x, y ( ) = ax + by + cxy + d
v x, y ( ) = a
ij
x
i
y
j
j=0
3
!
i=0
3
!
Image Interpolation
rd
edition
Image reduced to 72dpi
and zoomed back to
original size using
nearest neighbor
interpolation
and zoomed back to
original size using
bilinear interpolation
and zoomed back to
original size using
bicubic
Image reduced to 150
dpi and zoomed back to
original size using
nearest neighbor
interpolation
original size using
bilinear interpolation
original size using
bicubic
Resolution: How Much Is Enough?
!The big question with resolution is always how
much is enough?
! This all depends on what is in the image and what
you would like to do with it
! Key questions include
" Does the image look aesthetically pleasing?
" Can you see what you need to see within the image?
!"#$"!%
*
Resolution: How Much Is Enough?
!The picture on the right is fine for counting the
number of cars, but not for reading the number
plate
Basic Relationships between Pixels
44
! Neighbors of a pixel
! Adjacency, Connectivity, Regions, and Boundaries
! 4-adjacency
! 8-adjacency
! M-adjacency (mixed adjacency)
! Distance Measures
Introduction to Mathematical Tools
45
N=5 N=10
N=20 N=50 N=100
Image with Gaussian noise
Averaging n noisy images
rd
edition
Image Subtraction
47
! To enhance differences between images
rd
edition
Difference image,
scaled to [0 255]
! Mask mode radiography
Mask Live
Difference Enhancement
rd
edition
g x, y ( ) = f x, y ( ) ! h x, y ( )
mask X-ray image
!"#$"!%
+
Shade correction
49

! Application of image multiplication or division
! If h(x,y) is known, can recover f(x)
! If h(x,y) is not known but have access to imaging system,
can approximate h(x,y) by imaging sample with
constant intensity
g x, y ( ) = f x, y ( )h x, y ( )
Shading function
Perfect image
rd
edition
Region of Interest (ROI)
50
! Application of Multiplication
! Masking
! Multiplying image by mask, image that has 1s in
ROI and 0s elsewhere
! Shape of ROI can be arbitrary
rd
edition
Set and Logical Operations
51
! Subset
! Union
! Intersection
! Disjoint
! Mutually exclusive
! Set Universe
! Complement
! Difference
Set membership based on coordinates
rd
edition
Set Operations Involving Gray Scale
Images
52
! The union and
intersection operations
for gray-scale values
usually are defined as
the max and min of
corresponding pixel
pairs, complement as
pairwise difference
between a constant and
intensity of every pixel
in an image
! Array Operations
A! B = max a, b ( ) | a !A, b !B { }
rd
edition
original Set complement
Union of original
and constant
image
Logical Operations
53
! Binary images
! Foreground 1 valued
! Background 0 valued set
of pixels
! Objects: regions composed
of foreground
Union Intersect Complement

OR AND NOT
rd
edition
Fuzzy Sets
54
! Set of logical results are crisps concepts. Elements
are either or are not members of a set
! Membership functions that are graduate between
limit values
!"#$"!%
!,
Spatial Operations
55
1. Single Pixel Operations
2. Neighborhood Operations
3. Geometrical Transformations
rd
edition
Intensity transformation
function to obtain negative
of an 8-bit image
Neighborhood Operations
56
! S
x,y
set of coordinates
centered on arbitrary point (x,y)
in an image, f
! Neighborhood processing
generates a corresponding pixel
at the same coordinate in an
output (processed) image, g,
such that the value of that pixel
is determined from the pixels in
the input image with coordinates
S
x,y
g x, y ( ) =
1
mn
f r, c ( )
r,c ( )!Sx,y
"
rd
edition
Geometrical Transformations and
image registration
57
! Modify spatial
relationship
between pixels in
an image
! Consists of two
operations
! Spatial
transformation of
the coordinates
! Intensity
interpolation to
spatially
transformed pixels
rd
edition
Spatial Transformation and Intensity
Interpolation
58
1. 300 dpi image of the letter T
2. Image rotated 21
o
using nearest neighbor interpolation to
assign intensity values to spatially transformed pixels
3. Image rotated 21
o
using bilinear interpolation
4. Image rotated 21
o
using bicubic interpolation
rd
edition
Image Registration
59
! Used to align two ore
more images of the same
scene
! Input image
! Reference image
! Tie (Control) points
! Corresponding points
between input and
reference image
! Physical markers within
the system to help to
generate tie points
rd
edition
Reference
Input
Difference
Registered
Vector and Matric Operations
60
! Multispectral image processing
! RGB image
" Each pixel has 3 components can be organized
into column vector
! Can have n vector components
! Can use tools of vector-matrix theory
z =
z
1
z
2
z
3
!
"
#
#
#
$
%
&
&
&
rd
edition
!"#$"!%
!!
Approach for Operating in Linear
Transform Domain
rd
edition
Probabilistic Methods
62
! Mean
! Variance
! Moment
rd
edition
Homework Assignments
63
! Matlab Report Generator
Next Week
64
! Chapter 3: Intensity Transformations and Spatial
Filtering
!"#$"!%
!
E4830 DIGITAL IMAGE
PROCESSING
SPRING 2014
Intensity Transformations and
Spatial Filtering
Lecture 2
January 30, 2014
1/30/14
Christine P. Hendon 1
Lecture 1 Review
2
Lecture Overview
! Basic Intensity Transformation Functions
! Histogram Processing
! Fundamentals of Spatial Filtering
! Smoothing Spatial Filters
! Sharpening Spatial Filters
3
Image Pre-processing
! Image enhancement
! Image restoration
4
The Basics of Intensity Transformations
and Spatial Filtering
! Spatial vs. Fourier Domain
! Spatial domain: plane containing the pixels
of an image
! Spatial filtering
! Spatial filter: spatial mask, kernel, template or
window
! For neighborhood of size 1x1
! gray level mapping or transformation function
g x, y ( ) = T f x, y ( ) !
"
#
$
input output
operator
s = T r ( )
rd
edition
5
Advantages of Spatial Filtering
! Linear and non-linear filtering
6
!"#$"!%
'
Brightness Correction and
Grayscale Transformations
! Position depdent brightness correction
! Gray scale transformations
! Example: Negative images are useful for enhancing
white or grey detail embedded in dark regions of an
image Original
Image
Negative
Image
s = L !1! r
g x, y ( ) = f x, y ( )e x, y ( )
rd
edition
7
Point Processing
Example: Thresholding
! Thresholding transformations are particularly useful
for segmentation in which we want to isolate an
object of interest from a background
s =
1.0
0.0 r <= threshold
r > threshold
rd
edition
8
Basic Grey Level Transformations
! There are many different kinds of grey level
transformations
" Linear
" Negative/Identity
" Logarithmic
" Log/Inverse log
" Power law
" n
th
power/n
th
root
rd
edition
9
Logarithmic Transformations
! The log transformation maps a narrow range of low
input grey level values into a wider range of output
values
! The inverse log transformation performs the
opposite transformation
s = clog 1+ r ( )
10
! Log functions are particularly useful when the input
grey level values may have an extremely large
range of values
s = log(1 + r)
Usually set c to 1
Grey levels must be in the range [0.0, 1.0]
rd
edition
11
Power Law Transformations
! Map a narrow range
of dark input values
into a wider range of
output values or vice
versa
! Varying ( gives a whole
family of curves
s = cr
!
Usually set c to 1
rd
edition
12
!"#$"!%
#
Gamma Correction
! A variety of devices used
for image capture,
printing and display
respond according to a
power law
! Exponent of power-law
equation called gamma
! Process used to correct
power-law response
called gamma correction
rd
edition
13
Power Law Example
! Magnetic
resonance
(MR) image of a
fractured human
spine
! Different curves
highlight different
detail
s = r
0.6

s
=
r
0
.3

0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.2 0.4 0.6 0.8 1
Original Intensities
Transform
ed Intensities
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.2 0.4 0.6 0.8 1
Transform
ed Intensities
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.2 0.4 0.6 0.8 1
Old Intensities
Transform
ed Intensities
s

=

r

0
.4

rd
edition
! Power law
transforms are
used to darken
the image
! Different curves
highlight
different detail
s = r
3.0

s

=

r

4
.0

s
=
r
5
.0

rd
edition
15
Piecewise Linear Transformation
Functions
! Rather than using a well defined mathematical
function we can use arbitrary user-defined
transforms
! Contrast stretching linear transform to add contrast
to a poor quality image
rd
edition
16
Intensity Level Slicing
!Highlights a specific range of grey levels
! Similar to thresholding
! Other levels can be
suppressed or maintained
! Useful for highlighting features
in an image
rd
edition
17
Bit Plane Slicing
! Often by isolating particular bits of the pixel values
in an image we can highlight interesting aspects of
that image
" Higher-order bits usually contain most of the significant visual
information
" Lower-order bits contain
subtle details
rd
edition
18
!"#$"!%
%
Bit Plane Slicing
rd
edition
19
Bit Plane Slicing
Reconstructed image using
only bit planes 8 and 7
only bit planes 8, 7 and 6
rd
edition
20
Image Histograms
! The histogram of an image shows us the distribution
of grey levels in the image
rd
edition
21
Contrast Stretching
! We can fix images that have poor contrast by
applying a pretty simple contrast specification
! The interesting part is how do we decide on this
transformation function?
22
Histogram Equalization
Spreading out the frequencies in an image (or
equalising the image) is a simple way to improve dark
or washed out images
The formula for histogram
equalization is given where
! r
k
: input intensity
! s
k
: processed intensity
! k: the intensity range
(e.g 0.0 1.0)
! n
j
: the frequency of intensity j
! n: the sum of all frequencies
) (
k k
r T s =
!
=
=
k
j
j r
r p
1
) (
!
=
=
k
j
j
n
n
1
23
Histogram of Different
Types of Images
! A selection of images and
their histograms
! Notice the relationships
between the images and
their histograms
rd
edition
24
!"#$"!%
)
! Note that the high
contrast image has
an evenly spaced
histogram
High contrast images
rd
edition
25
Equalisation Transformation Function
rd
edition
26
Equalisation Examples
1
rd
edition
27
2
rd
edition
28
3
4
rd
edition
29
Histogram Equalization and
Specification
Histogram specification Histogram equalization
rd
edition
30
!"#$"!%
*
Matlab function: adapthisteq
! Contrast limited adaptive histogram equalization
! Processing small regions of the image called tiles using
histogram specification for each tile individually
! Neighboring tiles combined using bilinear interpolation
to eliminate artificially induced boundaries
rd
edition
31
32
Local Pre-Processing or Image
Enhancement
! Objective: use small neighborhood of a pixel in an
input image to produce new brightness value in
output image # filtering
! Smoothing
! Gradient Operations
33
! Neighbourhood operations simply operate on a
larger neighborhood of pixels than point operations
! Neighborhoods are
mostly a rectangle
around a central pixel
! Any size rectangle
and any shape filter
are possible
Origin x
y Image f (x, y)
(x, y)
Neighbourhood
34
The Spatial Filtering Process
r s t
u v w
x y z
Origin x
y Image f (x, y)
e
processed
= ve +
ra + sb + tc +
ud + wf +
xg + yh + zi
Filter
Simple 3*3
Neighbourhood
e 3*3 Filter
a b c
d e f
g h i
Original Image
Pixels
*
The above is repeated for every pixel in the original
image to generate the filtered image
35
M
e
c
h
a
n
i
c
s

o
f

S
p
a
t
i
a
l

F
i
l
t
e
r
i
n
g

!!
" = " =
+ + =
a
a s
b
b t
t y s x f t s w y x g ) , ( ) , ( ) , (
! Correlation
! Convolution
! Zero-patting
! Cropped
results
rd
edition
!"#$"!%
+
Correlation and Convolution
! The filtering we have been talking about so far is
referred to as correlation with the filter itself referred
to as the correlation kernel
! Convolution is a similar operation, with just one subtle
difference
! For symmetric filters it makes no difference
e
processed
= ve +
za + yb + xc +
wd + ue +
tf + sg + rh
r s t
u v w
x y z
Filter
a b c
d e e
f g h
Original Image
Pixels
*
37
S
p
a
t
i
a
l

C
o
r
r
e
l
a
t
i
o
n

a
n
d

C
o
n
v
o
l
u
t
i
o
n

rd
edition
38
S
p
a
t
i
a
l

C
o
r
r
e
l
a
t
i
o
n

a
n
d

C
o
n
v
o
l
u
t
i
o
n

rd
edition
39
Image Smoothing Spatial Filters
! Smoothing operation
" Average all pixels in a neighbourhood around a central value
" Especially useful
in removing noise
from images
" Useful for
highlighting gross
detail
1
/
9

1
/
9
1
/
9
1
/
9
1
/
9
1
/
9
1
/
9
1
/
9
1
/
9
Simple
averaging
filter
Smoothing poses problems for
blurring sharp edges
40
Smoothing Spatial Filtering
1
/
9
1
/
9
1
/
9
1
/
9
1
/
9
1
/
9
1
/
9
1
/
9
1
/
9
Origin x
y Image f (x, y)
e =
1
/
9
(106 +
104 + 100 + 108 +
99 + 98 +
95 + 90 +85)
= 98.3333
Filter
Simple 3*3
Neighbourhood
106
104
99
95
100 108
98
90 85
1/9
1/9
1/9
1/9
1/9
1/9
1/9
1/9
1/9
3*3 Smoothing
Filter
104 100 108
99 106 98
95 90 85
Original Image
Pixels
*
image to generate the smoothed image
Image Smoothing Example
! The image at the top left
is an original image of
size 500 x 500 pixels
! The subsequent images
show the image after
filtering with an averaging
filter of increasing sizes
" 3, 5, 9, 15 and 35
rd
edition
42
!"#$"!%
,
Weighted Smoothing Filters
! More effective smoothing filters can be generated
by allowing different pixels in the neighbourhood
different weights in the averaging function
" Pixels closer to the
central pixel are more
important
" Weighted averaging
1
/
16

2
/
16
1
/
16
2
/
16
4
/
16
2
/
16
1
/
16
2
/
16
1
/
16
Weighted
averaging filter 1/30/14 Christine P. Hendon
43
Weighted
averaging filters
1
9
1 1 1
1 1 1
1 1 1
!
"
#
#
#
$
%
&
&
&
1
16
1 2 1
2 4 2
1 2 1
!
"
#
#
#
$
%
&
&
&
1
10
1 1 1
1 2 1
1 1 1
!
"
#
#
#
$
%
&
&
&
Original Image Smoothed Image Thresholded Image
rd
edition
45
Averaging with limited data validity
! Apply image averaging only to pixels in the
original image with brightness in a predefined
interval
h x, y ( ) =
1, for f x + m, y + n ( ) ! min, max [ ]
0 otherwise
"
#
$
%
&
'
46
Non-linear operators, Rank filtering or
Ordered Statistics
! Some simple neighbourhood operations include:
" Min: Set the pixel value to the minimum in the neighbourhood
" Max: Set the pixel value to the maximum in the neighbourhood
" Median: The median value of a set of numbers is the midpoint
value in that
47
Median Filter
! Used to remove salt and pepper noise
! Main disadvantage of median filtering with
rectangular neighborhood is damaging of thin line
and sharp corners
! # change shape of neighborhood used
48
!"#$"!%
-
Rank filtering Example
123 127 128 119 115 130
140 145 148 153 167 172
133 154 183 192 194 191
194 199 207 210 198 195
164 170 175 162 173 151
Original Image
x
y
Enhanced Image
x
y
Max filter, 3x3
49
Averaging Filter Vs. Median Filter Example
! Filtering is often used to remove noise from images
! Sometimes a median filter works better than an
averaging filter
Original Image
With Noise
Image After
Averaging Filter
Image After
Median Filter
rd
edition
50
Edge Effects
! At the edges of an image we are missing pixels to
form a neighborhood
Origin x
y Image f (x, y)
e
e
e
e
e e
e
51
Dealing with Edge Effects
! There are a few approaches to dealing with missing
edge pixels:
" Omit missing pixels
" Only works with some filters
" Can add extra code and slow down processing
! Pad the image
" Typically with either all white or all black pixels
! Replicate border pixels
! Truncate the image
! Allow pixels wrap around the image
" Can cause some strange image artifacts
52
Edge Effects
Original
Image
Filtered Image: Zero
Padding
Filtered Image:
Replicate Edge Pixels
Filtered Image:
Wrap Around Edge Pixels
rd
edition
Sharpening Spatial Filters
! Sharpening spatial filters seek to highlight fine detail
! Remove blurring from images
! Highlight edges
! Sharpening filters are based on spatial
differentiation
! Edge Detectors
! Magnitude
! Gradient
54
!"#$"!%
!$
Spatial Differentiation
Differentiation measures the rate of change of a
function
Lets consider a simple 1 dimensional example
55
A B
56
First Derivative
The formula for the 1
st
derivative of a function is as
follows:

Its just the difference between subsequent values and
measures the rate of change of the function

) ( ) 1 ( x f x f
x
f
! + =
"
"
57
First Derivative
Image Strip
0
1
2
3
4
5
6
7
8
1st Derivative
-8
-6
-4
-2
0
2
4
6
8
5 5 4 3 2 1 0 0 0 6 0 0 0 0 1 3 1 0 0 0 0 7 7 7 7
0 -1 -1 -1 -1 0 0 6 -6 0 0 0 1 2 -2 -1 0 0 0 7 0 0 0
58
Second Derivative
nd
follows:

Takes into account the values both before and after
the current value

) ( 2 ) 1 ( ) 1 (
2
2
x f x f x f
x
f
! ! + + =
"
"
59
Second Derivative
Image Strip
0
1
2
3
4
5
6
7
8
5 5 4 3 2 1 0 0 0 6 0 0 0 0 1 3 1 0 0 0 0 7 7 7 7
2nd Derivative
-15
-10
-5
0
5
10
-1 0 0 0 0 1 0 6 -12 6 0 0 1 1 -4 1 1 0 0 7 -7 0 0
60
!"#$"!%
!!
Using Second Derivatives For Image Enhancement
The 2
nd
derivative is more useful for image
enhancement than the 1
st
derivative
! Stronger response to fine detail
! Simpler implementation
The first sharpening filter we will look at is the
Laplacian
! Isotropic
! One of the simplest sharpening filters
61
The Laplacian
The Laplacian is defined as follows:

where the partial 1
st
order derivative in the x direction
is defined as follows:

and in the y direction as follows:
y
f
x
f
f
2
2
2
2
2
!
!
+
!
!
= "
) , ( 2 ) , 1 ( ) , 1 (
2
2
y x f y x f y x f
x
f
! ! + + =
"
"
) , ( 2 ) 1 , ( ) 1 , (
2
2
y x f y x f y x f
y
f
! ! + + =
"
"
62
The Laplacian
So, the Laplacian can be given as follows:

Filter kernel:
) , 1 ( ) , 1 ( [
2
y x f y x f f ! + + = "
)] 1 , ( ) 1 , ( ! + + + y x f y x f
) , ( 4 y x f !
0 1 0
1 -4 1
0 1 0 1/30/14 Christine P. Hendon
63
The Laplacian Operator
h =
0 1 0
1 !4 1
0 1 0
"
#
$
$
$
%
&
'
'
'
h =
1 1 1
1 !8 1
1 1 1
"
#
$
$
$
%
&
'
'
'
h =
2 !1 2
!1 !4 !1
2 !1 2
"
#
$
$
$
%
&
'
'
'
h =
!1 2 !1
2 !4 2
!1 2 !1
"
#
$
$
$
%
&
'
'
'
4-neighborhood 8-neighborhood
Laplace stressed significance of central pixel or its neighborhood
64
The Laplacian
Applying the Laplacian to an image we get a new
image that highlights edges and other discontinuities
Original
Image
Laplacian
Filtered Image
Laplacian
Filtered Image
Scaled for Display
rd
edition
65
Image Sharpening
The result of a Laplacian filtering is not an enhanced
image
Subtract the Laplacian result from the original image
to generate our final sharpened enhanced image
Laplacian
Filtered Image
Scaled for Display
f y x f y x g
2
) , ( ) , ( ! " =
rd
edition
66
!"#$"!%
!'
Laplacian Image Enhancement
In the final sharpened image edges and fine
detail are much more obvious
-
=
Original
Image
Laplacian
Filtered Image
Sharpened
Image
rd
edition
67
rd
edition
68
Simplified Image Enhancement
The entire enhancement can be combined into a single
filtering operation
) , 1 ( ) , 1 ( [ ) , ( y x f y x f y x f ! + + ! =
) 1 , ( ) 1 , ( ! + + + y x f y x f
)] , ( 4 y x f !
f y x f y x g
2
) , ( ) , ( ! " =
) , 1 ( ) , 1 ( ) , ( 5 y x f y x f y x f ! ! + ! =
) 1 , ( ) 1 , ( ! ! + ! y x f y x f
69
This gives us a new filter which does the whole job for
us in one step
0 -1 0
-1 5 -1
0 -1 0
rd
edition
70
rd
edition
71
Variants On The Simple Laplacian
There are lots of slightly different versions of the
Laplacian that can be used:
0 1 0
1 -4 1
0 1 0
1 1 1
1 -8 1
1 1 1
-1 -1 -1
-1 9 -1
-1 -1 -1
Simple
Laplacian
Variant of
Laplacian
72
!"#$"!%
!#
1
st
Derivative Filtering
Implementing 1
st
derivative filters is difficult in
practice
For a function f(x, y) the gradient of f at coordinates
(x, y) is given as the column vector:
!
!
!
!
"
#
$
$
$
$
%
&
'
'
'
'
=
!
"
#
$
%
&
= (
y
f
x
f
G
G
y
x
f
73
1
st
The magnitude of this vector is given by:

For practical reasons this can be simplified as:

) f (! = ! mag f
[ ]
2
1
2 2
y x
G G + =
2
1
2 2
!
!
"
#
$
$
%
&
'
'
(
)
*
*
+
,
-
-
+ '
(
)
*
+
,
-
-
=
y
f
x
f
y x
G G f + ! "
74
1
st
There is some debate as to how best to calculate these
gradients but we will use:

which is based on these coordinates

( ) ( )
3 2 1 9 8 7
2 2 z z z z z z f + + ! + + " #
( ) ( )
7 4 1 9 6 3
2 2 z z z z z z + + ! + + +
z
1
z
2
z
3
z
4
z
5
z
6
z
7
z
8
z
9
75
Sobel Operators
Based on the previous equations we can derive the
Sobel Operators

To filter an image it is filtered using both operators
the results of which are added together
-1 -2 -1
0 0 0
1 2 1
-1 0 1
-2 0 2
-1 0 1
76
Sobel Example

Sobel filters are typically used for edge detection
rd
edition
77
Additional First Derivative Filters
! Roberts Operator
! Prewit Operator
h =
1 0
0 !1
"
#
$
%
&
'
h =
0 1
!1 0
"
#
$
%
&
'
h =
1 1 1
0 0 0
!1 !1 !1
"
#
$
$
$
%
&
'
'
'
h =
!1 0 1
!1 0 1
!1 0 1
"
#
$
$
$
%
&
'
'
'
h =
0 1 1
!1 0 1
!1 !1 0
"
#
$
$
$
%
&
'
'
'
78
!"#$"!%
!%
1
st
& 2
nd
Derivatives
Comparing the 1
st
and 2
nd
derivatives we can
conclude the following:
! 1
st
order derivatives generally produce thicker edges
! 2
nd
order derivatives have a stronger response to fine
detail e.g. thin lines
! 1
st
order derivatives have stronger response to grey
level step
! 2
nd
order derivatives produce a double response at
step changes in grey level
79
Matlab
80
Matlab
81
Matlab
82
Combining Spatial Enhancement Methods
! Successful image enhancement
is typically not achieved using a
single operation
! Rather we combine a range of
techniques in order to achieve a
final result
rd
edition
83
Laplacian filter of
bone scan (a)
Sharpened version of
bone scan achieved by
subtracting (a) and (b)
Sobel filter of bone
scan (a)
(a)
(b)
(c)
(d)
rd
edition
!"#$"!%
!)
The product of (c) and
(e) which will be used
as a mask
Shar pened i mage
which is sum of (a) and
(f)
Result of applying a
power-law trans. to
(g)
(e)
(f)
(g)
(h)
Image (d) smoothed with a
5*5 averaging filter Gonzalez and Woods. Digital Image Processing. 3
rd
edition
85
Compare the original and final images
rd
edition
86
Summary
87
Next Time
! Reminder: No Class or Office Hours Next Week
! Post questions regarding homework on discussion board
on class website
! Lecture 3:
! Fuzzy Techniques for Intensity Transformation and
Spatial Filtering
! Start of Chapter 4: Filtering in the Frequency Domain
88
!"#$"!%
!
E4830 DIGITAL IMAGE
PROCESSING
SPRING 2014
Intensity Transformations and
Spatial Filtering
Lecture 2
January 30, 2014
1/30/14
Lecture 1 Review
2
Lecture Overview
3
Image Pre-processing
! Image enhancement
! Image restoration
4
The Basics of Intensity Transformations
and Spatial Filtering
! Spatial vs. Fourier Domain
! Spatial domain: plane containing the pixels
of an image
! Spatial filtering
window
g x, y ( ) = T f x, y ( ) !
"
#
$
input output
operator
s = T r ( )
rd
edition
5
Advantages of Spatial Filtering
! Linear and non-linear filtering
6
!"#$"!%
'
Brightness Correction and
Grayscale Transformations
! Position depdent brightness correction
! Gray scale transformations
! Example: Negative images are useful for enhancing
white or grey detail embedded in dark regions of an
image Original
Image
Negative
Image
s = L !1! r
g x, y ( ) = f x, y ( )e x, y ( )
rd
edition
7
Point Processing
Example: Thresholding
! Thresholding transformations are particularly useful
for segmentation in which we want to isolate an
object of interest from a background
s =
1.0
0.0 r <= threshold
r > threshold
rd
edition
8
Basic Grey Level Transformations
! There are many different kinds of grey level
transformations
" Linear
" Negative/Identity
" Logarithmic
" Log/Inverse log
" Power law
" n
th
power/n
th
root
rd
edition
9
! The log transformation maps a narrow range of low
input grey level values into a wider range of output
values
! The inverse log transformation performs the
opposite transformation
s = clog 1+ r ( )
10
! Log functions are particularly useful when the input
grey level values may have an extremely large
range of values
s = log(1 + r)
Usually set c to 1
rd
edition
11
! Map a narrow range
of dark input values
into a wider range of
output values or vice
versa
! Varying ( gives a whole
family of curves
s = cr
!
Usually set c to 1
rd
edition
12
!"#$"!%
#
Gamma Correction
! A variety of devices used
for image capture,
printing and display
respond according to a
power law
! Exponent of power-law
equation called gamma
! Process used to correct
power-law response
called gamma correction
rd
edition
13
Power Law Example
! Magnetic
resonance
(MR) image of a
fractured human
spine
! Different curves
highlight different
detail
s = r
0.6

s
=
r
0
.3

0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.2 0.4 0.6 0.8 1
Transform
ed Intensities
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.2 0.4 0.6 0.8 1
Transform
ed Intensities
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.2 0.4 0.6 0.8 1
Old Intensities
Transform
ed Intensities
s

=

r

0
.4

rd
edition
! Power law
transforms are
used to darken
the image
! Different curves
highlight
different detail
s = r
3.0

s

=

r

4
.0

s
=
r
5
.0

rd
edition
15
Piecewise Linear Transformation
Functions
! Rather than using a well defined mathematical
function we can use arbitrary user-defined
transforms
! Contrast stretching linear transform to add contrast
to a poor quality image
rd
edition
16
Intensity Level Slicing
!Highlights a specific range of grey levels
! Similar to thresholding
! Other levels can be
suppressed or maintained
! Useful for highlighting features
in an image
rd
edition
17
Bit Plane Slicing
! Often by isolating particular bits of the pixel values
in an image we can highlight interesting aspects of
that image
" Higher-order bits usually contain most of the significant visual
information
" Lower-order bits contain
subtle details
rd
edition
18
!"#$"!%
%
Bit Plane Slicing
rd
edition
19
Bit Plane Slicing
only bit planes 8 and 7
rd
edition
20
Image Histograms
! The histogram of an image shows us the distribution
of grey levels in the image
rd
edition
21
Contrast Stretching
! We can fix images that have poor contrast by
applying a pretty simple contrast specification
! The interesting part is how do we decide on this
transformation function?
22
Spreading out the frequencies in an image (or
equalising the image) is a simple way to improve dark
or washed out images
The formula for histogram
equalization is given where
! r
k
: input intensity
! s
k
: processed intensity
! k: the intensity range
(e.g 0.0 1.0)
! n
j
: the frequency of intensity j
! n: the sum of all frequencies
) (
k k
r T s =
!
=
=
k
j
j r
r p
1
) (
!
=
=
k
j
j
n
n
1
23
Histogram of Different
Types of Images
! A selection of images and
their histograms
! Notice the relationships
between the images and
their histograms
rd
edition
24
!"#$"!%
)
! Note that the high
contrast image has
an evenly spaced
histogram
High contrast images
rd
edition
25
Equalisation Transformation Function
rd
edition
26
1
rd
edition
27
2
rd
edition
28
3
4
rd
edition
29
Histogram Equalization and
Specification
Histogram specification Histogram equalization
rd
edition
30
!"#$"!%
*
Matlab function: adapthisteq
! Contrast limited adaptive histogram equalization
! Processing small regions of the image called tiles using
histogram specification for each tile individually
! Neighboring tiles combined using bilinear interpolation
to eliminate artificially induced boundaries
rd
edition
31
32
Local Pre-Processing or Image
Enhancement
! Objective: use small neighborhood of a pixel in an
input image to produce new brightness value in
output image # filtering
! Smoothing
! Gradient Operations
33
! Neighbourhood operations simply operate on a
larger neighborhood of pixels than point operations
! Neighborhoods are
mostly a rectangle
around a central pixel
! Any size rectangle
and any shape filter
are possible
Origin x
y Image f (x, y)
(x, y)
Neighbourhood
34
The Spatial Filtering Process
r s t
u v w
x y z
Origin x
y Image f (x, y)
e
processed
= ve +
ra + sb + tc +
ud + wf +
xg + yh + zi
Filter
Simple 3*3
Neighbourhood
e 3*3 Filter
a b c
d e f
g h i
Original Image
Pixels
*
image to generate the filtered image
35
M
e
c
h
a
n
i
c
s

o
f

S
p
a
t
i
a
l

F
i
l
t
e
r
i
n
g

!!
" = " =
+ + =
a
a s
b
b t
t y s x f t s w y x g ) , ( ) , ( ) , (
! Correlation
! Convolution
! Zero-patting
! Cropped
results
rd
edition
!"#$"!%
+
Correlation and Convolution
! The filtering we have been talking about so far is
referred to as correlation with the filter itself referred
to as the correlation kernel
! Convolution is a similar operation, with just one subtle
difference
! For symmetric filters it makes no difference
e
processed
= ve +
za + yb + xc +
wd + ue +
tf + sg + rh
r s t
u v w
x y z
Filter
a b c
d e e
f g h
Original Image
Pixels
*
37
S
p
a
t
i
a
l

C
o
r
r
e
l
a
t
i
o
n

a
n
d

C
o
n
v
o
l
u
t
i
o
n

rd
edition
38
S
p
a
t
i
a
l

C
o
r
r
e
l
a
t
i
o
n

a
n
d

C
o
n
v
o
l
u
t
i
o
n

rd
edition
39
Image Smoothing Spatial Filters
! Smoothing operation
" Average all pixels in a neighbourhood around a central value
" Especially useful
in removing noise
from images
" Useful for
highlighting gross
detail
1
/
9

1
/
9
1
/
9
1
/
9
1
/
9
1
/
9
1
/
9
1
/
9
1
/
9
Simple
averaging
filter
Smoothing poses problems for
blurring sharp edges
40
Smoothing Spatial Filtering
1
/
9
1
/
9
1
/
9
1
/
9
1
/
9
1
/
9
1
/
9
1
/
9
1
/
9
Origin x
y Image f (x, y)
e =
1
/
9
(106 +
104 + 100 + 108 +
99 + 98 +
95 + 90 +85)
= 98.3333
Filter
Simple 3*3
Neighbourhood
106
104
99
95
100 108
98
90 85
1/9
1/9
1/9
1/9
1/9
1/9
1/9
1/9
1/9
3*3 Smoothing
Filter
104 100 108
99 106 98
95 90 85
Original Image
Pixels
*
image to generate the smoothed image
! The image at the top left
is an original image of
size 500 x 500 pixels
! The subsequent images
show the image after
filtering with an averaging
filter of increasing sizes
" 3, 5, 9, 15 and 35
rd
edition
42
!"#$"!%
,
! More effective smoothing filters can be generated
by allowing different pixels in the neighbourhood
different weights in the averaging function
" Pixels closer to the
central pixel are more
important
" Weighted averaging
1
/
16

2
/
16
1
/
16
2
/
16
4
/
16
2
/
16
1
/
16
2
/
16
1
/
16
Weighted
averaging filter 1/30/14 Christine P. Hendon
43
Weighted
averaging filters
1
9
1 1 1
1 1 1
1 1 1
!
"
#
#
#
$
%
&
&
&
1
16
1 2 1
2 4 2
1 2 1
!
"
#
#
#
$
%
&
&
&
1
10
1 1 1
1 2 1
1 1 1
!
"
#
#
#
$
%
&
&
&
Original Image Smoothed Image Thresholded Image
rd
edition
45
Averaging with limited data validity
! Apply image averaging only to pixels in the
original image with brightness in a predefined
interval
h x, y ( ) =
1, for f x + m, y + n ( ) ! min, max [ ]
0 otherwise
"
#
$
%
&
'
46
Non-linear operators, Rank filtering or
Ordered Statistics
! Some simple neighbourhood operations include:
" Min: Set the pixel value to the minimum in the neighbourhood
" Max: Set the pixel value to the maximum in the neighbourhood
" Median: The median value of a set of numbers is the midpoint
value in that
47
Median Filter
! Used to remove salt and pepper noise
! Main disadvantage of median filtering with
rectangular neighborhood is damaging of thin line
and sharp corners
! # change shape of neighborhood used
48
!"#$"!%
-
Rank filtering Example
123 127 128 119 115 130
140 145 148 153 167 172
133 154 183 192 194 191
194 199 207 210 198 195
164 170 175 162 173 151
Original Image
x
y
Enhanced Image
x
y
Max filter, 3x3
49
averaging filter
Original Image
With Noise
Image After
Averaging Filter
Image After
Median Filter
rd
edition
50
Edge Effects
! At the edges of an image we are missing pixels to
form a neighborhood
Origin x
y Image f (x, y)
e
e
e
e
e e
e
51
Dealing with Edge Effects
! There are a few approaches to dealing with missing
edge pixels:
" Omit missing pixels
" Only works with some filters
" Can add extra code and slow down processing
! Pad the image
" Typically with either all white or all black pixels
! Replicate border pixels
! Truncate the image
! Allow pixels wrap around the image
" Can cause some strange image artifacts
52
Edge Effects
Original
Image
Filtered Image: Zero
Padding
Filtered Image:
Replicate Edge Pixels
Filtered Image:
Wrap Around Edge Pixels
rd
edition
Sharpening Spatial Filters
! Sharpening spatial filters seek to highlight fine detail
! Remove blurring from images
! Highlight edges
! Sharpening filters are based on spatial
differentiation
! Edge Detectors
! Magnitude
! Gradient
54
!"#$"!%
!$
Differentiation measures the rate of change of a
function
Lets consider a simple 1 dimensional example
55
A B
56
First Derivative
st
follows:

Its just the difference between subsequent values and
measures the rate of change of the function

) ( ) 1 ( x f x f
x
f
! + =
"
"
57
First Derivative
Image Strip
0
1
2
3
4
5
6
7
8
1st Derivative
-8
-6
-4
-2
0
2
4
6
8
5 5 4 3 2 1 0 0 0 6 0 0 0 0 1 3 1 0 0 0 0 7 7 7 7
0 -1 -1 -1 -1 0 0 6 -6 0 0 0 1 2 -2 -1 0 0 0 7 0 0 0
58
Second Derivative
nd
follows:

Takes into account the values both before and after
the current value

) ( 2 ) 1 ( ) 1 (
2
2
x f x f x f
x
f
! ! + + =
"
"
59
Second Derivative
Image Strip
0
1
2
3
4
5
6
7
8
5 5 4 3 2 1 0 0 0 6 0 0 0 0 1 3 1 0 0 0 0 7 7 7 7
2nd Derivative
-15
-10
-5
0
5
10
-1 0 0 0 0 1 0 6 -12 6 0 0 1 1 -4 1 1 0 0 7 -7 0 0
60
!"#$"!%
!!
Using Second Derivatives For Image Enhancement
The 2
nd
derivative is more useful for image
enhancement than the 1
st
derivative
! Stronger response to fine detail
! Simpler implementation
The first sharpening filter we will look at is the
Laplacian
! Isotropic
! One of the simplest sharpening filters
61
The Laplacian
The Laplacian is defined as follows:

where the partial 1
st
order derivative in the x direction
is defined as follows:

and in the y direction as follows:
y
f
x
f
f
2
2
2
2
2
!
!
+
!
!
= "
) , ( 2 ) , 1 ( ) , 1 (
2
2
y x f y x f y x f
x
f
! ! + + =
"
"
) , ( 2 ) 1 , ( ) 1 , (
2
2
y x f y x f y x f
y
f
! ! + + =
"
"
62
The Laplacian
So, the Laplacian can be given as follows:

Filter kernel:
) , 1 ( ) , 1 ( [
2
y x f y x f f ! + + = "
)] 1 , ( ) 1 , ( ! + + + y x f y x f
) , ( 4 y x f !
0 1 0
1 -4 1
0 1 0 1/30/14 Christine P. Hendon
63
The Laplacian Operator
h =
0 1 0
1 !4 1
0 1 0
"
#
$
$
$
%
&
'
'
'
h =
1 1 1
1 !8 1
1 1 1
"
#
$
$
$
%
&
'
'
'
h =
2 !1 2
!1 !4 !1
2 !1 2
"
#
$
$
$
%
&
'
'
'
h =
!1 2 !1
2 !4 2
!1 2 !1
"
#
$
$
$
%
&
'
'
'
4-neighborhood 8-neighborhood
Laplace stressed significance of central pixel or its neighborhood
64
The Laplacian
Applying the Laplacian to an image we get a new
image that highlights edges and other discontinuities
Original
Image
Laplacian
Filtered Image
Laplacian
Filtered Image
Scaled for Display
rd
edition
65
Image Sharpening
The result of a Laplacian filtering is not an enhanced
image
Subtract the Laplacian result from the original image
to generate our final sharpened enhanced image
Laplacian
Filtered Image
Scaled for Display
f y x f y x g
2
) , ( ) , ( ! " =
rd
edition
66
!"#$"!%
!'
In the final sharpened image edges and fine
detail are much more obvious
-
=
Original
Image
Laplacian
Filtered Image
Sharpened
Image
rd
edition
67
rd
edition
68
The entire enhancement can be combined into a single
filtering operation
) , 1 ( ) , 1 ( [ ) , ( y x f y x f y x f ! + + ! =
) 1 , ( ) 1 , ( ! + + + y x f y x f
)] , ( 4 y x f !
f y x f y x g
2
) , ( ) , ( ! " =
) , 1 ( ) , 1 ( ) , ( 5 y x f y x f y x f ! ! + ! =
) 1 , ( ) 1 , ( ! ! + ! y x f y x f
69
This gives us a new filter which does the whole job for
us in one step
0 -1 0
-1 5 -1
0 -1 0
rd
edition
70
rd
edition
71
Variants On The Simple Laplacian
There are lots of slightly different versions of the
Laplacian that can be used:
0 1 0
1 -4 1
0 1 0
1 1 1
1 -8 1
1 1 1
-1 -1 -1
-1 9 -1
-1 -1 -1
Simple
Laplacian
Variant of
Laplacian
72
!"#$"!%
!#
1
st
Implementing 1
st
derivative filters is difficult in
practice
For a function f(x, y) the gradient of f at coordinates
(x, y) is given as the column vector:
!
!
!
!
"
#
$
$
$
$
%
&
'
'
'
'
=
!
"
#
$
%
&
= (
y
f
x
f
G
G
y
x
f
73
1
st
The magnitude of this vector is given by:

For practical reasons this can be simplified as:

) f (! = ! mag f
[ ]
2
1
2 2
y x
G G + =
2
1
2 2
!
!
"
#
$
$
%
&
'
'
(
)
*
*
+
,
-
-
+ '
(
)
*
+
,
-
-
=
y
f
x
f
y x
G G f + ! "
74
1
st
There is some debate as to how best to calculate these
gradients but we will use:

which is based on these coordinates

( ) ( )
3 2 1 9 8 7
2 2 z z z z z z f + + ! + + " #
( ) ( )
7 4 1 9 6 3
2 2 z z z z z z + + ! + + +
z
1
z
2
z
3
z
4
z
5
z
6
z
7
z
8
z
9
75
Sobel Operators
Based on the previous equations we can derive the
Sobel Operators

To filter an image it is filtered using both operators
the results of which are added together
-1 -2 -1
0 0 0
1 2 1
-1 0 1
-2 0 2
-1 0 1
76
Sobel Example

Sobel filters are typically used for edge detection
rd
edition
77
Additional First Derivative Filters
! Roberts Operator
! Prewit Operator
h =
1 0
0 !1
"
#
$
%
&
'
h =
0 1
!1 0
"
#
$
%
&
'
h =
1 1 1
0 0 0
!1 !1 !1
"
#
$
$
$
%
&
'
'
'
h =
!1 0 1
!1 0 1
!1 0 1
"
#
$
$
$
%
&
'
'
'
h =
0 1 1
!1 0 1
!1 !1 0
"
#
$
$
$
%
&
'
'
'
78
!"#$"!%
!%
1
st
& 2
nd
Derivatives
Comparing the 1
st
and 2
nd
derivatives we can
conclude the following:
! 1
st
order derivatives generally produce thicker edges
! 2
nd
order derivatives have a stronger response to fine
detail e.g. thin lines
! 1
st
order derivatives have stronger response to grey
level step
! 2
nd
order derivatives produce a double response at
step changes in grey level
79
Matlab
80
Matlab
81
Matlab
82
! Successful image enhancement
is typically not achieved using a
single operation
! Rather we combine a range of
techniques in order to achieve a
final result
rd
edition
83
Laplacian filter of
bone scan (a)
Sharpened version of
bone scan achieved by
subtracting (a) and (b)
Sobel filter of bone
scan (a)
(a)
(b)
(c)
(d)
rd
edition
!"#$"!%
!)
The product of (c) and
(e) which will be used
as a mask
Shar pened i mage
which is sum of (a) and
(f)
Result of applying a
power-law trans. to
(g)
(e)
(f)
(g)
(h)
Image (d) smoothed with a
5*5 averaging filter Gonzalez and Woods. Digital Image Processing. 3
rd
edition
85
Compare the original and final images
rd
edition
86
Summary
87
Next Time
! Reminder: No Class or Office Hours Next Week
! Post questions regarding homework on discussion board
on class website
! Lecture 3:
! Fuzzy Techniques for Intensity Transformation and
Spatial Filtering
! Start of Chapter 4: Filtering in the Frequency Domain
88
2/14/14
1
E4830 DIGITAL IMAGE PROCESSING
SPRING 2014
Frequency Domain Filtering
Lecture 3
February 13, 2014
2/14/14
Lecture 2 Review
2
rd
edition
Todays Lecture
! Fourier Transform Review
! Sampling and the Fourier Transform
! The Discrete Time Fourier Transform
! The Two Dimensional Fourier Transform
! Filtering Basics
Chapter 4
4
rd
edition
The Big Idea
=
Any function that periodically repeats itself can be expressed as
a sum of sines and cosines of different frequencies each
multiplied by a different coefficient a Fourier series
6
rd
edition
2/14/14
2
Fourier Series
7

f t ( ) = c
n
e
j
2!n
T
t
n="#
#
$
c
n
=
1
T
f t ( )
"
T
2
T
2
%
e
" j
2!n
T
t
dt
for n=0, 1, 2,
Impulse and Their Sifting Property
8
! t ( ) =
" if t = 0
0 if t # 0
$
%
&
'
(
)
! t ( )dt
*"
"
+
=1
f t ( )! t ( )dt
*"
"
+
= f 0 ( )
f t ( )! t * t
0
( )dt
*"
"
+
= f t
0
( )
! x ( ) =
1 if x = 0
0 if x " 0
#
$
%
&
'
(
! x ( )
x=)*
*
+
=1
f x ( )! x ( )
x=)*
*
+
= f 0 ( )
f x ( )! x ) x
0
( )
x=)*
*
+
= f x
0
( )
s
!T
t ( ) = " x # n!T ( )
n=#$
$
%
rd
edition
The Fourier Transform of Functions of
One Continuous Variable
! Fourier Transform
! Fourier Spectrum of Frequency Spectrum
9
F ( ) = F f t ( ) { } = f t ( )
!"
"
#
e
! j 2$t
dt
f t ( ) % F ( )
rd
edition
Convolution
10
f t ( )!h t ( ) = f " ( )
#$
$
%
h t #" ( )d"
f t ( )!h t ( ) & H ( )F ( )
f t ( )h t ( ) & H ( )!F ( )
Sampling
11
! Sampling
! Continuous functions converted into
sequence of discrete values
! Sampled function

!
f t ( ) = f t ( )s
!T
t ( )
!
f t ( ) = f t ( )d t " n!T ( )
n="#
#
$
!T uniform intervals
rd
edition
Fourier Transform of Sampled
Functions
12

f t ( ) ! F ( )
!
f t ( ) = f t ( )" impulse train
" using convolution theorem
!
F ( ) = F f t ( )s
#T
t ( ) { }
!
F ( ) = F ( )$S ( )
where
S ( ) =
1
#T
% &
n
#T
'
(
)
*
+
,
n=&-
-
.
/
!
F ( ) =
1
#T
F &
n
#T
'
(
)
*
+
,
n=&-
-
.
rd
edition
2/14/14
3
Functions
! is sampled function
! is continuous because it consists of
copies of F() which is a continuous
function
! Sampling rate needs to be high
enough to provide sufficient
separation between periods to
preserve integrity of F()
a. Over-sampled signal
b. Critically sampled signal
c. Under-sampled signal
13

!
f t ( ) !
!
F ( )

!
f t ( )

!
F ( )
1
!T
rd
edition
The Sampling Theorem
! Establish conditions under which a continuous function can
be recovered uniquely from a set of its samples
! Bandlimited, f(t)
! Whose Fourier transform is zero for frequencies outside a
finite interval (band), [-
max
,
max
]
! f(t) can be recovered from sampled version if we can
isolate F() from periodic sequence
14
! No information is loss if continuous
bandlimited function represented by
samples acquired at rate higher
than twice frequency
! Or
! Max frequency that can be
captured by sampling a signal at a
rate
15
1
!T
> 2
max
1
!T
is
max
=
1
2!T
rd
edition
Recover F(!)
! Low pass filter
16

H ( ) =
!T -
max
" "
max
0 otherwise
#
$
%
&
'
(
F ( ) = H ( )
!
F ( )
f t ( ) = F
)*
*
+
( )e
j 2,t
d
rd
edition
Aliasing
! What happens if bandlimited
function sampled at rate less than
twice its highest frequency
! Undersampling
will contain periods that
overlap
! Aliasing always present in sampled
signals
" Infinite frequency components
introduced once we limit duration of
the signal
17

!
F ( )
rd
edition
Fourier transform of an under-
sampled, band-limited function
18
rd
edition
2/14/14
4
Relationship between the Sampling
and Frequency Interval
! Discrete equivalent
19
Sampled data reconstruction
20
rd
edition
Extension to Functions of Two Variables
! The 2-D Impulse and Its Sifting Property
! The 2-D Continuous Fourier Transform Pair
21
rd
edition
Two Dimensional Sampling and the 2-
D Sampling Theorem
! Two-dimensional sampling theorem
! A continuous, band-limited function f(t,z) can be recovered
with no error from a set of its samples if the sampling
intervals are
22
!T <
1
2
max
and
!Z <
1
2"
max
rd
edition
Aliasing in Images
! f(t,z) bandlimited only if it extends infinitely in both
coordinate directions
! Aliasing always present in digital images
! We can not infinitely sample
! Spatial aliasing (focus of class)
! Temporal aliasing related to time interval between
images (wagon wheel)
23
rd
edition
Aliasing in Images
! Let imaging system have 96 x 96 pixels

! Aliasing can crease results that look misleading
24
Squares size
16 pixels
Squares size
less than 1
pixel
Squares size
less than 0.5
pixel
Squares size
6 pixels
rd
edition
2/14/14
5
Anti-Aliasing
! Has to occur before image is sampled
! Blurred or defocused to reduce high frequency content
25
Image Interpolation and Resampling
! Effect of aliasing generally worse as size of image reduced
! Perfect reconstruction of bandlimited images from set of samples
requires 2D convolution in spatial domain with sinc function
! 2D interpolation and zooming
! Zooming viewed as oversampling
! Shrinking viewed as undersampling
26
rd
edition
Jaggies
27
rd
edition
Image Zooming
rd
edition
28
Moire Effect
! Moire patterns: type of artifact
! Results from sampling scences with
periodic or nearly periodic compoennts
! Occurs in images with periodic
components whose spacing is
comparable to spacing between
samples
! Sampling artifact
rd
edition
29
Moire Pattern
rd
edition
30
2/14/14
6
Properties of the 2-D Discrete Fourier
Transform
! Relationship between spatial and frequency intervals
! Translation and Rotation
! Periodicity
! Symmetry Properties
31
C
e
n
t
e
r
i
n
g

t
h
e

F
o
u
r
i
e
r

T
r
a
n
s
f
o
r
m

rd
edition
Symmetry Properties of the 2-D DFT
and Invserse
rd
edition
33
Fourier Spectrum and Phase Angle
rd
edition
34
Fourier Spectrum Examples
rd
edition
35
Phase Angle Examples
rd
edition
36
2/14/14
7
2-D Convolution Theorem
rd
edition
37
Summary of Discrete Fourier Transform
Properties
rd
edition
38
Summary of DFT
rd
edition
39
rd
edition
rd
edition
! Filtering in frequency domain consists of modifying the
Fourier Transform of an image and computing the
inverse transform to obtain a processed result
Basics of Filtering in the Frequency
Domain
rd
edition
42
2/14/14
8
rd
edition
rd
edition
rd
edition
rd
edition
rd
edition
rd
edition
2/14/14
9
Summary of Steps for Filtering in the
Frequency Domain
49
Correspondence between filtering in
the spatial and frequency domains
rd
edition
50
Gaussian Filter Functions
rd
edition
51
rd
edition
Obtaining a frequency domain filter
from a small spatial mask
rd
edition
53
! Ideal Low Pass Filters
Image Smoothing Using Frequency
Domain Filters
rd
edition
54
2/14/14
10
rd
edition
Butterworth Lowpass Filters
rd
edition
56
Filtering with BLPFs
rd
edition
57
BLPFs
rd
edition
58
Guassian Lowpass Filters
rd
edition
59
rd
edition
2/14/14
11
Examples of Lowpass Filtering
rd
edition
61
rd
edition
62
Example Filtering with GLPF
rd
edition
63
Example Filtering with GLPF
rd
edition
64
Image Sharpening using Frequency
Domain Filters
! Highpass filter from a low pass filter
! Ideal Highpass Filters
rd
edition
65
Spatial Representation of Highpass
Filters
rd
edition
66
2/14/14
12
Result of Highpass filtering
rd
edition
67
Butterworth Highpass Filters
rd
edition
68
Gaussian Highpass Filtering
rd
edition
69
rd
edition
The Laplacian in the Frequency
Domain
rd
edition
71
Unsharp Masking, Highboost Filtering,
and High-Frequency Emphasis Filtering
rd
edition
72
2/14/14
13
! Illumination-reflection model can be
used to develop frequency domain
procedure for improved appearance
of image by simultaneous intensity
range compression and contrast
enhancement
Homomorphic Filtering
rd
edition
73
Steps in homomorphic filtering
rd
edition
74
Enhancement using homomorphic
filtering
rd
edition
75
rd
edition
! Bandreject and Bandpass filters
! Notch Filters
Selective Filtering
rd
edition
77
Example filtering with notch filter
rd
edition
78
2/14/14
14
Example filtering with notch reject
filter
rd
edition
79
Fast Fourier Transform
! The reason that Fourier based
techniques have become so popular
is the development of the Fast
Fourier Transform (FFT) algorithm
! Allows the Fourier transform to be
carried out in a reasonable amount
of time
! Reduces the amount of time required
to perform a Fourier transform by a
factor of 100 600 times!
80
rd
edition
Frequency Domain Filtering & Spatial
Domain Filtering
! Similar jobs can be done in the spatial and frequency
domains
! Filtering in the spatial domain can be easier to
understand
! Filtering in the frequency domain can be much faster
especially for large images
81
Summary
! In this lecture we examined image enhancement in the
frequency domain
! The Fourier series & the Fourier transform
! Image Processing in the frequency domain
" Image smoothing
" Image sharpening
! Fast Fourier Transform
82
Next Time
! Spatial Filtering with Fuzzy Techniques (Chapter 3)
! More Frequency and Spatial Domain Filtering Examples
83
SPRING 2014
Spatial and Frequency Domain
Filtering
Lecture 4
February 20, 2014
2/21/14
Image Enhancement
! The principal objective of enhancement is to process an
image so hat the result is more suitable than the orignial
image for a specific application
! In Spatial Domain
! Direct manipulation of pixels in an image
! In Frequency Domain
! Modify the Fourier Transform of an image
2
rd
edition
Spatial domain filtering operations
Pre-
processing
h(x,y)
Post-
processing
g(x,y)=h(x,y)*f(x,y)
f(x,y)
In practice
! Intuition about what makes a good image
! A uniform histogram spanning a large variety of gray
levels
4
Spatial Filtering and Intensity
Transformation Review
2/21/14
5
Christine P. Hendon
The Basics of Intensity Transformations and
Spatial Filtering
! Spatial domain: plane containing the pixels of
an image
! Spatial filtering
window
g x, y ( ) = T f x, y ( ) !
"
#
$
input
output
operator
s = T r ( )
rd
edition
6
Contrast Stretching and Intensity
Transformations
rd
edition
7
rd
edition
Intensity transformation with MATLAB:
imadjust
8
! Log functions are particularly useful when the input grey
level values may have an extremely large range of
values
s = log(1 + r)
Usually set c to 1
rd
edition
9
M
e
c
h
a
n
i
c
s

o
f

S
p
a
t
i
a
l

F
i
l
t
e
r
i
n
g

!!
" = " =
+ + =
a
a s
b
b t
t y s x f t s w y x g ) , ( ) , ( ) , (
! Correlation
! Convolution
! Zero-patting
! Cropped
results
rd
edition
S
p
a
t
i
a
l

C
o
r
r
e
l
a
t
i
o
n

a
n
d

C
o
n
v
o
l
u
t
i
o
n

rd
edition
11
S
p
a
t
i
a
l

C
o
r
r
e
l
a
t
i
o
n

a
n
d

C
o
n
v
o
l
u
t
i
o
n

rd
edition
12
Matlab
13
Matlab function:
imfilter
rd
edition
Imfilter with
default zero
padding replication
symmetric circular Uint8 then
replication
14
M
a
t
l
a
b

F
u
n
c
t
i
o
n
:

f
s
p
e
c
i
a
l

rd
edition
Enhancement with the Laplacian Filter
y
f
x
f
f
2
2
2
2
2
!
!
+
!
!
= "
) , 1 ( ) , 1 ( [
2
y x f y x f f ! + + = "
)] 1 , ( ) 1 , ( ! + + + y x f y x f
) , ( 4 y x f !
h =
0 1 0
1 !4 1
0 1 0
"
#
$
$
$
%
&
'
'
'
h =
2 !1 2
!1 !4 !1
2 !1 2
"
#
$
$
$
%
&
'
'
'
h =
!1 2 !1
2 !4 2
!1 2 !1
"
#
$
$
$
%
&
'
'
'
h =
1 1 1
1 !8 1
1 1 1
"
#
$
$
$
%
&
'
'
'
rd
edition
averaging filter
Original Image
With Noise
Image After
Averaging Filter
Image After
Median Filter
rd
edition
17
Median Filtering
rd
edition
Image with salt
and pepper
noise
Median filtering
with function
medfilt2 with
default settings
Median filtering
with function
medfilt2 with
option
symmetric
18
Fuzzy Sets
2/21/14
19
Christine P. Hendon
Fuzzy Sets
20
! Membership Functions
Fuzzy Sets: Definitions
Fuzzy logic doesnt equal probability
21
Definitions
! Empty
! Equality
! Complement
! Subset
! Union
! Intersection
22
Membership Functions of Two Fuzzy
Sets
23
C
o
m
m
o
n
l
y

U
s
e
d

M
e
m
b
e
r
s
h
i
p

F
u
n
c
t
i
o
n
s

! Problem statement:

! Express as Fuzzy IF THEN rules
! R1: if the frequency is low, then
motor operation is normal. OR
! R2: if the frequency is mid then
motor operation is marginal. OR
! R3: if the frequency is high, then
motor operation is near failure
Using Fuzzy Sets
Fuzzifying a specific measurement, z
0 2/21/14 Christine P. Hendon
25
! Use inputs and knowledge base from IF THEN rules to
create outputs for fuzzy system
! Because we are dealing with fuzzy inputs, outputs are
also fuzzy
! Membership functions need to be defined for outputs
! Example: % of abnormality for motor
Implication or Inference
26
27
Complete output,
Q1, Q2, Q3
corresponding to
specific input
! Last step " obtain a crisp output, v
o
, from fuzzy set Q
! One method: compute center of gravity of Q(v)
! i.e. for z
0
= 0.7, v
0
= 0.76 " 76% degree of
abnormality
Defuzzification
v
0
=
vQ v ( )dv
!
Q v ( )dv
!
29
Principal Steps
! Fuzzify the inputs
! Perform any required fuzzy logical operations
! Apply an implication method
! Apply an aggregation method to the fuzzy set
from step 3
! Defuzzify the final fuzzy set
30
! Rule: if frequency low and the temperature is moderate,
then motor is normal
! And " Min
Multiple Variables
! IF (z
1
, A
11
) AND (z
2
, A
12
) ANDAND (z
N
, A
1N
) THEN (v, B
1
)
! ..
! IF (z
1
, A
M1
) AND (z
2
, A
M2
) ANDAND (z
N
, A
MN
) THEN (v, B
M
)
! ELSE (v, B
E
)
32

i
th
rule
!
i
= min
Aij
z
j
( )
; j =1, 2,, N
{ }
else rule
NOT ( A)
z ( ) =
A
z ( ) =1"
A
z ( )
!
E
= min 1" !
i
;i =1, 2,, M
{ }
! If a pixel is dark, then
make it darker
! If a pixel is gray, then
make it gray
! If a pixel is bright, then
make it brighter
! " Increasing separation
of dark and light which
increasing contrast
Fuzzy Sets for Intensity Transformations
33
Input and Output Membership
Functions for Contrast Enhancement
! For rule based contrast enhancement
Input Output
34
Histogram Equalization vs Fuzzy Rule-
Based Contrast Enhancement
Low contrast Result of histogram
equalization
Result of fuzzy rule based
contrast enhancement
35
Kept same basic
characteristics of
the original image
! Price of improvement in image quality is increased
processing complexity
! Define fuzzy neighborhood properties the capture the
essence of what filters are supposed to detect
! Example: if a pixel belongs to uniform region, then make it
white else make it black

! Rules:
! If d2 is zero AND d6 is zero then z5 is white
! If d4 is zero and d2 is zero then z5 is white
! Else z5 is black
Fuzzy Sets for Spatial Filtering
37
Example: Membership Function for
Spatial Filtering
Fuzzy set for Zero
Membership functions for Black
and White
38
Fuzzy Rules for Boundary Detection
39
Example: Fuzzy Spatial Filtering
With rules:
Result of fuzzy spatial filtering
With intensity scaling
40
Frequency Domain Filtering Review
2/21/14
41
Christine P. Hendon
42
The Big Idea
=
Any function that periodically repeats itself can be expressed as
a sum of sines and cosines of different frequencies each
multiplied by a different coefficient a Fourier series
43
rd
edition
Match 2D Fourier Transform Pairs
Function 2D Fourier Transform
a
f
g
e
d
c
b
h
1
2
4
6
7
8
5
3
Answers
http://dukemil.egr.duke.edu/Ultrasound/k-space/node3.html
Impulse and Their Sifting Property
46
! t ( ) =
" if t = 0
0 if t # 0
$
%
&
'
(
)
! t ( )dt
*"
"
+
=1
f t ( )! t ( )dt
*"
"
+
= f 0 ( )
f t ( )! t * t
0
( )dt
*"
"
+
= f t
0
( )
! x ( ) =
1 if x = 0
0 if x " 0
#
$
%
&
'
(
! x ( )
x=)*
*
+
=1
f x ( )! x ( )
x=)*
*
+
= f 0 ( )
f x ( )! x ) x
0
( )
x=)*
*
+
= f x
0
( )
s
!T
t ( ) = " x # n!T ( )
n=#$
$
%
rd
edition
Convolution
47
f t ( )!h t ( ) = f " ( )
#$
$
%
h t #" ( )d"
f t ( )!h t ( ) & H ( )F ( )
f t ( )h t ( ) & H ( )!F ( )
F ( ) = F f t ( )
{ }
= f t ( )
!"
"
#
e
! j 2$t
dt
f t ( ) % F ( )
Sampling
48
! Sampling
! Continuous functions converted into
sequence of discrete values
! Sampled function

!
f t ( ) = f t ( )s
!T
t ( )
!
f t ( ) = f t ( )d t " n!T ( )
n="#
#
$
!T uniform intervals
rd
edition
Functions
49

f t ( ) ! F ( )
!
f t ( ) = f t ( )" impulse train
" using convolution theorem
!
F ( ) = F f t ( )s
#T
t ( )
{ }
!
F ( ) = F ( )$S ( )
where
S ( ) =
1
#T
% &
n
#T
'
(
)
*
+
,
n=&-
-
.
/
!
F ( ) =
1
#T
F &
n
#T
'
(
)
*
+
,
n=&-
-
.
rd
edition
Functions
! is sampled function
! is continuous because it consists of
copies of F() which is a continuous
function
! Sampling rate needs to be high
enough to provide sufficient
separation between periods to
preserve integrity of F()
a. Over-sampled signal
b. Critically sampled signal
c. Under-sampled signal
50

!
f t ( ) !
!
F ( )

!
f t ( )

!
F ( )
1
!T
rd
edition
! Establish conditions under which a continuous function can
be recovered uniquely from a set of its samples
! Bandlimited, f(t)
! Whose Fourier transform is zero for frequencies outside a
finite interval (band), [-
max
,
max
]
! f(t) can be recovered from sampled version if we can
isolate F() from periodic sequence
51
! No information is loss if continuous
bandlimited function represented by
samples acquired at rate higher
than twice frequency
! Or
! Max frequency that can be
captured by sampling a signal at a
rate
52
1
!T
> 2
max
1
!T
is
max
=
1
2!T
rd
edition
Recover F(!)
! Low pass filter
53

H ( ) =
!T -
max
" "
max
0 otherwise
#
$
%
&
'
(
F ( ) = H ( )
!
F ( )
f t ( ) = F
)*
*
+
( )e
j 2,t
d
rd
edition
Aliasing
! What happens if bandlimited
function sampled at rate less than
twice its highest frequency
! Undersampling
will contain periods that
overlap
! Aliasing always present in sampled
signals
# Infinite frequency components
introduced once we limit duration of
the signal
54

!
F ( )
rd
edition
Fourier transform of an under-
sampled, band-limited function
55
rd
edition
Two Dimensional Sampling and the 2-
D Sampling Theorem
! Two-dimensional sampling theorem
! A continuous, band-limited function f(t,z) can be recovered
with no error from a set of its samples if the sampling
intervals are
56
!T <
1
2
max
and
!Z <
1
2"
max
rd
edition
Aliasing in Images
! f(t,z) bandlimited only if it extends infinitely in both
coordinate directions
! Aliasing always present in digital images
! We can not infinitely sample
! Spatial aliasing (focus of class)
! Temporal aliasing related to time interval between
images (wagon wheel)
57
rd
edition
Aliasing in Images
! Let imaging system have 96 x 96 pixels

! Aliasing can crease results that look misleading
58
Squares size
16 pixels
Squares size
less than 1
pixel
Squares size
less than 0.5
pixel
Squares size
6 pixels
rd
edition
Anti-Aliasing
! Has to occur before image is sampled
! Blurred or defocused to reduce high frequency content
59
Image Interpolation and Resampling
! Effect of aliasing generally worse as size of image reduced
! Perfect reconstruction of bandlimited images from set of samples
requires 2D convolution in spatial domain with sinc function
! 2D interpolation and zooming
! Zooming viewed as oversampling
! Shrinking viewed as undersampling
60
rd
edition
2D DFT
61
C
e
n
t
e
r
i
n
g

t
h
e

F
o
u
r
i
e
r

T
r
a
n
s
f
o
r
m

rd
edition
Mathworks
Transfer Functions
h x, y
( )
!H(u, v)
Centered low pass function
Format used for DFT
64
Fourier Spectrum and Phase Angle
rd
edition
65
Fourier Spectrum Examples
rd
edition
66
2-D Convolution Theorem
rd
edition
67
Summary of Discrete Fourier Transform
rd
edition
68
! Filtering in frequency domain consists of modifying the
Fourier Transform of an image and computing the
inverse transform to obtain a processed result
Basics of Filtering in the Frequency
Domain
rd
edition
69
rd
edition
DC Term
70
rd
edition
71
High-frequency Emphasis Filtering
rd
edition
72
High frequency filtered
High frequency emphasis filtered
With histogram equalization
rd
edition
Need for Zero Padding
! When working in discrete domain F and H are periodic
! Convolution performed in discrete frequency domain is also
periodic
! Called circular convolution
! Only way to ensure that spatial filtering and circular
convolution give the same result is with zero-padding
73
f t ( )!h t ( ) " H ( )F ( )
f t ( )h t ( ) " H ( )!F ( )
rd
edition
Need for Zero-Padding
74
Zero padding: How much?
! f(x,y) and h(x,y) are of size AxB
and CxD
! Wrap around error avoided for
P!A+C-1 and Q!B+D-1
! For functions of the same size,
MxN, P!2M-1 and Q!2N-1
! For effective FFT, P and Q are
chosen such that they are powers
of 2
75
rd
edition
Summary of Steps for Filtering in the
Frequency Domain
77
! Convert input image to floating point
! Obtain padding parameters
! Obtain Fourier transform with padding
! Generate filter function, H, with size PxQ
! Center shift, if center shifted F
! G=HF (array multiplication)
! Obtain inverse fft of G
! Crop top left rectangle
! Convert filtered image to class of input image if desired
Steps for Filtering in the Frequency
Domain
rd
edition
78
rd
edition
Fourier Spectrum
79
rd
edition
L
o
w

P
a
s
s

F
i
l
t
e
r
s

rd
edition
81
I
m
a
g
e

S
h
a
r
p
e
n
i
n
g

u
s
i
n
g

F
r
e
q
u
e
n
c
y

D
o
m
a
i
n

F
i
l
t
e
r
s

rd
edition
! Highpass filter from a low pass filter
Spatial Representation of Highpass
Filters
rd
edition
83
Result of Highpass filtering
rd
edition
84
The Laplacian in the Frequency
Domain
rd
edition
85
Unsharp Masking, Highboost Filtering,
and High-Frequency Emphasis Filtering
rd
edition
86
! Illumination-reflection model can be
used to develop frequency domain
procedure for improved appearance
of image by simultaneous intensity
range compression and contrast
enhancement
Homomorphic Filtering
rd
edition
87
rd
edition
88
Enhancement using homomorphic
filtering
rd
edition
89
! Bandreject and Bandpass filters
! Notch Filters
Selective Filtering
rd
edition
90
Example filtering with notch filter
rd
edition
91
Example filtering with notch reject
filter
rd
edition
92
Example Filtering with Notch Filters
93
94
Spatial Domain Kernel to Frequency
Domain Filter
rd
edition
95
Gaussian Filter Functions
Sobel Operators
Based on the previous equations we can derive the Sobel
Operators

To filter an image it is filtered using both operators the
results of which are added together
-1 -2 -1
0 0 0
1 2 1
-1 0 1
-2 0 2
-1 0 1
96
Frequency Domain Spectrum of
Vertical Sobel Filter
Example: Filtering with Vertical Sobel
Filter
98
Example: Sobel Filtered Image after
Thresholding
99
Summary and Next Time
100
SPRING 2014
Image Restoration and Reconstruction
Lecture 5
February 27, 2014
2/27/14
Review of Lecture 4
rd
edition
Spatial domain filtering operations
Pre-
processing
h(x,y)
Post-
processing
g(x,y)=h(x,y)*f(x,y)
f(x,y)
Steps for Filtering in the Frequency
Domain
rd
edition
3
rd
edition
4
Todays Lecture
! What is image restoration
! Scope and applications
! Mode for (linear) image degradation
! Restoration from noise
! Different types of noise
! Example of restoration operations
! Restoration from linear degradation
! Inverse and pseudo-inverse filtering
! Wiener filters
! Blind de-convolution
! Geometric distortion and its corrections
5
Degraded images
! What causes the image to blur?
! Camera: translation, shake, out-of-focus.
! Environment: scattered and reflected light
! Device noise: CCD/CMOS sensor and circuitry
! Quantization noise
! Can we improve the image, or undo the effects?
6
! Image enhancement: improve an image subjectively
! Image restoration: remove distortion from image in order
to go back to the original " objective process
7
Image Restoration
! Started from the 1950s
! Application domains
! Scientific explorations
! Legal investigations
! Film making and archival
! Image and video (de-)coding
!
! Consumer photography
! Related problem: image reconstruction in radio
astronomy, radar imaging and tomography
8
! Image enhancement: improve an image subjectively.
! Image restoration: remove distortion from image, to go
back to the original objective process
Model of the degradation/restoration
process
9
rd
edition
g x, y ( ) = H f x, y ( ) !
"
#
$
+% x, y ( )
g x, y ( ) = h x, y ( )& f x, y ( ) +% x, y ( )
G u, v ( ) = H u, v ( )F u, v ( ) + N u, v ( )
! Image restoration
! Use a priori knowledge of the degradation
! Modeling the degradation and apply the inverse process
! Formulate and evaluate objective criteria of goodness
process
10
rd
edition
g x, y ( ) = H f x, y ( ) !
"
#
$
+% x, y ( )
& design restoration filters such that
f x, y ( ) is as close to f x, y ( ) as possible
Usual Assumptions for the distortion
model
! Noise
! Independent of spatial location
# Exception: periodic noise
! Degradation function H
! Linear
! Position-invariant
11
Divide-and-conquer step#1: image
degraded only by noise
rd
edition
Noise Models
! Principal sources of noise in digital images
! Image acquisition and/or transmission
! Spatial and frequency properties of noise
12
Important Probability Density Functions
13
rd
edition
Images and Histogram Showing
Effect of Additional Noise
14
rd
edition
Periodic Noise
! Periodic noise in images typically
arise from electrical or
electromechanical interference
during image acquisition
! Spatially dependent noise
! Reduced by frequency domain
filtering
15
rd
edition
! Periodic noise estimated by inspection of Fourier
spectrum of image
! Estimate parameters of PDF from small patches of
reasonably constant background intensity
! Calculate mean and variance of intensity levels
Estimating Noise parameters
z = z
i
p
s
z
i
( )
i=0
L!1
"
#
2
= z
i
! z
( )
2
p
s
z
i
( )
i=0
L!1
"
16
rd
edition
MATLAB Function
roipoly
rd
edition
Restoration in the presence of noise
only Spatial Filtering
! When the only degradation present in image is noise
! Noise, (x,y), is generally not known so can not just be
subtracted
! For periodic noise, can estimate N(u,v) from G(u,v)
! Otherwise, spatial filtering is needed
18
g x, y ( ) = f x, y ( ) +! x, y ( )
G u, v ( ) = F u, v ( ) + N u, v ( )
Spatial Filtering
! S
xy
rectangular subimage, window, mxn centered at
(x,y)
! Mean Filters
! Arithmetic mean filter
! Geometric mean filter
# Achieves smoothing but tens to loose less image detail
! Harmonic mean filter
# Works well for salt noise
! Contraharmonic mean filter
# Good for: Q -order of filter reducing or eliminating salt and
pepper noise. Q positive eliminates pepper noise. Q negative
eliminates salt noise.
19
rd
edition
Recovering from noise
! Overall process
! Observe and estimate noise type and parameters " apply
optimal (spatial) filtering (if known) " observe result, adjust
filter type/parameters .
! Example noise-reduction filters
! Mean/median filter family
! Adaptive filter family
! Other filter family
# Homomorphic filtering for multiplicative noise
21
Example: Image with additive
Gaussian Noise
Additive
Gaussian
noise
Filtering with
3x3
arithmetic
mean
Filtering with
3x3
geometric
mean
22
rd
edition
Example: Image with salt and pepper
noise
Pepper noise
with probability
of 0.1
Salt noise with
probability of
0.1
Filtering (a) with
3x3 contra-
harmonic filter of
order 1.5
Filtering (b) with
3x3 contra-
harmonic filter of
order -1.5
23
rd
edition
! Results of selecting the wrong size for Q when dealing
with pepper or salt noise
Example: Contra-harmonic filtering
24
rd
edition
Order Statistic Filters
! Spatial filters whose response is based on ordering
(ranking) the values of the pixels contained in the
neighborhood
! Median filter
! Max and min filters
! Midpoint Filter
! Alpha-trimmed mean filter
25
rd
edition
Example: Median Filter
27
rd
edition
Example: Max and Min Filter
3x2 min filter 3x2 max filter
28
rd
edition
E
x
a
m
p
l
e
s

Image corrupted
by additive
uniform noise
additionally
corrupted with salt
and pepper noise
Filtered (b) with
5x5 arithmetic
mean filter
Filtered (b) with
5x5 geometric
mean filter
Filtered (b) with
5x5 median filter
Filtered (b) with
alpha trimmed
mean filter with
d=5
rd
edition
Adaptive Filters
! Filters whose behavior changes based on statistical
changes of the image inside filter region defined by mxn
window of Sxy
! Increased filter power comes with increased filter
complexity
! Adaptive, local noise reduction filter
! Adaptive filter based off mean and variance
! Mean " average intensity in the region
! Variance " gives a measure of contrast in the region
30
Adaptive Filters
! Response of filter at any point (x,y) based on
! g(x,y) value of noisy image at (x,y)
!
2
variance of noise corrupting f(x,y) to form g(x,y)
! m
L
local mean in Sxy
!
L
2
local variance in Sxy
! Behavior of filters
! If
2
=0, g(x,y)=f(x,y)
!
L
2
high with respect to
2
, filter return value close to g(x,y)
# High variance typically associated with edges and should be preserved
!
2
~
L
2
, filter return arithmetic mean value of Sxy local area has same
properties as overall image " local noise is reduced by averaging
31
f x, y ( ) = g x, y ( ) !
"
#
2
"
L
2
g x, y ( ) ! m
L
$
%
&
'
Adaptive Median Filter
! Adaptive median filter changes size of Sxy during filter
operations
32
Example: Adaptive Noise Reduction
Filtering
Image corrupted by
additive Gaussian
noise of zero mean
and variance of 1000
7x7 arithmetic
mean filtering
7x7 geometric
mean filtering
7x7 adaptive
noise reduction
filtering
33
rd
edition
Example: Adaptive median filtering
(a) Image corrupted by salt-and-pepper noise with probabilities
P
a
=P
b
=0.25. (b) Result of filtering with a 7x7 median filter. (c) Result
of adaptive median filtering with S
max
=7
34
rd
edition
! Periodic noise appears as concentrated bursts of energy
in the Fourier Transform
! Band reject filters
! Band pass filters
! Notch filters
Periodic Noise Reduced by Frequency
Domain Filtering
Plot of idea,
Butterworth (order
1) and Gaussian
band reject filters
35
rd
edition
! Band reject noise removal where general location of
noise components in frequency domain is apparently
known
Example: Bandreject filter
(a) Image
corrupted by
sinusoidal noise.
(b) spectrum of
(a). (c)
Butterworth
bandreject filter.
(d) Result of
filtering
36
rd
edition
Bandpass filter
! Generally, BP filtering of images
are not done because it removes
too much detail
! Useful in isolating artifacts on
image caused by stretched
frequency bands
H
BP
u, v ( ) = 1! H
BR
u, v ( )
37
rd
edition
Notch Filters
! A notch filter rejects (or
passes) frequencies in
predefined neighborhood
about a center frequency
! Notch filters must appear as
symmetric pairs about the
origin
38
rd
edition
H
NP
u, v ( ) = 1! H
NR
u, v ( )
Example: Notch Filtering
Image with
horizontal
scan lines
spectrum
Notch pass
filter
Spatial noise
pattern
Result of
notch reject
filtering
39
rd
edition
Optimal Notch Filtering
! When several interference components are presence,
pervious methods not always suitable because they may
remove too much information in filtering process
! Also, interference are generally not single frequency
bursts but have broad skirts that carry information about
the interference pattern
! Optimum " minimize local variance of the restored
image
40
Optimal Notch Filtering
1. Procedure consists of first isolating principal
contributions of the interference pattern and then
subtracting a variable, weighted portion of the pattern
from the corrupted image
41
Example: Periodic Interference
42
rd
edition
Linear Position Invariant Degradation
! Degradation estimation by impulse characterization
a. Impulse of light (shown magnified)
b. Imaged (degraded) impulse
g x, y ( ) = H f x, y ( ) !
"
#
$
+% x, y ( )
43
rd
edition
Estimating the Degradation Function
! Observation

! Experimentation

! Mathematical Modeling
44
H
s
u, v ( ) =
G
s
u, v ( )
F
s
u, v ( )
H u, v ( ) =
G u, v ( )
A
H u, v ( ) = e
k u
2
+v
2
( )
5 6
Example: Atmospheric turbulence
model
Negligible
turbulence
Severe
turbulence,
k=0.0025
Mild
turbulence,
k=0.001
Low
turbulence,
k=0.00025
45
rd
edition
Image blurring
Image
blurred
using
fspecial with
lens=7 and
theta =-45
Noise image
Noise +
blurred
image
46
rd
edition
! Objective to restore image degraded by a degradation
function H
! Simplest approach is direct inverse filtering, where
computing an estimate of F
hat
(u,v) by dividing transform
of degraded image, G(u,v), by the degradation function

! For regions where H(u,v) is zero or very small " limit
frequencies within analysis, and concentrate on area
around H(0,0)
Inverse Filtering
47
F u, v ( ) =
G u, v ( )
H u, v ( )
F u, v ( ) = F u, v ( ) +
N u, v ( )
H u, v ( )
Inverse filtering of noisy and blurred
image
Inverse
filtering
Wiener filtering
using constant
ratio
Wiener filtering
using
autocorrelation
function
48
rd
edition
49
rd
edition
Example: Image degradation by
blurring
50
rd
edition
Example: Inverse Filtering
51
rd
edition
Wiener filter
! Incorporates degradation function and statistical
characteristics of noise into restoration process
! Images and noise are random variables
! Objective to find estimate of f
hat
of the uncorrupted
image f such that the mean square error between them
is minimized

52
e
2
= E f !

f
( )
2
{ }
Wiener filter defined
53
Wiener filter: Useful Measures
54
Example: Wiener filtering
a. Result of full inverse filtering
b. Radially limited inverse filtering
c. Wiener filtering
55
rd
edition
Example: Filtering image corrupted by
motion blur and additive noise
a. 8-bit image corrupted by
motion blur and additive
noise
b. Result of inverse filtering
c. Result of wiener filtering
d-f same sequence, but with
noise variance one order of
magnitude less
g-i same sequence, but with
noise variance reduce by five
orders of magnitude
56
rd
edition
Constrained Least Square Filtering
57
Example: Constrained Least Squares
Filtering
a. Corrupted image. b. results of constrained least squares filtering. c. Wiener filtering
58
rd
edition
Example: Constrained least square
restoration
59
rd
edition
Image Reconstruction from Projections
60
rd
edition
Reconstruction with multiple projections
61
rd
edition
Reconstruction of multiple objects and
absorption profiles with backprojection
62
rd
edition
Computed Tomography
63
rd
edition
Projections
64
rd
edition
Sinograms and Radon Transform
backprojection
65
rd
edition
Fourier Slice Theorem
! Relates 1D Fourier Transform of projection and 2D
Fourier Transform of region of projection

66
rd
edition
Reconstruction using Parallel Beam
Filtered Back Projections
67
Filtered Backprojections
Ramp filter and hamming windowed
ramp filter
Ramp filter and hamming windowed
ramp filter
68
rd
edition
Reconstruction using fan-beam filtered
backprojections
69
rd
edition
rd
edition
Example: Reconstruction with filtered
fan back projections
71
rd
edition
Reconstruction using filtered
backprojection
Reconstruction of head phantom
using filtered back projections
with smaller angular increments
72
rd
edition
rd
edition
rd
edition
rd
edition
rd
edition
77
SPRING 2014
Color Image Processing
Lecture 6
March 6, 2014
3/6/14
Todays Lecture

! Chapter 6
! Color fundamentals
! Color Models
! Pseudocolor Image Processing
! Basics of Full-Color Image Processing
! Color Transformations
! Brief Midterm Review
2
Color Image Processing
! Motivated by:
! Color is a powerful descriptor that simplifies object
identification and extraction from a scene
! Humans can discern thousands of color shades and intensities
vs ~ two dozen shades of gray
" Important for manual image analysis
! Full Color Processing
! Pseudocolor processing
3
! Colors that we perceive in an object are determined by
the nature of the light reflected from the object
! Visible light composed of relatively narrow band of
frequencies in the electromagnetic spectrum
! A body that reflects light that is balanced in all visible
wavelengths appears white
! Body that favors reflectance in a limited range exhibits
shades of color
Color Fundamentals
rd
edition
4
! If light is achromatic (void of color),
its only attribute is intensity
! Chromatic light spans the EM
spectrum from ~400-700m
! Three basic quantizes to describe
quality of chromatic light
! Radiance
! Luminance
! Brightness
Chromatic Light
rd
edition
5
Absorption of light by cones in
human eye functions of
wavelength
Chromaticity
Diagram
rd
edition
rd
edition
Color Monitor and Printing Color
Gamut
7
! Each color appears in primary spectral
components: Red, Green, Blue
! Grayscale: points of equal values of
RGB
! Extend on light from black to white
! All other colors are on or inside cube
RGB Color Model
rd
edition
8
RGB Color Model
! Generating
RGB image of
cross sectional
color plane ie.
(127, G, B)
9
rd
edition
(2
8
)
3
=16,777,216 colors
rd
edition
216 Safe RGB Colors
! Assume 256 colors is minimum number of
colors that can be reproduced faithfully by
any system with desired result
! 40/256 known to be processed differently
on various systems
! 216 safe colors standard especially for
internet applications
! Formed by RGB values, when each where
each value can be 0, 51, 102, 153, 204,
or 255. (6)
3
=216
! Each safe color formed by 3 of the 2 digit
hex numbers
10
The CMY and CMYK Color Models
! Used in printers
! Primary pigment colors,
secondary light colors
! Subtractive vs. Additive
C
M
Y
!
"
#
#
#
$
%
&
&
&
=
1
1
1
!
"
#
#
#
$
%
&
&
&
'
R
G
B
!
"
#
#
#
$
%
&
&
&
11
The HSI Color Model
! Hue
! Saturation
! Intensity
rd
edition
12
The HSI Color Model
! Saturated colors on the
outer points
! Maximum saturation at
intermediate intensity
levels
! Distance between
colors:
13
rd
edition
D C
1
, C
2
( ) = w
1
!H + w
2
!S + w
3
!I
Converting colors from RGB to HSI
14
H =
! if B" G
360 #! if B> G
$
%
&
'
(
)
with
! = cos
#1
1
2
R# G ( ) + R# B ( ) *
+
,
-
R# G ( )
2
+ R# B ( ) G # B ( )
*
+
,
-
1
2
$
%
.
&
.
'
(
.
)
.
S =1#
3
R+ G+ B
min R, G, B ( ) *
+
,
-
I =
1
3
R+ G+ B ( )
Converting colors from HSI to RGB
15
RG sector 0
o
! H <120
o
( )
B = I 1" S ( )
R = I 1+
Scos H
cos 60
o
" H
( )
#
$
%
%
&
'
(
(
G = 3I " R+ B ( )
GB sector 120
o
! H < 240
o
( )
H = H "120
o
R = I 1" S ( )
G = I 1+
Scos H
cos 60
o
" H
( )
#
$
%
%
&
'
(
(
B = 3I " R+ G ( )
BR sector 240
o
! H < 360
o
( )
H = H " 240
o
G = I 1" S ( )
B = I 1+
Scos H
cos 60
o
" H
( )
#
$
%
%
&
'
(
(
R = 3I " G+ B ( )
rd
edition
HSI Component of Image
16
Hue Saturation
Intensity
rd
edition
Manipulating HSI component images
! HSI values of primary and
secondary colors
! HSI allows for independent
manipulation of colors
! Hue of Green & Blue set to Zero
! Saturation of Cyan reduced by
!
! Intensity of white reduced by !
3/6/14
18
Christine P. Hendon
rd
edition
Pseudocolor Image Processing
! Assigning colors to gray scale values based on specified
criterion
! Principal use of pseudocolor for human visualization and
interpretation of gray scale events in an image or
sequence of images
19
! One of simplest examples of
pseudocolor image processing
! Intensity to color assignment
! When more levels are used
mapping function takes on a
staircase form
Intensity Slicing
rd
edition
f x, y ( ) = c
k
if f x, y ( ) !V
k
20
Example: Intensity Slicing
! Result of intensity slicing into eight color regions
! Regions that appear of constant intensity in monochrome
image are quite variable and picking out variations in
intensity is difficult
rd
edition
21
rd
edition
! Intensity slicing to assign all
pixels with 255 intensities
(saturated) to color
22
rd
edition
23
! Perform 3 independent
transformations on the intensity of
the input pixel
! Transformations in intensity image
not a function of position
! Can enable smoothing and non-
linear functions
Intensity to Color Transformations
rd
edition
24
Pseudocolor enhancement
rd
edition
25
Pseudo Color with Several
Monochrome Images
! Combine several monochrome images into one color
composite
! Multispectral image processing
rd
edition
26
Example: Color Coding of
Multispectral Images
rd
edition
27
Example: Color Coding of
Multispectral Images
rd
edition
28
Basics of Full-Color Image Processing
! Full Color Image Processing Approaches
1. Process each component image individually and then form
composite processed color image from individually
processed components
2. Work on color pixels directly
29
rd
edition
Basics of Full-Color Image Processing
! For MxN image, MN vectors
! For per-color component and vector-based processing to
be equivalent
1. Process has to be applicable to vectors and scalars
2. Operation on each component of a vector must be
independent of the other component
c =
c
R
c
G
c
B
!
"
#
#
#
$
%
&
&
&
=
R
G
B
!
"
#
#
#
$
%
&
&
&
c x, y ( ) =
c
R
x, y ( )
c
G
x, y ( )
c
B
x, y ( )
!
"
#
#
#
#
$
%
&
&
&
&
=
R x, y ( )
G x, y ( )
B x, y ( )
!
"
#
#
#
#
$
%
&
&
&
&
Color Transformations: Formulation
g x, y
( )
= T f x, y
( ) !
"
#
$
Transformed color
output image
Operator on f over
a spatial
neighborhood
Color input image
31
rd
edition
Full Color Image and Color Space
Components
32
rd
edition
33
Color Complements
! Complements: hues
directly opposite one
another on color circle
! Analogous to the gray
scale negative
! Useful for enhancing
details embedded in dark
regions of color image
34
rd
edition
Color Complements
rd
edition
35
rd
edition
Color-Slicing Transformation
! Highlight specific region
for separating objects
! Display color of interest so
they stand out from
background
! Use region defined by
color as mask for further
processing
! Simple way to slice
! Map colors outside a
range to non-prominent
neutral color
36
s
i
=
0.5 if r
j
! a
j
>
W
2
"
#
$
%
&
'
r
i
otherwise
(
)
*
+
*
,
-
*
.
*
for sphere,
s
i
=
0.5 if r
i
! a
j
( )
2
> R
o
2
j=1
n
/
r
i
otherwise
(
)
*
+
*
,
-
*
.
*
Tone and Color Corrections
! Tonal range of image, or called
key type, refers to general
distribution of color intensities
! Like with monochromatic images,
desirable to distribute intensities
of color image equally between
highlights and shadows
! Adjust experimentally with image
brightness and contrast
38
rd
edition
Color Balancing Correction for
CMYK Image
! Humans are good
at assessing proper
tone of skin
39
rd
edition
! Histogram equalization
applied in automatic way
! Unwise to apply histogram
equalization to components
of color image
! # apply to intensity portion
of HSI color space
40
Image Smoothing
! Color image smoothing
41
c x, y ( ) =
1
K
c s, t ( )
s,t ( )!S
xy
"
c x, y ( ) =
1
K
R s, t ( )
"
1
K
G s, t ( )
"
1
K
B s, t ( )
"
#
$
%
%
%
%
%
%
%
&
'
(
(
(
(
(
(
(
rd
edition
Examples: Image Smoothing
42
rd
edition
Color Image Sharpening
43
!
2
c x, y ( ) "
#
$
%
=
!
2
R x, y ( )
!
2
G x, y ( )
!
2
B x, y ( )
"
#
&
&
&
$
%
'
'
'
Image Segmentation Based on Color:
HSI Color Space
44
rd
edition
Example: Image Segmentation in HSI
Space
45
Image Segmentation Based on Color:
RGB Vector Space
rd
edition
46
! Segmentation better when represented in RGB space
! Segment color range in RGB
! Assess desired average color a
! Measure Euclidian distance to assess similarity to a
D z, a ( ) = z ! a
D z, a ( ) = z ! a ( )
T
z ! a ( )
"
#
$
%
1
2
Generalized
D z, a ( ) = z ! a ( )
T
C
!1
z ! a ( )
"
#
$
%
1
2
C : Covariance matrix
rd
edition
Example: Image Segmentation in RGB
Space
47
Color Edge Detection
! Gradient operation defined in Chapter 3 note for vector
operation
! Can not apply to individual images
48
rd
edition
Color Edge Detection
50
rd
edition
Example: Gradients in RGB Image
51
rd
edition
Example: Component Gradient Images
52
Noise in Color Images
! Usually noise content of
color image has same
characteristics in each
color channel, but possible
for one channel to be
affected differently
53
rd
edition
Example: Noise in HSI Images
54
rd
edition
Example Noise in Image
55
Color Image Compression
56
MATLAB Predefined Color Maps
57
MATLAB Toolbox functions for
Converting between RGB, indexed,
and gray-scale images
Lecture 1 Highlights
3/6/14
59
Christine P. Hendon
Human perception phenomena
60
Simultaneous contrast regions perceived brightness does not depend
simply on its intensity
rd
edition
61
! Monochromatic light
! Described by intensity or gray level
! Chromatic (color) light
! Described by radiance, luminance, and
brightness
" Radiance total amount of energy from the
light source [W]
" Luminance amount of energy observer
perceives from the light source [lm]
" Brightness subjective descriptor of light
perception
Digital Image Acquisition Process
62
! Model of image formation, image f(x,y)
! Characterized by
! Amount of source illumination incident on the scene being
viewed, i(x,y)
! Amount of illumination reflected by objects in scene, r(x,y)
0 < f x, y ( ) < !
f x, y ( ) = i x, y ( )r x, y ( )
0 < i x, y ( ) < !
0 < r x, y ( ) <1
0 < t x, y ( ) <1
rd
edition
Image Sampling and Quantization
63
! Sampling: digitizing the coordinates
! Quantization: digitizing the amplitude
Continuous image projected onto
sensor array results of sampling and
quantization
rd
edition
Dynamic Range
64
! Dynamic range of imaging system
! Upper limit determined by saturation
! Lower limit determined by noise
! Contrast
! Difference in intensity in between the
highest and lowest intensity levels in
an image
rd
edition
G
r
a
y
-
l
e
v
e
l

R
e
s
o
l
u
t
i
o
n

256 level image
2 level image
rd
edition
Artifact: False contouring insufficient number of
intensity levels in smooth areas of digital image
Image Interpolation
3/6/14
! Image interpolation or Image resampling methods
! Applied to image resizing (shrinking and zooming)
! Interpolation process of using known data to estimate values at unknown
locations
! Nearest neighbor interpolation assigns new location intensity of its
nearest neighbor in the original image
! Bilinear uses four nearest neighbors to estimate intensity at given location
! Bicubic uses sixteen nearest neighbors to estimate intensity at given
location
v x, y ( ) = ax + by + cxy + d
v x, y ( ) = a
ij
x
i
y
j
j=0
3
!
i=0
3
!
! Mask mode radiography
Mask Live
Difference Enhancement
rd
edition
g x, y ( ) = f x, y ( ) ! h x, y ( )
mask
X-ray image
Shade correction
68

! Application of image multiplication or division
! If h(x,y) is known, can recover f(x)
! If h(x,y) is not known but have access to imaging system, can
approximate h(x,y) by imaging sample with constant intensity
g x, y ( ) = f x, y ( )h x, y ( )
Shading function
Perfect image
rd
edition
Set and Logical Operations
69
! Subset
! Union
! Intersection
! Disjoint
! Mutually exclusive
! Set Universe
! Complement
! Difference
Set membership based on coordinates
rd
edition
Set Operations Involving Gray Scale
Images
70
! The union and intersection
operations for gray-scale
values usually are defined
as the max and min of
corresponding pixel pairs,
complement as pairwise
difference between a
constant and intensity of
every pixel in an image
! Array Operations
A! B = max a, b ( ) | a !A, b !B
{ }
rd
edition
original Set complement
Union of original
and constant
image
Logical Operations
71
! Binary images
! Foreground 1 valued
! Background 0 valued set of
pixels
! Objects: regions composed of
foreground
Union Intersect Complement

OR AND NOT
rd
edition
Chapter 5
3/6/14
72
Christine P. Hendon
! Image enhancement: improve an image subjectively.
! Image restoration: remove distortion from image, to go
back to the original objective process
process
73
rd
edition
g x, y ( ) = H f x, y ( ) !
"
#
$
+% x, y ( )
g x, y ( ) = h x, y ( )& f x, y ( ) +% x, y ( )
G u, v ( ) = H u, v ( )F u, v ( ) + N u, v ( )
! Periodic noise estimated by inspection of Fourier
spectrum of image
! Estimate parameters of PDF from small patches of
reasonably constant background intensity
! Calculate mean and variance of intensity levels
Estimating Noise parameters
z = z
i
p
s
z
i
( )
i=0
L!1
"
#
2
= z
i
! z
( )
2
p
s
z
i
( )
i=0
L!1
"
74
rd
edition
! Image restoration
! Use a priori knowledge of the degradation
! Modeling the degradation and apply the inverse process
! Formulate and evaluate objective criteria of goodness
process
75
rd
edition
g x, y ( ) = H f x, y ( ) !
"
#
$
+% x, y ( )
& design restoration filters such that
f x, y ( ) is as close to f x, y ( ) as possible
Restoration in the presence of noise
only Spatial Filtering
! When the only degradation present in image is noise
! Noise, (x,y), is generally not known so can not just be
subtracted
! For periodic noise, can estimate N(u,v) from G(u,v)
! Otherwise, spatial filtering is needed
76
g x, y ( ) = f x, y ( ) +! x, y ( )
G u, v ( ) = F u, v ( ) + N u, v ( )
Adaptive Filters
! Filters whose behavior changes based on statistical
changes of the image inside filter region defined by mxn
window of Sxy
! Increased filter power comes with increased filter
complexity
! Adaptive, local noise reduction filter
! Adaptive filter based off mean and variance
! Mean # average intensity in the region
! Variance # gives a measure of contrast in the region
77
! Periodic noise appears as concentrated bursts of energy
in the Fourier Transform
! Band reject filters
! Band pass filters
! Notch filters
Periodic Noise Reduced by Frequency
Domain Filtering
Plot of idea,
Butterworth (order
1) and Gaussian
band reject filters
78
rd
edition
Estimating the Degradation Function
! Observation

! Experimentation

! Mathematical Modeling
79
H
s
u, v ( ) =
G
s
u, v ( )
F
s
u, v ( )
H u, v ( ) =
G u, v ( )
A
H u, v ( ) = e
k u
2
+v
2
( )
5 6
! Objective to restore image degraded by a degradation
function H
! Simplest approach is direct inverse filtering, where
computing an estimate of F
hat
(u,v) by dividing transform
of degraded image, G(u,v), by the degradation function

! For regions where H(u,v) is zero or very small # limit
frequencies within analysis, and concentrate on area
around H(0,0)
Inverse Filtering
80
F u, v ( ) =
G u, v ( )
H u, v ( )
F u, v ( ) = F u, v ( ) +
N u, v ( )
H u, v ( )
Image Reconstruction from Projections
81
rd
edition
82
SPRING 2014
Wavelets and Multiresolution
Processing
Lecture 7
March 27, 2014
3/27/14
Schedule for the Remainder of the
Semeter
2
Todays Lecture

! Chapter 7: Wavelet and Multiresolution Processing
! Background
! Multiresolution Expansions
! Wavelet Transforms in One Dimension
! The Fast Wavelet Transform
! Wavelet Transforms in Two Dimensions
! Wavelet Packets
3
Overview
! Fourier Transform
! Basis functions are sinusoids
! Wavelet Transform
! Based on small waves, wavelets, of varying frequency and
limited duration
! Provides musical score for an image
4
Fourier Transform vs Discrete Wavelet
Transform
! The familiar Fourier expansion functions are sinusoids of
varying frequency and infinite duration
! Discrete wavelet transform expansion functions are small
waves of finite duration and varying frequency
5
rd
edition
STFT
! Time/Frequency localization depends on window size.
! Once you choose a particular window size, it will be the same for all
frequencies.
! Many signals require a more flexible approach - vary the window
size to determine more accurately either time or frequency.
The Wavelet Transform
! Overcomes the preset resolution problem of the STFT by
using a variable length window:

! Use narrower windows at high frequencies for better time
resolution.
! Use wider windows at low frequencies for better frequency
resolution.
Background
3/27/14
8
Christine P. Hendon
Motivation for Multiresolution
Processing
! Image: connected regions
of similar intensities /
textures form objects
! If object
! Small/low contrast "
examine at high resolution
! Large/high contrast "
coarse view is all that is
needed
9
rd
edition
Image Pyramid
! Representing images at more than one resolution
! Collection of decreasing resolution images arranged in
shape of pyramid
! As move up pyramid, both size and resolution of image
decreases
10
rd
edition
Image Pyramid
! Base level J size 2
J
x2
J
or NxN
! Apex level 0 size 1x1
! General level j size 2
j
x2
j
! Most pyramids go to level P
+1
! Limit ourselves to P
approximations
11
Example: Image Pyramid
12
rd
edition
Approximation pyramid
Prediction residual pyramid
Subband coding
! Subband coding
! image is decomposed into set of band limited components
called subbands
! Decomposition performed so that subbands can be
reassembled to reconstruct original image without error
! Decomposition and reconstruction performed with digital
filters
13
Digital Image Processing (DSP) Review
! Digital filters contructed of
three basic components
! Unit delays
! Multipliers
! Adders
! Along top of filter, unit
delays connected in series to
create K-1 delayed versions
of the input sequence f(n)
14
rd
edition
f n ( ) = h k ( )
k=!"
"
#
f n ! k ( )
f n ( ) = f n ( )*h n ( )
Impulse Response of Filter
! K multiplication constants are the filter coefficients
! Order K filter
! Substituting (n) for input f(n) " impluse response of
the filter is the K-element sequence of filter coefficients
that define the filter
! Because there are K coefficients, the impulse response of
length K and the filter are called finite impuse response
(FIR) filter
15
rd
edition
f n ( ) = h k ( )
k=!"
"
#
$ n ! k ( )
f n ( ) = h n ( )
Example: Two-band subband coding
and decoding system
! System composed of two filter
banks, each containing two FIR
filters
! Analysis filter bank: ho(n) and
h
1
(n) " will break sequence f(n)
into have length sequences f
lp
(n)
and f
hp
(n)
! Synthesis bank filters g
0
(n) and
g
1
(n) combine f
lp
(n) and f
hp
(n) to
produce f
hat
(n)
16
rd
edition
Goal of subband coding is to select
h0(n), h1(n), g0(n), and g1(n) so that
f n ( ) = f n ( )
Example: Subband coding
! Two-dimensional, four-band filter bank for subband image coding
! Separable filters first applied in one dimension (e.g. vertically) and
then in the other (e.g. horizontally)
! Results are called approximation, vertical detail, horizontal detail,
and diagonal detail subbands of the input image
! These subbands can be split into four smaller subbands, which can be
split again, and so on
17
rd
edition
Example: Daubechies 8-tap
orthonormal filter
18
rd
edition
Example: Daubechies 8-tap
orthonormal filter
! A four-band split of 512x512 image based
on previously described filters
! Each quadrant is a subband of size
256x256
! Approximation subband, horizontal
detail subband, vertical subband, and
diagonal subband
! Original Image can be reconstructed
without error even with aliasing in the
horizontal and vertical subbands
19
rd
edition
The Haar Transform
T=HFH
T

! For the Haar transform, H contains the Haar basis
functions h
k
(z). For k=0,1,,N-1
! Haar scaling and wavelet functions:
20
Scaling function
Wavelet function
!(t) "(t)
1D Haar Wavelets
21
Discrete Wavelet Transform using
Haar H
2
22
rd
edition
Discrete Wavelet Transform using
Haar H
2
! Decomposition of 512x512 image that
combines key features from pyramid
coding, subband coding, and the Haar
transform " discrete wavelet
transform
! Sub-images can be used to construct
both coarse and fine resolution
approximation of the original image
! Simple real coefficient FIR filter bank
used to produce images
! Each subimage represents a specific
band of spatial frequencies in the
original image
23
rd
edition
Multiresolution Expansions
3/27/14
24
Christine P. Hendon
Multiresolution Analysis (MRA)
! In MRA, a scaling function is used to create a series of
approximations of a function or image, each differing
by a factor of 2 in resolution from its nearest
neighboring approximations
! Additional functions, called wavelets, are then used to
encode the difference in information between adjacent
approximations
25
Series Expansions
! A signal or function f(x) can often be better analyzed as
a linear combination of expansion functions
! The expressible functions form a function space that is
referred to as the closed span of the expansion set
26
f x ( ) = !
k
k
"
#
k
x ( )
V = Span
k
! x ( )
{ }
f x ( ) "V
Scaling Functions
! Set of expansion functions composed of integer
translation and binary scaling of real square-integrable
functions
! Increasing the size of V
j
allows functions with smaller
variation or finer details to be included in the subspace
27
!
j ,k
= 2
j 2
! 2
j
x " k
( )
V
j
0
= Span
k
!
j
0
,k
x ( )
{ }
f x ( ) "V
j
0
f x ( ) = #
k
!
j
0
,k
x ( )
k
$
Example Haar scaling functions
! If f(x) is an element of
V
0
, it is also an
element of V
1.
All V0
expansion are
contained in V1

28
rd
edition
! x ( ) =
1 0 " x<1
0 otherwise
#
$
%
&
'
(
f x ( ) = 0.5!
1,0
x ( ) + 0.5!
1,1
x ( ) " 0.25!
1,4
x ( )
Four fundamental requirements of
multiresolution analysis
! MRA Requirement 1: the scaling function is orthogonal to
its integer translates
! MRA Requirement 2: The subspaces spanned by the
scaling function at low scales are nested within those
spanned at higher scales.
! MRA Requirement 3: The only function common to all Vj is
f(x)=0
! MRA Requirement 4: Any function can be represented
with arbitrary precision
29
rd
edition
Wavelet Functions
! Given a scaling function that meets the MRA
requirements, we can define a wavelet function (x)
that, together with integer translates and binary scaling,
spans the difference between any two adjacent scaling
subspaces, Vj and Vj+1
30
rd
edition
Relationship between
scaling and wavelet function
space
Haar wavelet functions in W
0
and W
1
! Using coefficients
corresponding to second
row of H
2
31
rd
edition
! x ( ) =
1 0 " x<0.5
#1 0.5 " x<1
elsewhere
$
%
&
'
&
(
)
&
*
&
Wavelet Transforms in One Dimension
! Closely related wavelet transformations
! Wavelet series expansion
! Discrete wavelet transform
! Continuous wavelet transform
! The Wavelet Series Expansion
32
f x ( ) = c
j 0
k ( )!
j 0,k
k
"
(x) + d
j
k ( )#
j ,k
x ( )
k
"
j= j
0
$
"
Example: Wavelet series expansion of
y=x
2
using Haar wavelets
33
rd
edition
The Discrete Wavelet Transform
! If the function being expanded is discrete, resulting
coefficients are called the discrete wavelet transform
34
The Continuous Wavelet Transform
! Transform a continuous function into a highly redundant
function of two continuous variables " translation and
scale
! Valuable for time frequency analysis
35
CWT: Main Steps
1. Take a wavelet and compare it to a section at the start of the
original signal.
2. Calculate a number, C, that represents how closely correlated
the wavelet is with this section of the signal. The higher C is, the
more the similarity.
CWT: Main Steps
3. Shift the wavelet to the right and repeat steps 1 and 2 until
you've covered the whole signal.

CWT: Main Steps
4. Scale the wavelet and repeat steps 1 through 3.

5. Repeat steps 1 through 4 for all scales.

Coefficients of CTW Transform
! Wavelet analysis produces a time-scale view of the input
signal or image.
Example: Continuous Wavelet
Transform
40
rd
edition
The Fast Wavelet Transform (FWT)
! Computationally efficient implementation of discrete
wavelet transform
41
rd
edition
Example: Two-stage FWT analysis
bank
42
rd
edition
Example: two-scale fast wavelet
transform of sequence {1,4,-3,0} using
Haar scaling and wavelet vectors
rd
edition
Synthesis Filters
44
rd
edition
Example: two-scale inverse fast
wavelet transform of sequence
{1,4,-1.5!2, -1.5!2}
rd
edition
Time-frequency tiling
46
rd
edition
Sampled data
FFT
FWT
Wavelet Transforms in Two Dimensions
! One dimensional transforms easily extended to two-
dimensional functions like images
! Two dimensional scaling function
!(x,y)=!(x) !(y)
! And three two-dimensional wavelets
!
H
(x,y)=(x) !(y)
!
V
(x,y)=! (x) (y)
!
D
(x,y)=(x) (y)
! Define scaled and translated basis functions " can
define discrete wavelet transform of image of size MxN
and inverse discrete wavelet

47
2D Fast
Wavelet
Transform
48
rd
edition
Example: Computing a 2-D three-scale FWT
a. Original 128 x 128
computer generated
image with sine-like
pulses
b. One scale FWT
c. Two scale FWT
d. Three scale FWT
Symlet
! Symlet: well known family of
wavelets
1. Decomposition Filters
2. Reconstruction filters
3. 1D wavelet and 1D scaling
function
4. 2D wavelet,
v
(x,y)
50
rd
edition
Basic Approach
! Step 1: Compute a 2-D wavelet transform of an image
! Step 2: Alter the transform
! Step 3: Compute the inverse transform
! Because the DWTs scaling and wavelet vectors are used
as lowpass and high pass filters, most Fourier based
filtering techniques have an equivalent wavelet domain
counterpart
51
Christine P. Hendon
Example: Modifying a DWT for Edge
Detection
1. Lowest scale
approximation component
of discrete wavelet
transform eliminated by
setting value to zero " net
effect of computing inverse
wavelet is edge
enhancement
2. By zeroing the horizontal
details we can isolate the
vertical edges 3/27/14
52
rd
edition
Modifying a DWT for Noise
Removal
! Image uniformly corrupted with additive noise
! Wavelet-based procedure for denoising
! Step 1. Choose wavelet (e.g. Haar, symlet,) and number of levels
(scales),P, for the decomposition. Then compute FWT of the noisy
image
! Step 2. Threshold detail coefficients from scales J-1 to J-P by harsh
thresholding or soft thresholding
! Step 3. Compute inverse wavelet transform (wavelet reconstruction)
using original approximation coefficients at level J-P and modified
detail coefficients for levels J-1 to J-P
53
rd
edition
Modifying a
DWT for Noise
Removal
! Various reconstructions
after thresholding the
detail coefficients and
the information removed
from the reconstructions
rd
edition
Wavelets in Edge Detection
a. Test image
b. Wavelet Transform
c. Transform modified by
zeroing all approximation
coefficients
d. Edge image resulting from
computing absolute value of
inverse of transform
55
rd
edition
Wavelet Based
Image Smoothing
a. Test image
b. Wavelet transform
c. Inverse transform after
zeroing the first level of
detail coeffs
d. - f. similar trends after
zeroing the second, third,
and fourth detail levels
rd
edition
Progressive Reconstruction
a. 4 scale wavelet transform
b. The four level
approximation image from
upper left corner
c. Refined approximation
incorporating the 4
th
level
details
d. -f, further resolution
improvement incorporating
higher level details
57
rd
edition
Wavelet Packets
! If want more control over partition of time-frequency plane, FWT
must be generalized to yield a more flexible decomposition called
wavelet packets " increased cost of complexity
! Example: 2-scale filter bank " represent decomposition as a binary
tree.
! Detail structure of the tree and links the FWT scaling and wavelet
coefficients to its nodes
! Root node assigned highest scale approximation coefficient
! Leaves inherent the transforms approximation and data
coefficient
! Intermediate nodes contain filter bank approximations
58
rd
edition
Example: Three scaled FWT Filter
Bank
59
rd
edition
Analysis Trees
! Analysis trees represent wavelet packets in which the
details are filtered iteratively
! First subscript of double script node identifies the FWT
parent node.
! The second, variable length string of As and Ds, encodes
path from parent to node. A-approximation filters, D-
detailed filters
60
rd
edition
Example: full wavelet packet analysis
tree
61
rd
edition
Decomposition of a 2D FWT
62
rd
edition
Three-scale, full wavelet packet
decomposition tree
63
rd
edition
Example: Three-scale full wavelet
packet decomposition of fingerprint
64
rd
edition
Optimal wavelet packet
deocomposition
! Reasonable criterion for selecting decomposition for the compression
of an image is the additive cost function

! Provides one possible measure of energy of the two dimensional
function f
! A cost function that maximized the number of near zero values is
good for compression point of view
! Optimization algorithm must use the cost function to minimize the cost
of the leaf nodes in the decomposition tree
! Minimizing cost of nodes favors for greater compression
65
E f ( ) = f m, n ( )
m,n
!
Example: optimal wavelet packet
decomposition of fingerprint
66
rd
edition
Cohen-Daubechies-Feauveau wavelet
family
! Decomposition filter
coefficients
! Reconstruction filter
coefficients
! Dual wavelet and scaling
functions
67
rd
edition
Wavelet Toolbox FWT Filters and
Filter Family Names
68
rd
edition
Wavelet Toolbox Image Extension or
Padding Modes
69
rd
edition
! Summary
! Like FT, wavelet transform can be used for edge detection,
image smoothing, noise reduction, etc
! Because wavelets provide significant insight into both spatial
and frequency characteristics, wavelets can also be used in
applications where Fourier methods are not well suited
! Next Time
! Chapter 8: Image Compression
70
E4830 DIGITAL IMAGE
PROCESSING
SPRING 2014
Image Compression
Lecture 8
April 3, 2014
Goal of Image Compression
2
! Image compression: art and science of reducing
amount of data required to represent an image
! The goal of image compression is to reduce the
amount of data required to represent a digital
image.
rd
edition
Approaches
3
! Lossless
! Information preserving
! Low compression ratios

! Lossy
! Not information preserving
! High compression ratios

! Trade-off: image quality vs compression ratio
Data ! Information
4
! Data and information are not synonymous terms!

! Data is the means by which information is
conveyed.
! Data compression aims to reduce the amount of
data required to represent a given quantity of
information while preserving as much information as
possible.
Data vs Information (contd)
5
! The same amount of information can be
represented by various amount of data, e.g.:
Your wife, Helen, will meet you at Logan Airport in Boston at 5
minutes past 6:00 pm tomorrow night

Your wife will meet you at Logan Airport at 5 minutes past
6:00 pm tomorrow night

Helen will meet you at Logan at 6:00 pm tomorrow night
Ex1:
Ex2:
Ex3:
Relative Data Redundancy
4/3/14
6
! Let b and b denote the number of bits in two representations of the same
information, the relative data redundancy R is
R = 1-1/C

C is called the compression ratio, defined as

C = b/b

e.g., C = 10, the corresponding relative data redundancy of the larger
representation is 0.9, indicating that 90% of its data is redundant
Christine P. Hendon
Why do we need compression?
4/3/14
7
! Data storage
! Data transmission
! Example: 2 hr movie, 30 fps, 720x480x24 bit pixel arrays /
frame
! (30 fps x (720x480) pixels/frame x 3 bytes/pixel = 31,104,000
bytes/sec
! 2hr movie:
" 31,104,000 bytes/sec x (60
2
)sec/h x 2 hrs ~2.24x10
11
bytes
" 224 GB # 27 8.5GB dual layer DVDs to store it!!
Christine P. Hendon
How can we implement compression?
4/3/14
8
! Coding redundancy
Most 2-D intensity arrays contain more bits than are needed to
represent the intensities

! Spatial and temporal redundancy
Pixels of most 2-D intensity arrays are correlated spatially
and video sequences are temporally correlated

! Irrelevant information
Most 2-D intensity arrays contain information that is ignored by
the human visual system
Christine P. Hendon
Examples of Redundancy
4/3/14
9
rd
edition
Coding redundancy
Spatial redundancy
Irrelevant Information
Christine P. Hendon
Coding Redundancy
10
! Code: a list of symbols (letters, numbers, bits etc.)
! Code word: a sequence of symbols used to
represent a piece of information or an event (e.g.,
gray levels).
! Code word length: number of symbols in each code
word
Coding Redundancy
4/3/14
11
1
0
The average number of bits required to represent each pixel is
( ) ( ) 0.25(2) 0.47(1) 0.24(3) 0.03(3) 1.81
L
avg k r k
k
L l r p r bits
!
=
= = + + + =
"
8
4.42
1.81
1 1/ 4.42 0.774
C
R
= !
= " =
rd
edition
Christine P. Hendon
Spatial and Temporal Redundancy
4/3/14
12
! All 256 intensities are equally probable
! The pixels along each line are identical
! The intensity of each line was selected randomly
rd
edition
Christine P. Hendon
Spatial and Temporal Redundancy
4/3/14
13
! Run-length pair specifies the start of a new intensity
and the number of consecutive pixels that have that
intensity
! Each 256-pixel line of the original representation is
replaced by a single 8-bit intensity value and
length 256 in the run-length representation

f (x)og(x) = f (x)g(x + a) da
!"
"
#
The compression ratio is
256 256 8
128:1
(256 256) 8
! !
=
+ !
Christine P. Hendon
14
! The human eye does not respond with equal sensitivity
to all visual information.
! It is more sensitive to the lower frequencies than to the
higher frequencies in the visual spectrum.
! Idea: discard data that is perceptually insignificant!
4/3/14
15
256 256 8 / 8
65536:1
! !
=
rd
edition
Christine P. Hendon
Example: Irrelevant Information
16
256 gray levels 16 gray levels/random noise 16 gray levels
C=8/4 = 2:1

i.e., add to each pixel a small pseudo-random
number prior to quantization
Example: quantization
Measuring Information
17

! What is the information content of a message/
image?
! What is the minimum amount of data that is
sufficient to describe completely an image without
loss of information?
Modeling Information
18
! We assume that information generation is a
probabilistic process.
! Idea: associate information with probability!
Note: I(E)=0 when P(E)=1
A random event E with probability P(E) contains:
How much information does a pixel
contain?
19

! Suppose that gray level values are generated by a
random variable, then r
k
contains:
units of information!
How much information does an image
contain?
20
! Average information content of an image:
units/pixel
(e.g., bits/pixel)
1
0
( ) Pr( )
L
k k
k
E I r r
!
=
=
"
using
(assumes statistically independent random events)
Entropy:
Example: Entropy Estimation
21
image
Entropy Estimation
22
! First order estimate of H:
Estimating Entropy
23
! Second order estimate of H:
! Use relative frequencies of pixel blocks :
image
Estimating Entropy
24
! The first-order estimate provides only a lower-bound
on the compression that can be achieved.
! Differences between higher-order estimates of
entropy and the first-order estimate indicate the
presence of interpixel redundancy!
Need to apply some transformations to
deal with interpixel redundancy!
Estimating Entropy
25
! Example: consider differences:
Estimating Entropy
26
! Entropy of difference image:
An even better transformation could be found since:
Better than before (i.e., H=1.81 for original image)
Fidelity Criteria
27
! How close is to ?
! Criteria
! Subjective: based on human observers
! Objective: mathematically defined criteria
Subjective Fidelity Criteria
28
rd
edition
Fidelity Criteria
4/3/14
29
1/ 2
2
1 1
0 0
Let ( , ) be an input image and ( , ) be an approximation
of ( , ). The images are of size .

The - - is
1
( , ) ( , )
M N
rms
x y
f x y f x y
f x y M N
root mean square error
e f x y f x y
MN
!
" "
!
= =
#
$ %
$ %
= "
& '
& '
( )
( )
**
ms
2
1 1
0 0
ms 2
1 1
0 0
The - - - of the output image,
denoted SNR
( , )
SNR
( , ) ( , )
M N
x y
M N
x y
mean square signal to noise ratio
f x y
f x y f x y
! !
"
= =
! !
"
= =
# $
% &
' (
=
# $
!
% &
' (
))
))
Christine P. Hendon
4/3/14 30
Fidelity Criteria:
Subjective vs Objective
! Three approximations of
RMSE = 5.17, excellent
RMSE = 15.67, marginal RMSE = 14.17, unusable
rd
edition
Christine P. Hendon
4/3/14 31
Image Compression
Models
! Mapper: transforms input data in a
way that facilitates reduction of
interpixel redundancies.
! Quantizer: reduces the accuracy of
the mappers output in accordance
with some pre-established fidelity
criteria.
! Symbol encoder: assigns the shortest
code to the most frequently occurring
output values.
! Inverse steps are performed.
! Note that quantization is irreversible
in general.
Encoder Decoder
rd
edition
Christine P. Hendon
Image Compression Standards
4/3/14
32
rd
edition
Christine P. Hendon
4/3/14 33
rd
edition
Christine P. Hendon
4/3/14 34
rd
edition
Christine P. Hendon
Huffman Coding (coding
redundancy)
35
! A variable-length coding technique.
! Symbols are encoded one at a time!
! There is a one-to-one correspondence between source
symbols and code words
! Optimal code (i.e., minimizes the number of code
symbols per source symbol).
Huffman
Coding
36
! Forward Pass
1. Sort probabilities per symbol
2. Combine the lowest two probabilities to form
compound symbol. Compound system and
probability are placed in the first source reduction
column
3. Repeat Step2 until only two probabilities remain.
rd
edition
Huffman Coding
37

! Backward Pass
Assign code symbols going backwards

The average length of this code is
0.4*1 0.3*2 0.1*3 0.1*4 0.06*5 0.04*5
= 2.2 bits/pixel
avg
L = + + + + +
rd
edition
Golomb Coding
4/3/14
38
Given a nonnegative integer and a positive integer divisor 0,
the Golomb code of with respect to , denoted ( ), constructed
as follows:
Step 1. Form the unary code of quotient /
(The unary
m
n m
n m G n
n m
>
! "
# $
2
code of integer is defined as 1s followed by a 0)
Step2. Let k= log , 2 , mod , and compute truncated
remainder ' such that
truncated to -1 bits 0
'
trunca
k
q q
m c m r n m
r
r k r c
r
r c
= % =
& '
! "
( <
=
+ ted to bits otherwise
Step 3. Concatenate the results of steps 1 and 2.
k
)
*
+
Christine P. Hendon
Golomb Coding
4/3/14
39
2
(The unary code of integer is defined as
1s followed by a 0)
Step2. Let k= log , 2 , mod ,
and compute truncated remainder ' such that

k
n m
q
q
m c m r n m
r
r
! "
# $
= % =
& '
! "
'
truncated to bits otherwise
r k r c
r c k
( <
)
=
*
+
+
4
2
2
4
(9) :
9 / 4 2,
the unary code is 110
k= log 4 2, 2 4 0,
9mod 4 1.
' 01
(9) 11001
G
c
r
r
G
=
! "
# $
= = % =
& '
! "
= =
=
=
Christine P. Hendon
Golomb Coding
4/3/14
40
2
1s followed by a 0)

k
n m
q
q
m c m r n m
r
r
! "
# $
= % =
& '
! "
'
r k r c
r c k
( <
)
=
*
+
+
Christine P. Hendon
Golomb Coding
4/3/14
41
2
1s followed by a 0)

k
n m
q
q
m c m r n m
r
r
! "
# $
= % =
& '
! "
'
r k r c
r c k
( <
)
=
*
+
+
Christine P. Hendon
Arithmetic Coding
4/3/14
42
! Entire sequence of source symbols (or
message) is assigned a single arithmetic
doe word
! Code word defines an interval of real
numbers between 0 and 1
! As the number of symbols in the message
increases, the interval used to represent it
becomes smaller and the number of
information units required to represent the
interval becomes larger
rd
edition
Christine P. Hendon
LZW (Dictionary coding)
4/3/14
43
! LZW (Lempel-Ziv-Welch) coding, assigns fixed-
length code words to variable length sequences of
source symbols, but requires no a priori knowledge
of the probability of the source symbols.
! LZW is used in:
Tagged Image file format (TIFF)
Graphic interchange format (GIF)
Portable document format (PDF)
! LZW was formulated in 1984
Christine P. Hendon
The Algorithm:
4/3/14
44
! A codebook or dictionary containing the
source symbols is constructed.
! For 8-bit monochrome images, the first 256
words of the dictionary are assigned to the
gray levels 0-255
! Remaining part of the dictionary is filled with
sequences of the gray levels
Christine P. Hendon
Important features of LZW
4/3/14
45
1. The dictionary is created while the data are
being encoded. So encoding can be done on the
fly
2. The dictionary is not required to be transmitted.
The dictionary will be built up in the decoding
3. If the dictionary overflows then we have to
reinitialize the dictionary and add a bit to each
one of the code words.
4. Choosing a large dictionary size avoids overflow,
but spoils compressions
Christine P. Hendon
Some Basic Compression Methods:
LZW Coding
4/3/14
46
39 39 126 126
39 39 126 126
39 39 126 126
39 39 126 126
rd
edition
Christine P. Hendon
Decoding LZW
4/3/14
47
! Let the bit stream received be:
39 39 126 126 256 258 260 259 257 126

! In LZW, the dictionary which was used for
encoding need not be sent with the image. A
separate dictionary is built by the decoder, on the
fly, as it reads the received code words.
Christine P. Hendon
Run-Length Coding
4/3/14
48
1. Run-length Encoding, or RLE is a technique used to
reduce the size of a repeating string of characters.
2. This repeating string is called a run, typically RLE
encodes a run of symbols into two bytes , a count and
a symbol.
3. RLE can compress any type of data
4. RLE cannot achieve high compression ratios compared to
other compression methods
Christine P. Hendon
Run-Length Coding
4/3/14
49
5. It is easy to implement and is quick to execute.
6. Run-length encoding is supported by most
bitmap file formats such as TIFF, BMP and PCX
Christine P. Hendon
Run-Length Coding
4/3/14
50
! WWWWWWWWWWWWBWWWWWWWWW
WWWBBBWWWWWWWWWWWWWWWWW
WWWWWWWBWWWWWWWWWWWWWW

! RLE coding:
! 12W1B12W3B24W1B14W

Christine P. Hendon
Symbol-Based Coding
4/3/14
51
! In symbol- or token-based coding, an image is
represented as a collection of frequently
occurring sub-images, called symbols.
! Each symbol is stored in a symbol dictionary
! Image is coded as a set of triplets
! {(x1,y1,t1), (x2, y2, t2), }

Christine P. Hendon
Symbol-Based Coding
4/3/14
52
! As long as the triplet required to locate the symbols
in the image together with the bitmap required to
define them are smaller than the original image,
compression occurs
rd
edition
Christine P. Hendon
Bit-plane coding (interpixel
redundancy)
53
! Process each bit plane individually.
(1) Decompose an image into a series of binary images.
(2) Compress each binary image (e.g., using run-length coding)
rd
edition
Bit-Plane Coding
4/3/14
54
! An m-bit gray scale image can be converted into m binary
images by bit-plane slicing. These individual images are then
encoded using run-length coding.
! Code the bitplanes separately, using RLE (flatten each plane
row-wise into a 1D array), Golomb coding, or any other
lossless compression technique.
! Let I be an image where every pixel value is n-bit long
! Express every pixel in binary using n bits
! Form n binary matrices (called bitplanes), where the i-th
matrix consists of the i-th bits of the pixels of I.

Christine P. Hendon
Bit-Plane Coding
4/3/14
55
! Example: Let I be the following 2x2 image where
the pixels are 3 bits long
101 110
111 011
The corresponding 3 bitplanes are:
1 1 0 1 1 0
1 0 1 1 1 1

Christine P. Hendon
4/3/14
56
! However, a small difference in the gray level of
adjacent pixels can cause a disruption of the run
of zeroes or ones.
! Eg: Let us say one pixel has a gray level of 127
and the next pixel has a gray level of 128.
! In binary: 127 = 01111111
! & 128 = 10000000
! Therefore a small change in gray level has
decreased the run-lengths in all the bit-planes!
Christine P. Hendon
Gray Code
4/3/14
57
1. Gray coded images are free of this problem
which affects images which are in binary
format.
2. In gray code the representation of adjacent
gray levels will differ only in one bit (unlike
binary format where all the bits can change.
Christine P. Hendon
4/3/14
58
! Let g
m-1
.g
1
g
0
represent the gray code
representation of a binary number.
! Then:
1 1
1
2 0
! !
+
=
! " " # =
m m
i i i
a g
m i a a g
In gray code:
127 = 01000000
128 = 11000000
Christine P. Hendon
Gray Coding
4/3/14
59
! To convert a binary number b
1
b
2
b
3
..b
n-1
b
n
to its
corresponding binary reflected Gray code.
! Start at the right with the digit b
n.
If the b
n-1
is 1,
replace b
n
by 1-b
n
; otherwise, leave it unchanged. Then
proceed to b
n-1
.
! Continue up to the first digit b
1
, which is kept the same
since it is assumed to be a b
0
=0.
! The resulting number is the reflected binary Gray code.

Christine P. Hendon
4/3/14
60
rd
edition
Christine P. Hendon
4/3/14
61
rd
edition
Christine P. Hendon
Block Transform Coding
! Compression technique that divides an image into small non-
overlapping blocks of equal size and processes each block
independently using a 2D transform
! Linear transform (i.e. Fourier Transform) used to map block to
set of transform coefficients which are then quantized and
coded
4/3/14
62
rd
edition
Christine P. Hendon
4/3/14
63
1 1
0 0
Consider a subimage of size whose forward, discrete
transform ( , ) can be expressed in terms of the relation
( , ) ( , ) ( , , , )
for , 0,1, 2,..., -1.
n n
x y
n n
T u v
T u v g x y r x y u v
u v n
! !
= =
"
=
=
##
Christine P. Hendon
4/3/14
64
1 1
0 0
Given ( , ), ( , ) similarly can be obtained using the
generalized inverse discrete transform
( , ) ( , ) ( , , , )
for , 0,1, 2,..., -1.
n n
u v
T u v g x y
g x y T u v s x y u v
x y n
! !
= =
=
=
""
Christine P. Hendon
Image transform
4/3/14
65
! Two main types:
-orthogonal transform:
e.g. Walsh-Hdamard transform, DCT

-subband transform:
e.g. Wavelet transform
Christine P. Hendon
Discrete Cosine Transform (DCT)
4/3/14
66
( , , , ) ( , , , )
(2 1) (2 1)
( ) ( ) cos cos
2 2
1
for / 0
where ( / )
2
for / 1, 2,..., -1
r x y u v s x y u v
x u y v
u v
n n
u v
n
u v
u v n
n
! !
" "
"
=
+ +
# $ # $
=
% & % &
' ( ' (
)
=
*
*
=
+
*
=
*
,
Christine P. Hendon
Example
4/3/14
67
rd
edition
Christine P. Hendon
4/3/14 68
In each case, 50% of the resulting coefficients were truncated and taking the inverse
transform of the truncated coefficients arrays.
RMSE = 2.32 RMSE = 1.78
RMSE = 1.13
rd
edition
Fourier
Walsh-
Hdamard
Cosine
E
r
r
o
r
Christine P. Hendon
Reconstruction Error vs Subimage Size
4/3/14
69
! Subimages size nxn,
for n=2,4,8,16,.,512
! Compute transform of
each subimage,
truncating 75% of the
resulting coefficients,
take inverse transform
of truncated array
rd
edition
Christine P. Hendon
Example with DCT
4/3/14
70
! Approximation of Lena using 25% of DCT
coefficients and 2x2, 4x4, 8,8 sub-images
compared with original
rd
edition
Christine P. Hendon
Bit Allocation
4/3/14
71
! The overall process of truncating, quantizing, and coding the
coefficients of a transformed subimage is commonly called bit
allocation
! Zonal coding
! The retained coefficients are selected on the basis of maximum
variance
! Threshold coding
! The retained coefficients are selected on the basis of maximum
magnitude

Christine P. Hendon
4/3/14 72
! Approximations
using 12.5% of
8x8 DCT
coefficients.
! Threshold
coding
! Zonal coding
RMSE = 6.5
RMSE = 4.5
rd
edition
Christine P. Hendon
Threshold Coding
4/3/14
73
( ) ( )
( , )
( , )
( , )
(0, 0) 0,1 ... 0, 1
(1, 0) ... ... ...
.... ... ... ...
( 1, 0) ... ... ( 1, 1)
T u v
T u v round
Z u v
Z Z Z n
Z
Z
Z n Z n n
!
" #
=
$ %
& '
" ( #
$ %
$ %
=
$ %
$ %
( ( (
& '
Christine P. Hendon
Threshold Coding
4/3/14
74
rd
edition
Christine P. Hendon
4/3/14 75
Threshold Coding
12:1
30:1
19:1
85:1
49:1
182:1
rd
edition
Christine P. Hendon
Fact about JPEG Compression
4/3/14
76
! JPEG stands for Joint Photographic Experts Group
! Used on 24-bit color files.
! Works well on photographic images.
! Although it is a lossy compression technique, it yields an
excellent quality image with high compression rates.
Christine P. Hendon
Fact about JPEG Compression
4/3/14
77
! It defines three different coding systems:
1. a lossy baseline coding system, adequate for most
compression applications

2. an extended coding system for greater compression, higher
precision, or progressive reconstruction applications

3. A lossless independent coding system for reversible
compression
Christine P. Hendon
Steps in JPEG Compression
4/3/14
78
1. (Optionally) If the color is represented in RGB mode, translate it to YUV.

2. Divide the file into 8 X 8 blocks.

3. Transform the pixel information from the spatial domain to the frequency
domain with the Discrete Cosine Transform.

4. Quantize the resulting values by dividing each coefficient by an integer
value and rounding off to the nearest integer.

5. Look at the resulting coefficients in a zigzag order. Do a run-length
encoding of the coefficients ordered in this manner. Follow by Huffman
coding.
Christine P. Hendon
Step 1a: Converting RGB to YUV
4/3/14
79
! YUV color mode stores color in terms of its luminance
(brightness) and chrominance (hue).
! The human eye is less sensitive to chrominance than luminance.
! YUV is not required for JPEG compression, but it gives a better
compression rate.
Christine P. Hendon
RGB vs. YUV
4/3/14
80
! Its simple arithmetic to convert RGB to YUV. The formula is
based on the relative contributions that red, green, and blue
make to the luminance and chrominance factors.
! There are several different formulas in use depending on the
target monitor.

For example:
Y = 0.299 * R + 0.587 * G +0.114 * B
U = -0.1687 * R 0.3313* G + 0.5 * B +128
V = 0.5 * R 0.4187 * G 0.813 * B + 128
Christine P. Hendon
Discrete Cosine Transform
4/3/14
81
! The DCT transforms the data from the spatial domain to the
frequency domain.
! The spatial domain shows the amplitude of the color as you
move through space
! The frequency domain shows how quickly the amplitude of the
color is changing from one pixel to the next in an image file.
Christine P. Hendon
Step 3: DCT
4/3/14
82
! The frequency domain is a better representation for the data
because it makes it possible for you to separate out and
throw away information that isnt very important to human
perception.
! The human eye is not very sensitive to high frequency changes
especially in photographic images, so the high frequency
data can, to some extent, be discarded.
Christine P. Hendon
Step 3: DCT
4/3/14
83
! The color amplitude information can be thought of as a wave
(in two dimensions).
! Youre decomposing the wave into its component frequencies.
! For the 8 X 8 matrix of color data, youre getting an 8 X 8
matrix of coefficients for the frequency components.
Christine P. Hendon
Step 4: Quantize the Coefficients
Computed by the DCT
4/3/14
84
! The DCT is lossless in that the reverse DCT will give you back
exactly your initial information (ignoring the rounding error
that results from using floating point numbers.)
! The values from the DCT are initially floating-point.
! They are changed to integers by quantization.

Christine P. Hendon
Step 4: Quantization
4/3/14
85
! Quantization involves dividing each coefficient by an integer
between 1 and 255 and rounding off.
! The quantization table is chosen to reduce the precision of
each coefficient to no more than necessary.
! The quantization table is carried along with the compressed
file.
Christine P. Hendon
Step 5: Arrange in zigzag order
4/3/14
86
! This is done so that the coefficients are in order of increasing
frequency.
! The higher frequency coefficients are more likely to be 0 after
quantization.
! This improves the compression of run-length encoding.
! Do run-length encoding and Huffman coding.
Christine P. Hendon
Example JPEG Approximation
4/3/14
87
! Compression of 25:1 # RMSE = 5.4
! Compression of 52:1# RMSE = 10.7
! JPEG blocking artifact increases with compression
rd
edition
Christine P. Hendon
Predictive Coding
! Eliminating redundancies of closely spaced pixels in space and/or time
! New information of a pixel is defined as the difference between actual
and predicted value of pixel
! Can be implemented as lossless or lossy
88
Predictive Coding and Spatial
Redundancy
89
Predictive Coding and Temporal
Redundancy
90
! Coding differences between successive frames can
reduce temporal redundancy and provide
significant compression
! Video compression systems avoid problem of data
expansion by
! Tracking object movement and compensating for it
during the predication and differencing process
! Switching to alternate coding method when there is
insufficient inter-frame correlation to make predictive
coding advantageous
Motion Compensated
Prediction Residuals
91
! When there is sufficient inter-frame correlation to
make predictive coding effective, block-oriented 2-
D transform, like JPEG DCT coding
! These types of frames are called intra-frames or
independent (can be decoded without knowledge of
other frames)
! Helps to reduce propagation of errors
! Macroblocks, Reference Frame, Motion Vector
! Error measurements: Mean absolute distortion
Motion
Compensation
92
Motion Compensation
93
Subpixel Motion Compensated
Prediction Residuals
94
Predictive coding in video
compression standards
95
Motion Compensated Video
Encoder
96
Lossless Predictive Coding Model
98
Optimal Quantization
99
Wavelet Coding
! Wavelet coding based on idea that coefficients of a transform that
decorrelates the pixels of an image can be coded for efficiently than
original pixels themselves
! Do not need subimage processing stages of the transformer coder
! Eliminates blocking artifacts characterized by DCT based
approximations at high compression ratios
100
Image Pyramid
! Representing images at more than one resolution
! Collection of decreasing resolution images
arranged in shape of pyramid
! As move up pyramid, both size and resolution of
image decreases
101
rd
edition
Example Haar scaling functions
! If f(x) is an element
of V
0
, it is also an
element of V
1.
All
V0 expansion are
contained in V1

102
rd
edition
! x ( ) =
1 0 " x<1
0 otherwise
#
$
%
&
'
(
f x ( ) = 0.5!
1,0
x ( ) + 0.5!
1,1
x ( ) " 0.25!
1,4
x ( )
CWT: Main Steps
1. Take a wavelet and compare it to a section at the start of
the original signal.
2. Calculate a number, C, that represents how closely
correlated the wavelet is with this section of the signal.
The higher C is, the more the similarity.
103
CWT: Main Steps
3. Shift the wavelet to the right and repeat steps 1 and 2 until
you've covered the whole signal.

104
CWT: Main Steps
4. Scale the wavelet and repeat steps 1 through 3.

5. Repeat steps 1 through 4 for all scales.

105
Example: Computing a 2-D three-scale
FWT
a. Original 128 x 128
computer generated
image with sine-like
pulses
b. One scale FWT
c. Two scale FWT
d. Three scale FWT
rd
edition
3
-
S
c
a
l
e

T
r
a
n
s
f
o
r
m

rd
edition
! Initial decompositions are responsible for the
majority of the data compression
! Little change in number of truncated coefficients
above three decomposition levels
Decomposition Level Impact on
Wavelet Coding
rd
edition
108
JPEG 2000
! Extends popular JPED standard
to provide increased flexibility
in both compression of
continuous-tone still images and
access to the compressed data
! Coefficients quantization is
adapted to individual scales
and subbands and the
quantized coefficients are
coded on a bit-plane basis
rd
edition
Two-Scale Wavelet
Transform
109
rd
edition
110
Digital Image Watermarking
! Watermarked images protect rights of owners
! Copy right identification
! User identification or finger printing
! Authenticity determination
! Copy protection
f
w
= 1!" ( ) f +"w
rd
edition
111
Invisible Watermark
! Can not be seen with the
naked eye
! Can be recovered with
appropriate decoding
algorithm
! Invisibility assured by
inserting them as
visually redundant
information
rd
edition
112
Watermarking System
rd
edition
113
Example: DCT based invisible
robust watermark
rd
edition
114
4/3/14
115
Christine P. Hendon
E4830 DIGITAL IMAGE
PROCESSING
SPRING 2014
Morphological Processing
Lecture 9
April 10, 2014
Todays Lecture
! Binary morphological processing
! Erosion and dilation
! Opening and closing
! Gray-scale morphological processing
! Erosion and dilation
! Morphological gradients
Roger S. Gaborski 2
3
Mathematic Morphology
! Used to extract image components that are useful in
the representation and description of region shape,
such as
! boundaries extraction
! skeletons
! convex hull
! morphological filtering
! thinning
! pruning
4
Z
2
and Z
3
! Set in mathematic morphology represent objects in
an image
! binary image (0 = white, 1 = black) : the element of
the set is the coordinates (x,y) of pixel belong to the
object ! Z
2

! gray-scaled image : the element of the set is the
coordinates (x,y) of pixel belong to the object and the
gray levels ! Z
3
! Every element of a set A is also
an element of another set B, then
A is said to be a subset of B
! Union of two sets A and B
! Intersection of two sets A and B
! Disjoint or mutually Exclusive
The language of mathematical
morphology is Set Theory
5
rd
edition
! The complement of a set A is the set of
elements not contained in A
! The difference of two sets A and B,
denoted A-B
A ! B

C = A! B

D = A! B

A! B = !
A
c
= w| w !A
{ }

A! B = w| w "A, w #B { } = A! B
c
! Reflection of a set B denoted

! Translation of a set B by point
z=(z
1
,z
2
) denoted (B)
z
Background: Set Theory
(a) Set B, (b) its reflection, (c) its
translation by z
B = w| w = !b, for b "B { }

B ( )
z
= c | c = b + z, for b !B { }
6
rd
edition
B
! Using logical expression in MATLAB to perform set
operations on binary images
Background: Set Theory
7
rd
edition
Example: Set Theory
8
rd
edition
! Set reflection and translation
employed extensively in morphology
to formulate operations based on
structure elements
! (SEs): small sets or subimages used to
probe an image under study for
properties of interest
! Origin of structuring element must be
specified
Structure Elements
9
rd
edition
Example: Using Structuring Elements
10
rd
edition
Erosion
! Erosion of image f by structuring element s is
given by f ! s
! The structuring element s is positioned with its
origin at (x, y) and the new pixel value is
determined using the rule:
! The erosion of A by B is the set of all points z
such that B, translated by z, is contained in A.
! B being contained in A equivalent to B not
sharing any common elements with the
background
A!B = z | B ( )
z
" A
{ }
A!B = z | B ( )
z
#A
c
= $
{ }
11
!
"
#
=
otherwise 0
fits if 1
) , (
f s
y x g
rd
edition
Example: Erosion
12
rd
edition
Example: Erosion
13
rd
edition
! Eroding image with square structure
element 11x11, 15x15, 45x45
! Erosion shrinks or thins a binary image
! Can view erosion as morphological
filtering operation in which image
details smaller than the structuring
element are filtered (removed) from
the image
Example: Erosion
14
rd
edition
Example: Erosion
Original image Erosion by 3*3
square structuring
element
Erosion by 5*5
square structuring
element
rd
edition
Example: Erosion
! Erosion can split apart joined objects
! Erosion can strip away extrusions
! But note that erosion will also shrink objects
Erosion can split apart
! Dilation of image f by structuring element
s
! The structuring element s is positioned
with its origin at (x, y) and the new pixel
value is determined using the rule:
! Dialation of A by B is the set of all
displacements z, such that B_hat and A
overlap by at least one element
! Similar to convolution but based on set
operation " nonlinear
Dilation
A!B = z |

B
( )
z
"# $
{ }
A!B = z |

B
( )
z
"A
%
&
'
(
) A
{ }
17
!
"
#
=
otherwise 0
hits if 1
) , (
f s
y x g
rd
edition
Example: Dilation
18
rd
edition
Example:
Dilation
! Application of dilation: bridging/repairing gaps
! Dilation grows or thickens objects in a binary image
! The manner and extent of thickening is controlled by shape of the
structuring element
! Advantage of morphological method instead of low-pass filtering is that we
will end up with a binary image, instead of a grayscale image that needs
to thresholding
rd
edition
Example: Dilation
Original image
Dilation by 3*3
square structuring
element
Dilation by 5*5
square structuring
element
What Is Dilation For?
! Dilation can repair breaks

! Dilation can repair intrusions
! But note that dilation enlarges objects
Duality
! Erosion and dilation are duals of each other with
respect to set complementation and reflection
22
A!B
( )
c
= A
c
"

B
A"B
( )
c
= A
c
!
B
Compound Operations
! More interesting morphological operations can be
performed by performing combinations of erosions
and dilations
! The most widely used of these compound operations
are:
! Opening
! Closing
rd
edition
! The opening of image f by structuring element s,
denoted f s is simply an erosion followed by a
dilation
! Opening: generally smooths the contour of an
object, breaks narrow isthmus, and eliminates
protusions
Opening

A! B = A!B ( ) "B
A! B = " B ( )
z
| B ( )
z
# A
{ }
24
rd
edition
! The closing of image f by structuring element s, denoted f s is
simply a dilation followed by an erosion
! Closing also smooths sections of contours, but it generally fuses
narrow breaks and long thin gulfs, eliminates small holes and
fills gaps in the contour
Closing
A B = A!B ( )"B
25
rd
edition
Morphological
Opening and
Closing
! Opening and closing are
duals of each other with
respect to set
complementation and
reflection
rd
edition

A B ( )
c
= A
c
!

B
( )
A! B ( )
c
= A
c

B
( )
Example: Opening and Closing
27
rd
edition
Example: Morphological operations to
construct filters
! Objective: eliminate the noise and
its effects in the print while
distorting it as little as possible
! Morphological filter: opening
followed by closing
rd
edition
! Morphological hit or miss transformation is a basic
tool for shape detection
! Objective: find location of one of the shapes among
set of shapes template matching
! Composite SE: object part (B1) and background
part (B2). Does B1 fit the object while
simultaneously B2 misses the object, i.e. fits the
background?
Hit or Miss Transformation
29
rd
edition
A! B = (A"X) #[A
c
"(W $ X)]
Hit or Miss
Transformation
rd
edition
Example: Hit or Miss
31
rd
edition
Review of Basic Morphological
Algorithms
! Using the simple technique we have looked at so far
we can begin to consider some more interesting
morphological algorithms
! We will look at:
! Boundary extraction
! Region filling
! There are lots of others as well though:
! Extraction of connected components
! Thinning/thickening
! Skeletonization
32
! Extracting the boundary (or outline) of an object is often
extremely useful
! The boundary of a set A, denoted (A), can be obtained by
first eroding A by B and then performing the set difference
between A and its erosion
!
Boundary Extraction
33
rd
edition
! A
( )
= A (A"B)
! A simple image and the result of performing boundary extraction using a
square 3*3 structuring element
! Because of the size of the structuring element used, the boundary is only one
pixel thick
Example: Boundary Extraction
34
rd
edition
! Hole defined as background
region surrounded by
connected border of
foreground pixels
! Given a pixel inside a
boundary, region filling
attempts to fill that boundary
with object pixels (1s)
Hole/Region Filling
35
Hole/Region Filling
! The key equation for region filling is
! Where X
0
is simply the starting point inside the
boundary, B is a simple structuring element and A
c
is
the complement of A
! This equation is applied repeatedly until X
k
is equal
to X
k-1

! Finally the result is unioned with the original
boundary
36
..... 3 , 2 , 1 ) (
1
= ! " =
#
k A B X X
c
k k
! Example: image with circles with black inner spots
! Objective: remove reflection by holes
Example: Morphological Hole
Filling
37
rd
edition
Extraction of Connected
Components
! Objective: start with X
0

and find all the
connected components
! Procedure terminates
when X
k
=X
k-1
38
rd
edition
X
k
= (X
k!1
"B) #A k =1, 2, 3.....
Example: Using connected components to
detect foreign objects in packaged food
rd
edition
Labeling Connected Components
40
rd
edition
Neighbors and Connections
41
Connected Components and Labels
42
rd
edition
Example: Labeling Connected
Components
43
rd
edition
! A set A is said to be convex if
the straight line segment
joining any two points in A lies
entirely within A
! The convex hull H of an
arbitrary set S is the smallest
convex set containing S
! H-S called convex deficiency
of S
Convex Hull
44
rd
edition
Thinning
! The process to thin A by one pass with B
1
,
then to think the result with one pass of
B
2
, and so on, until A is thinned with one
pass of B
n
! The entire process is repeated until no
further changes occur
45
rd
edition

A!B = A" A# B
( )
= A! A# B
( )
c
Example: Thinning
46
rd
edition
Thinning
rd
edition
Thickening
48
rd
edition
! Thickening is the morphological dual of thinning
! Same structuring element as thinning but with 1s and 0s
interchanged
! Usual procedure is to thin the background of the set in question
and then complement the result " to thicken set A, C=A
c
, thin
C, then form C
c
! Can produce discontinuous points " post processing to remove
will be needed
Skeleton
49
rd
edition
Skeleton
50
rd
edition
Example: Skeleton
51
rd
edition
! Pruning methods are
essential complement to
thinning and skeletonizing
algorithms because these
procedures tend to leave
parasitic components that
need to be cleaned up
Pruning
52
rd
edition
Morphological Reconstruction
! Morphological reconstruction involves two images
and a structuring element
! Marker: contains starting points for transformation
! Mask: Constrains the transformation
! Structuring element used to define connectivity
53
! Geodesic dilation of size 1 of the marker with
respect to mask
! Geodesic dilation of size n of F with respect to G
! Mask will limit growth (dilation) of marker F
Geodesic Dilation
54
rd
edition

D
G
1 ( )
F ( ) = F !B ( ) !G
D
G
n ( )
F ( ) = D
G
1 ( )
D
G
n!1 ( )
F ( )
"
#
$
%
! Geodesic erosion of size 1 of marker F with respect
to mask G
! Geodesic erosion of size n of F with respect to G
Geodesic Erosion
55
rd
edition

E
G
1 ( )
F ( ) = F!B ( ) !G
E
G
n ( )
F ( ) = E
G
1 ( )
E
G
n!1 ( )
F ( )
"
#
$
%
Morphological reconstruction by
dilation and by erosion
! Morphological reconstruction by dilation of a mask image G from a marker
image F, is defined as the geodesic dilation of F with respect to G, iterated
until stability is achieved
! Morphological reconstruction by erosion of a mask image G from a marker
image F defined as the geodesic erosion of F with respect to G, iterated
until stability
56
rd
edition
Example: Morphological
Reconstruction
57
rd
edition
Example: opening by reconstruction
58
rd
edition
Filling holes
59
rd
edition
Example: Text hole filling
60
rd
edition
Border clearing
61
rd
edition
Basic types of
structuring
elements
rd
edition
Summary of morphological
operations and their properties
63
rd
edition
64
rd
edition
65
rd
edition
MATLAB
rd
edition
! Structure elements in gray-scale
morphology perform same basic
function as binary counterparts: used
as probes to examine for specific
properties
! Structure elements in grayscale
morphology belong to two
categories
! Nonflat
! flat
Gray-Scale Morphology
67
rd
edition
Erosion and Dilation: Similar to
spatial convolution
! Dilation: local maximum operation
# The general effect of performing dilation on a gray scale image
is twofold:
# If all the values of the structuring element are positive, the output
image tends to be brighter than the input
# Darker details either are reduced or eliminated depending on how
their values and shapes relate to the structuring element used for
dilation
! Erosion: local minimum operation
# The general effect of performing erosion on a gray-scale image
is two fold
# If all the elements of the structuring element are positive, the output
image tends to be darker than the input
# The effect of bright details in the input image that are smaller than
the structuring element are reduced
4/10/14
68
Example: Gray-scale Erosion and
Dialation
69
rd
edition
Example: Dilation and Erosion
70
rd
edition
Opening and Closing
71
rd
edition

f !b = f !b
( )
"b
f b = f "b
( )
!b
f b
( )
c
= f
c
!

b
f !b
( )
c
= f
c

b
Example: Gray-scale opening and
closing
72
rd
edition
Some Basic Gray-Scale
Morphological Algorithms
! Morphological smoothing
! Morphological gradient
! Top-hat transformation
! Textural segmentation
73
! Because opening suppresses
bright details smaller than SE
and closing suppresses dark
details, they are used in
combination as
morphological filters for
image smoothing and noise
removal
Morphological Smoothing
74
rd
edition
! Morphological gradient:
dilated image eroded
image
! Half gradient by erosion:
image eroded image
! Half gradient by dilation:
image dilated image
! Directional gradient: function
of structure element
Morphological Gradient
75
rd
edition
g = f !b ( ) " f #b ( )
Top-hat and bottom-hat
transformations
! Applications: removing objects from an image by using a structuring
element with opening or closing operation that does not fit the object to be
removed
! Top-hat " used for light objects on dark background. Bottom-hat
(converse)
! Also for correction of nonuniform illumination

T
hat
f ( ) = f ! f !b ( )
B
hat
f ( ) = f b ( ) ! f
76
rd
edition
! Granulometry: determining the size distribution of particles in
an image
! Morphology can be used to estimate particle size distribution
indirectly without having to identify and measure every
particle
Granulometry
77
rd
edition
! Objective: to find boundary
between two regions based on
their textural context
Textural Segmentation
78
rd
edition
Example: Textural Segmentation
79
rd
edition
Gray-scale morphological
reconstruction
! f and g marker and mask
! Geodesic dilation
! Geodesic erosion
80
D
g
1 ( )
f ( ) = f !b ( ) " g
D
g
n ( )
f ( ) = D
g
1 ( )
D
g
n#1 ( )
f ( )
$
%
&
'
E
g
1 ( )
f ( ) = f !b ( ) " g
E
g
n ( )
f ( ) = E
g
1 ( )
E
g
n#1 ( )
f ( )
$
%
&
'
Example: Gray-scale
morphological reconstruction
81
rd
edition
82
E4830 DIGITAL IMAGE
PROCESSING
SPRING 2014
Image Segmentation
Lecture 10
April 17, 2014
! Erosion
! Dilation
! Opening
! Closing
2
Review: Morphological Processing
rd
edition
Todays Class
! Image Segmentation, Chapter 10
! Overview
! Point, Line, and Edge Detection
! Thresholding
! Region-based Segmentation
! Segmentation using Morphological Watershed
! The use of Motion in Segmentation
3
Overview: Image Segmentation
! Segmentation subdivides an image into its
constituent regions or objects.
! The level to which the subdivision is carried depends
on the problem being solved.
! Image segmentation algorithms generally are
based on one of two basic properties of intensity
values: discontinuity and similarity.
4
Overview: Image Segmentation
! One approach is to partition an image based on
abrupt changes in intensity, such as edges in an
image.
! Second set of approaches are based on
partitioning an image into regions that are similar
according to a set of pre-defined criteria.
! Thresholding, region growing, and region splitting and
merging
5
Example: Segmentation based on
region properties
rd
edition
Research Examples
4/17/14
7
Christine P. Hendon
8
Lipid rich: identified by signal poor
region with diffuse boundaries with
overlying signal rich band
Fibrocalcific: Signal
poor region with sharp
borders
Fibrous: homogenous, highly
backscattering, devoid of signal poor
regions
CAD
9
Comprehensive
Intravascular Imaging
Tearney 2008
Lipid
Macrophages
Calcium
Stent
Artery Wall
5
m
m

CAD
(manual segmentation)
https://www.youtube.com/watch?feature=player_embedded&v=m9dkoGY4-7E
Jenkins 2008, 2009
Point, Line, and Edge Detection
! Segmentation methods based on detection sharp,
local changes in intensity
! Types of image features
! Points
! Lines
! Edges
11
Detection of Discontinuities: Spatial
Filter Mask
! Approach of choice for computing first and second
derivative at every pixel located in an image is to
use a spatial filter
! Response of the mask at the center point of region,
R:
12
R = w
1
z
1
+ w
2
z
2
+... + w
9
z
9
= w
i
z
i
i=1
9
!
rd
edition
Detection of Isolated Points
! Point detection based on the second derivative "
Laplacian

! Create a binary image
13
!
2
f x, y ( ) =
"
2
f
"x
2
+
"
2
f
"y
2
!
2
f x, y ( ) = f x +1, y ( ) + f x #1, y ( ) + f x, y +1 ( ) + f x, y #1 ( ) # 4 + f x, y ( )
g x, y ( ) =
1 if R x, y ( ) ! T
0 otherwise
"
#
$
%
$
&
'
$
(
$
Derivatives
! Abrupt local
change in intensity
can be detected
using derivatives
rd
edition
Example: Laplacian mask for point
detection
! Intuitively, the idea is that the
intensity of an isolated point
will be quite different from its
neighbors and detective with
this type of mask (weighted
difference between 8-
neighbors)
rd
edition
Line Detection
! For line detection, we can expect second derivative
to result in a stronger response and produce thinner
lines than first derivative
! Laplacian mask for line detection
16
Example: Line Detection using
Laplacian Mask
! Laplacian detector
is isotropic
(independent of
direction)
rd
edition
Line Masks
! Often, interest lies in detecting lines in a specific
direction
rd
edition
Example: +45
o

line detector
mask
rd
edition
Edge Detection
! Edge detection is the most common approach for detecting
meaning discontinuities (abrupt local change in intensity) in
gray scale image.
! Intuitively, an edge is a set of connected pixels that lie on the
boundary between two regions.
! A reasonable definition of edge requires the ability to
measure gray-level transitions in a meaningful way.
! Edge detecting operators can also mark image locations of
discontinuities in grey-level, color, texture, etc
20
Edge Models
! An ideal edge has the properties has the properties of the
model of a step.
! In the real world, digital images have edges that are blurred
and noisy. Therefore, edges are more closely modeled as
having ramp profiles
! Roof edges are models of lines through regions with the width
of the roof edge determined by the thickness and sharpness of
the line
! Blurred edges tend to be thick and sharp tend to be thin.
21
rd
edition
Edge Detection
! We conclude from these observations that the
magnitude of the first derivative can be used to
detect the presence of an edge at a point in an
image.
! Similarly, the sign of the second derivation can be
used to determine whether an edge pixel lies on the
dark or light side of an edge.
22
Edge Detection
! Two additional properties of the second derivative
around an edge:
! It produces two value for every edge in an image
! an imaging straight line joining the extreme positive
and negative values of the second derivative would
cross zero near the midpoint of the edge.
! The zero-crossing property of the second derivative
is quite useful for locating the centers of thick edges.
Example: Edges in an Image
rd
edition
! First derivative
! positive at the onset of the
ramp an points within the
ramp.
! Zero in areas of constant
intensity
! Second derivative
! Positive at beginning of
ramp
! Negative at end of ramp
! Zeros at points on the
ramp and points of
constant intensity
Derivatives within edges
rd
edition
Behavior of the
first and second
derivative of a
noisy image
rd
edition
Fundamental Steps Performed in
Edge Detection
1. Image smoothing for noise reduction
2. Detection of edge points
3. Edge localization
27
The Image Gradient and its Properties "
to find edge strength and direction
! The gradient of an image f(x,y) at location (x,y) is
defined as the vector
! Let (x,y) represent the direction angle of the
gradient. Then, from vector analysis,
! Where the angle is measured with respect to the x-
axis. The direction of an edge at (x,y) is
perpendicular to the direction of the gradient vector
at that point.
28
!f =
g
x
g
y
"
#
$
%
&
'
=
( f
(x
( f
(y
"
#
$
$
$
$
%
&
'
'
'
'
M x, y ( ) = mag !f ( ) = g
x
2
+ g
y
2
"
#
$
%
1 2
! x, y ( ) = tan
"1
g
y
g
x
#
$
%
&
'
(
Example: Properties of the
Gradient
! Edge is perpendicular to the direction of the
gradient vector at the point where the gradient is
computed
rd
edition
Gradient Operators
! One of the simplest ways to implement a first-order
partial derivative at point z
5
is to use the following
Roberts cross-gradient operators:
g
x
=(z
9
-z
5
)
g
y
=(z
8
-z
6
)
! An approach using masks of size 3x3 is given by called
the Prewitt operators
g
x
=(z
7
+z
8
+z
9
)-(z
1
+z
2
+z
3
)
g
y
=(z
3
+z
6
+z
9
)-(z
1
+z
4
+z
7
)
! A slight variation of these two equations uses a weight
of 2 in the center coefficient, Sobel operator:
g
x
=(z
7
+2z
8
+z
9
)-(z
1
+2z
2
+z
3
)
g
y
=(z
3
+2z
6
+z
9
)-(z
1
+2z
4
+z
7
)
30
rd
edition
Sobel and Prewitt Operators
! A weight value of 2 is used to achieve
some smoothing by giving more
importance to the center point, called the
Sobel operators.
! The Prewitt and Sobel operators are
among the most used in practice for
computing digital gradients.
! Prewitt masks are simpler to implement
than the Sobel masks, but the later have
slightly superior noise-suppression
characteristics, an important issue when
dealing with derivatives.
31
rd
edition
Gradient Operators
! An approach used frequently is to approximate the
gradient by absolute values:
32
M(x, y) ! g
x
+ g
y
Example: 2D Gradient Magnitude
rd
edition
Example: 2D Gradient Angle
rd
edition
Example: Smoothing Prior to
Gradient
rd
edition
Example: Diagonal Detection
rd
edition
! Fewer edges in the thresholded image and edges
are much sharper
! However, numerous edges are broken
! " to highlight principal edges and maintain as
much connectivity, common to use both smoothing
and thresholding
Combining the Gradient with
Thresholding
rd
edition
The Laplacian
! The Laplacian of a 2-D function (x,y) is a second-
order derivative defined as
! Digital approximations to the Laplacian were
introduced in Chapter 3.
38
!
2
f = 4z
5
" z
2
+ z
4
+ z
6
+ z
8
( )
The Laplacian
! A digital approximation including the diagonal
neighbors is given by
! As a second-order derivative, the Laplacian
typically is sensitive to noise.
! The Laplacian is unable to detect edge direction.
39
!
2
f = 8z
5
" z
1
+ z
2
+ z
3
+ z
4
+ z
5
+ z
6
+ z
7
+ z
8
+ z
9
( )
The Marr-Hildreth Edge Detection
! Marr and Hildreth argued that
1. That intensity changes are not independent of image scale and so
detection requires the use of operators of different sizes
2. Sudden intensity change will give rise to peak or trough in the first
derivative or to zero crossings in the second derivative
! Operators used for edge detection
! Should be capable of being at any desired scale
# Larger operator can be used to detect blurry edges and small
operators to detect sharp fine detail
! Laplacian of Gaussian
40
Laplacian of Gaussian
! Mexican hat
operator
! Masks of
arbitrary size can
be generated
rd
edition
The Laplacian of Gaussian (LoG)
! Role of the Laplacian in segmentation consist of (1)
using its zero-crossing property for edge location,
(2) using it for the complementary purpose of
establishing whether a pixel is on the dark or light
side of an edge.
! Convolution with a Gaussian performs low pass
filtering to reduce noise
! The combination is commonly referred to as the
Laplacian of a Gaussian (LoG)
42
Selection of LoG operator
! Gaussian part of the operator blurs the image "
reducing the intensity of structures (including noise)
at scales much smaller than
! Because Gaussian is smooth in both spatial and
frequency domain, less likely to produce artifacts
! First derivatives can be used for edge detection.
Laplacian is used because it is isotropic (invariant to
rotation)
43
Marr-Hildreth Edge Detection
Algorithm
1. Filter the input image with an nxn Gaussian low pass filtered
obtained by sampling G(x,y)
2. Compute the Laplaican of the image resulting from Step 1
3. Find the zero crossing of the image from step 2
! To specify the size of the Gaussian filter, remember 99.7% of
volume under a 2D Gaussian surface lies between 3 about
the mean
! " as rule of thumb, nxn LoG discrete filter should be such that n is
the smallest odd integer greater or equal to 6
44
Example: Marr-Hildreth Algorithm
! The LoG is the
image from which
zero crossings are
computed to find
edges.
! One approach for
approximating zero
crossings is to
threshold the LoG
image by setting all
its positive values
to, say, white, and
all negative values
to black.
rd
edition
! Possible to approximate the LoG filter by a
difference of Gaussian (DoG)
! Both LoG and DoG operators can be implemented
in 1D
Difference of Gaussian (DoG)
rd
edition
The Canny Edge Dector
! Cannys approach based on three objectives
1. Lower error rate
2. Edge points should be well localized
3. Single edge response
47
Canny
! Because it is generated using the gradient, M(x,y),
typically contains wide ridges around local maxima
! " need to thin ridges
! One approach: non-maxima suppression
48
! Let d
1
, d
2
, d
3
, d
4
denote the four basic
edge directions for a 3x3 region.
Nonmaximum support scheme for a 3x3
region centered at every point (x,y) in
!(x,y)
1. Find the direction d
k
closest to !(x,y)
2. If the value of M(x,y) is less than at
least one of its two neighbors along d
k
,
let g
N
(x,y)= suppression, otherwise let
g
N
(x,y)=M(x,y)
3. Then threshold g
N
(x,y). If threshold is
too low, there will still be false edges. If
threshold is too high, actual valid edges
will be eliminated
Canny
rd
edition
Edge Linking and Boundary
Detection
! One of the simplest approaches for linking edge
points is to analyze the characteristics of pixels in a
small neighborhood (say, 3x3 or 5x5).
! All points that are similar according to a set of
predefined criteria are linked.
! The two principal properties used for establishing
similarity of edge pixels in this kind of analysis are
(1) the strength of the response of the gradient
operator used to produce the edge pixel; and (2)
the direction of the gradient vector.
50
Summary of Canny Edge Detection
Algorithm
1. Smooth the input image with a Gaussian filter
2. Compute the gradient magnitude and angle
image
3. Apply non-maxima suppression to the gradient
magnitude image
4. Use double thresholding and connectivity analysis
to detect and link edges
51
Example: Marr Hildreth and Canny
rd
edition
Example: Marr Hildreth and Canny
rd
edition
Example: Edge linking using local
processing
rd
edition
Regional Processing
rd
edition
Example: Edge linking using a
polygon approximation
rd
edition
Example: Polygon fitting of an
image boundary
rd
edition
Global Processing via
the Hough Transform
! Points are linked by determining first if they lie on a
curve of specified shape.
! An approach based on the Hough transform is as
follows:
! Compute the gradient of an image and threshold it to
obtain a binary image.
! Specify subdivisions in the -plane.
! Examine the counts of the accumulator cells for high
pixel concentrations.
! Examine the relationship (principally for continuity)
between pixels in a chosen cell.
58
rd
edition
rd
edition
Illustration
of Basic
Hough
Transform
Properties
rd
edition
Example: Using the Hough
Transform for Edge Linking
rd
edition
Thresholding
! Because of its intuitive properties and simplicity of
implementation, image thresholding enjoys a central
position in applications of image segmentation.
! One obvious way to extract the objects from the
background is to select a threshold T that separates
these modes.
! Then any point (x,y) for which f(x,y)>T is called an
object point; otherwise, the point is called a
background point.
62
rd
edition
! Thresholding may be viewed as an operation that
involves tests against a function T of the form where
f(x,y) is the gray level of point (x,y) and p(x,y)
denotes some local property of this point.
! When T depends only on f(x,y) (that is, only on
graylevel values) the threshold is called global.
! If T depends on both f(x,y) and p(x,y), the threshold
is called local.
! If, in addition, T depends on the spatial coordinates
x and y, the threshold is called dynamic or
adaptive.
63
Role of Noise in Image Thresholding
rd
edition
Basic Global Thresholding
! The following algorithm can be used to obtain T automatically:
1. Select an initial estimate for T.
2. Segment the image using T. This will produce two groups
of pixels: G
1
consisting of all pixels with gray level > T
and G
2
consisting of pixels with values T.
3. Compute the average gray level value m
1
and m
2
for the
pixels in regions G
1
and G
2
.
4. Compute a new threshold value: T=0.5(m
1
+m
2
)
5. Repeat step 2 though 4 until the difference in T in
successive iterations is smaller than a predefined
parameter "T.
66
The Role of Illumination and
Reflection
rd
edition
Using Image Smoothing to Improve
Global Thresholding
68
Example: Global Thresholding
rd
edition
Example: Optimum Global
Thresholding using Otsus Method
rd
edition
Optimum Global Thresholding
Using Otsus Method
71
Example: Using Otsus Method
rd
edition
Using Edges to Improve Global
Thresholding
! Chances of selecting a good threshold are enhanced considerably if the
histogram peaks are tall, narrow, symmetric, and separated by deep
valleys
! One approach for improving the shape of histograms is to consider only
those pixels that lie on or near the edges between objects and the
background
! Histogram will be less dependent on relative sizes of objects and
background
! Resulting histogram would have peaks of approximately the same height
! Using pixels that satisfy some simple measures based on gradient and
Laplacian operators has a tendency to deepen the valley between
histogram peaks
73
Example: Using edge information based on
gradients to improve global thresholding
rd
edition
Example: Using edge information based on
the Laplacian to improve global thresholding
rd
edition
rd
edition
Multiple Thresholds
rd
edition
Basic Adaptive Thresholding
! Divide the original image into subimages and then
utilize a different threshold to segment each
subimage.
! The key issues in this approach are how to subdivide
the image and how to estimate the threshold for
each resulting subimage.
78
! Factors such as noise and non-uniform illumination play a major
role in thresholding algorithm
! Image partitioning " subdivide images into non-overalpping
rectangles
Variable Thresholding
rd
edition
Use of Boundary Characteristics for Histogram
Improvement and Local Thresholding
! Using pixels that satisfy some simple measures
based on gradient and Laplacian operators has a
tendency to deepen the valley between histogram
peaks.
! An indication of whether a pixel is on an edge may
be obtained by computing its gradient.
! In addition, use of the Laplacian can yields
information regarding whether a given pixel lies on
the dark or light side of an edge.
80
Use of Boundary Characteristics for Histogram
Improvement and Local Thresholding
! The transition (along a horizontal or vertical scan line) from a light
background to a dark object must be characterized by the occurrence of a
followed by a + in s(x,y).
! The interior of the object is composed of pixels that are labeled either 0 or
+.
! Finally, the transition from the object back to the background is
characterized by the occurrence of a + followed by a -.
! The innermost parentheses contains object points and are labeled 1.
! All other pixels along the same scan line are labeled 0, with the exception
of any other sequence of (0 or +) bounded by (-,+) and (+,-).
81
E
x
a
m
p
l
e
:

V
a
r
i
a
b
l
e

t
h
r
e
s
h
o
l
d
i
n
g

b
a
s
e
d

o
n

l
o
c
a
l

i
m
a
g
e

p
r
o
p
e
r
t
i
e
s

rd
edition
Variable Thresholding
! Euclidean Distance
D(z,a)=||z-a||
! Mahabolis Distance
D(z,a)=[(z-a)
T
C
-1
(z-a)]
1/2
rd
edition
Example: Global Thresholding and
Local Thresholding
rd
edition
! Segmentation techniques based on finding the
regions direction
! Region growing
! Region splitting
Region based segmentation
rd
edition
Region Growing
! Region growing is a procedure that groups pixels or sub-
regions into larger regions based on predefined criteria.
! The basic approach is to start with a set of seed points and
from these grow regions by appending to each seed those
neighboring pixels that have properties similar to the seed
(such as specific ranges of gray level or color).
! Selection of similarity criteria depends on the problem under
consideration and the type of image data available (i.e. color,
moments, texture)
! Will also need a stopping criteria
86
Example: Segmentation by region
growing
rd
edition
Region Splitting and Merging
! Region grows from a set of seed points
! Alternative is to subdivide an image initially into a
set of arbitrary, disjoint regions, and then merge
and/or split the regions in an attempt to satisfy the
conditions of segmentation
88
Example: Segmentation by Region
Splitting and Merging
rd
edition
Segmentation by Morphological
Watersheds
! So far, discussed segmentation based on three principal
concepts: a) edge detection, b) thresholding, c) region growing
! Each approach has advantages (i.e. speed of global
thresholding) and disadvantages (i.e. need for post-processing,
such as edge linking in edge-based segmentation).
! Morphological Watershed
! Embodies many of the concepts of the three approaches " often
produces more stable segmentation results, including connected
segmentation boundaries
! Also provides framework for incorporation knowledge-based
constraints into segmentation process
90
! Catchment basin or watershed
! Divide lines or watershed lines
rd
edition
Dam Construction
! Based on binary images
! Simplest way to
construct dams
separating sets of
binary pints is to use
morphological dilation
rd
edition
Example: Illustration of Watershed
Segmentation Algorithm
rd
edition
Use of Markers
! Direction application of watershed segmentation
algorithm directly leads to over-segmentation
rd
edition
The Use of Motion in Segmentation
! Motion is a powerful cue used by humans and
animals to extract objects or regions of interest from
a background of irrelevant detail
95
Spatial Techniques Basic approach
! One of the simplest approaches for detecting
changes between two image frames f(x,y,t
i
) and
f(x,y,t
j
) then at time t
i
and t
j
, respectively, is to
compare the two images pixel by pixel.
! A difference image between two image taken at
times t
i
and t
j
may be defined as
! This approach is applicable only if the two images
are registered spatially and if the illumination is
relatively constant within the bounds established by
T.
96
Accumulative differences
! Consider a sequence of image frames f(x,y,t1), f(x,y,t2),,
f(x,y,tn) and let f(x,y,t1) be the reference image.
! An accumulative difference image (ADI) is formed by
comparing this reference image with every subsequent image
in the sequence.
! A counter for each pixel location in the accumulative images is
incremented every time a difference occurs at that pixel
location between the reference and an image in the sequence.
! Often useful is consideration of three types of accumulative
difference images: absolute, positive, and negative ADIs.
97
Accumulative differences
! It is understood that these ADIs start out with all
zero values (counts).
! The direction and speed of the moving object can
be determined from
! The entries in the absolute and negative ADIs.
98
rd
edition
Example: Building a Difference
Image
rd
edition
100
E4830 DIGITAL IMAGE
PROCESSING
SPRING 2014
Representation and Description
Object Recognition
Lecture 11
April 24, 2014
Todays Lecture
! Representation
! Boundary Descriptors
! Regional Descriptors
! Use of Principal
Components for
Description
! Relational Descriptors
! Patterns and Pattern
Classes
! Recognition Based on
Decision-Theoretic
Methods
! Structural Methods
Representation and Description Object Recognition
2
Why representation and
description
Image
acquisition
Preprocessing Segmentation
Feature
Extraction
Classification
and
Interpretation
3
Representation and Description
! Representation
! Boundary (external characteristics)
! Whole region (internal characteristics)
! Describe region based on chosen representation
4
! Output is an ordered sequence of points. Required for
algorithms where the points in the boundary of a region needs
to be ordered in a clockwise (or counter clockwise) direction
Boundary (Border) Follower
5
! Can also be applied if you have a
region rather than a boundary.
Procedure extracts outer boundary of
a binary region. If need to find holds
in a region (inner boundary) "
extract holes then apply algorithm
Boundary Representation:
(Freeman) Chain Code
6
Original boundary Sub-sampled boundary Chain code of boundary
Chain code for 4-
neighborhood
Chain code for 8-
neighborhood ! Chain codes are used to
represent a boundary by a
connected sequence of
straight line segments of
specified length. Typically,
4- or 8-connectivity
Example: Chain Code
7
! Digital boundary approximated with arbitrary accuracy by a
polygon
! Goal of polygon approximation is to capture essence of shape
in given boundary with the fewest number of segments
! We will represent a boundary by the minimum-perimeter
polygon
Polygon Approximations using
Minimum-Perimeter Polygons
8
Minimum Perimeter Polygon
! Size of cell determines accuracy of polygon approximation
! Objective is to use the largest possible cell size acceptable in a
given application, thus producing MPPs with the fewest number
of vertices
9
! We can see that the vertices of
MPP coincide with either
convex vertices on inner wall
(white dots) or with concave
vertices (black dots) on outer
wall
E
x
a
m
p
l
e
:

M
P
P

A
l
g
o
r
i
t
h
m

! Merging/Splitting techniques based on average error

! Merging techniques merge points along a boundary until the
error exceeds a threshold. When this occurs, parameters for
line are restored, error set to 0 and procedure repeated
! Splitting subdivides segment successively into two parts until
specified criteria is satisfied
Polygon Approximation Approaches
Merging and Splitting Techniques
11
! Objective is to reduce the boundary representation to a 1-D
function that is easier to describe than the original 2D
boundary
! Try to normalize results so that relatively invariant to scale "
want to remove dependence on size while preserving
fundamental shape of waveform
Signatures
12
! Example signatures:
distance of centroid to
boundary as a function of
angle. Signature invariant
to translation but not to
scaling and rotation
E
x
a
m
p
l
e
:

S
i
g
n
a
t
u
r
e
s

! Another signature:
transverse boundary and
plot angle of line tangent
to boundary with respect
to reference line
r(!)
! Decomposition reduces the boundary's complexity and simplify
the description process
! Convex Hull, H, of an arbitrary set S is the smallest convex set
containing S
! Convex Deficiency. D=H-S
! Region boundary can be partitioned into meaningful segments
by following the contour of S and marking the points at which
transition is made into or out of a component of the convex
deficiency
Boundary Segments
14
! An important approach
is to represent structure
shape as a graph,
accomplished with a
skeleton via thinning
Skeletons
15
Boundary Descriptors
! Length: ~ # of pixels along a boundary
! Diameter:
! Major/Minor Axis
! Basic rectangle
! Eccentricity: ratio of major to minor axis
! Curvature: rate of change of slope
! Convex: in clockwise direction, vertex point p part of convex
segment if change in slope p is nonnegative. Otherwise
concave
! Corner: change exceeds 90
o
16
Shape Numbers
! Shape number of
boundary based on 4-
directional code
! The order n of a shape
number defined as number
of digits in representation
! N is even for a closed
boundary and value limits
number of possible
different shapes
17
Example: Shape number
18
! Boundary represented as sequence of
coordinates s(k) = [x(k), y(k)], for
k=0,1,2,.,K-1
! Each coordinate pair treated as a
complex number s(k)=x(k)+jy(k)
" can take DFT of s(k)
Fourier Descriptors
19
s(k) =
1
K
a u ( )
u=0
K!1
"
e
j 2#k/K
s(k) =
1
P
a u ( )
u=0
P!1
"
e
j 2#k/P
First p coefficients
Fourier Descriptors
20
! A few Fourier descriptors can be used to capture
the gross essence of a boundary
! Fourier descriptors are not directionally invariant,
but we can relationships
Fourier Descriptors
21
Boundary Description using
Statistical Moments
! Shape of boundary
segment (and of signature
functions) can be
described quantitatively
by using statistical
moments such as mean,
variance, and high order
moments
22
n
! ( ) = !
i
" m
( )
n
p !
i
( )
i=0
A"1
#
m = !
i
i=1
A"1
#
p !
i
( )
Region Descriptors - Simple
! Area
! Perimeter
! Compactness: (Perimeter)
2
/area
! Circularity Ratio, R
c
: ratio of region to area of circle
having the same perimeter. R
c
=4!A/P
2
.
Dimensionless " insensitive to uniform scale and
insensitive to orientation

! Mean/Median Intensity
! Max/Min Intensity
! Normalized area
23
! Topological properties: properties of image
preserved under rubber-sheet distortions
! H # holes in the image
! C # connected components
! Q # edges
! V # vertices
! F # faces
! Euler Number: E=C-H = V Q + F
Topological Region Descriptors
25
Example: Use of Connected Components
Extracting the Largest Features in
Segmented Image
27
Texture
No Formal Definition

Provides measure of properties such as
smoothness, coarseness, and regularity
Texture: Quantification Methods
! Statistical: compute local features at each point in image and
derive a set of statistics from the distribution of local features
! 1
st
, 2
nd
, and higher-order statistics based on how many
points are used to define local features
! Structural: texture is considered to be composed of texture
elements. Properties of the texture elements and their spatial
placement rules characterize the texture
! Original texture can be reconstructed from its structural
description
! Spectral: Based on properties of the Fourier Transform
28
Statistical Texture Analysis 1
st
order
statistics
! Image, f " histogram, h
f
! Statistical moments of the intensity histogram of
image or region
! Mean
! Variance
! Skewness
! Uniformity
! Entropy
29
! Texture measure for subimages
Example: Texture Measures Based
on Histograms
30
Statistical Texture Analysis: 2
nd

order statistics, Co-occurrence
! Texture measured from histogram contains no
information regarding relative position of pixels
with respect to each other
! Co-occurrence matrix
31
Statistical Texture Analysis: 2
nd

order statistics, Co-occurrence
Statistics
! Angular 2
nd
moment (energy)
! Measure of image homogeneity
! Maximum Probability
! Entropy
! Contrast
! Measure of local variations
! Correlation
! Measure of image linearity
32
D
e
s
c
r
i
p
t
o
r
s

f
o
r

C
h
a
r
a
c
t
e
r
i
z
i
n
g

C
o
-
o
c
c
u
r
r
e
n
c
e

m
a
t
r
i
x

Example:
Co-occurrence
Matrix
Co-occurrence Matrix: Correlation
! Correlation want to discover if there are any
sections of the image that contain repetitive
components (ie periodic texture)
35
Autocorrelation
! Large texture elements " autocorrelation
decreases slowly with increasing distance
! Small texture elements " autocorrelation decreases
rapidly with increasing distance
! Periodic texture elements " periodic increase and
decrease in autocorrelation value
36
C
ff
p, q ( ) =
MN
M ! p ( ) N ! q ( )
f k, l ( ) f k + p, l + q ( )
l=1
N!q
"
k=1
M!p
"
f
2
k, l ( )
l=1
N
"
k=1
M
"
! Basic idea: simple texture
primitive can be used to form
more complex texture patterns
by means of rules that limits
arrangements of primitive(s)
Structural Approaches
37
! Fourier spectrum ideally suited for describing directionality of
periodic or almost periodic 2D patterns in an image
! Features of Fourier Spectrum useful for texture description:
1. Prominent peaks in spectrum give principal direction of
texture pattern
2. Location of peals in frequency plane give fundamental
spatial period of pattern
3. Eliminating any periodic component using filtering leaves
nonperiodic image which can be described by statistical
techniques
Spectral Approaches
38
Spectral Approaches
39
! Detection and interpretation of
spectrum features simplified by
expressing spectrum in polar
coordinates
S r ( ) = S
!
r ( )
!=0
"
#
S ! ( ) = S
r
! ( )
r=1
R
o
#
Example: Spectral Texture
40
Example: Spectral Texture
41
Geometric Moment Invariants
! Moments that are invariant to translation, scale
change, mirroring (within a minus sign) and rotation
42
M
o
m
e
n
t

I
n
v
a
r
i
a
n
t
s

(
t
r
a
n
s
l
a
t
i
o
n
,

s
c
a
l
e
,

m
i
r
r
o
r
i
n
g
,

r
o
t
a
t
i
o
n
)

Use of Principal Components for
Description
! For sets of images that are spatially registered (i.e.
RGB image, multispectral image, etc)
44
Example: Using Principal
Components for Image Description
45
46
47
48
Using Principal Components for Normalizing
With Respect To Variation in Size,
Translation, and Rotation
Problem: Object recognition from
images
! Object or pattern recognition from image regions
! Approaches: decision theoretical or structural
! Central to the theme of recognition is the concept of
learning from sample patterns
! Pattern arrangement of descriptors
! Feature often used to denote descriptors
! Pattern class: family of patterns that share common
properties
50
Objects as Vectors
! Common pattern
arrangements used in practice
are vectors (for quantitative
descriptors) and strings and
trees (for structural
descriptors)
! x=[x
1
x
2
. x
n
]
T
. x
i
represents
the i
th
descriptor
! Can use technique called
discriminant analysis
51
Examples: Pattern Vector
Generation
52
A Tree description of an image
53
Pattern Classifier from Examples
! Goal: given x, infer y
! Learning from examples: supervised learning
! Given (xi, yi=f(xi)), i=1,,N for some unknown
function f
! Find a good approximation to f
! Rules versus data
! Encode human knowledge as rules
! e.g. the petal length and width of iris
! Appropriate scenarios for supervised learning
54
! Step 1: calculate class
prototypes as the
means
! Step 2: use the
protoypes to classify a
new example
! Discriminant function f
Minimum distance classifier
55
Nearest Neighbor classifier
! Steps:
! store all training examples
! classify a new example x by finding the training
example (xi, yi) thats nearest to x according to
Euclidean distance, and copying the labels
56
E
x
a
m
p
l
e
:

M
i
n
i
m
u
m

D
i
s
t
a
n
c
e

C
l
a
s
s
i
f
i
e
r

Matching by Correlation
! Template and correlation
referred to as template
matching
! Range [-1, 1]
! Correlation coefficient can
be computed using the
Fourier Transform
58
Example: Matching by Correlation
59
Optimum Statistical Classifier
! Probabilistic approach to recognition
! Bayes classifier for Gaussian pattern classes
60
B
a
y
e
s

C
l
a
s
s
i
f
i
c
a
t
i
o
n

o
f

M
u
l
t
i
s
p
e
c
t
r
a
l

I
m
a
g
e

D
a
t
a

Structural Methods
! Structural methods
seeks to achieve
pattern recognition by
capitalizing on
structural relationships
in patterns
! Recognition of
boundary shapes
based on string
representation
62
String Matching
! Measure of
similarity of
boundary coded
into strings a and
b through ration R
63
Research Example
4/24/14
64
Christine P. Hendon
Image
acquisition
Preprocessing Segmentation
Feature
Extraction
Classification
and
Interpretation
Intracoronary Imaging using Optical
Coherence Tomography
65
! Lipid rich plaque identified by signal poor region with diffuse boundaries
! Need additional contrast to reduce ambiguity when identifying lipid rich
plaques
Yabushita Circulation.106 (13) p1640.
(2002)
fibrocalcific aortic
plaque
lipid-rich carotid
plaque
Objective
Automated classification of plaques to
inform therapeutic decisions
Reduce ambiguity when identifying lipid rich
regions within OCT images through combine
morphological and spectroscopic analysis of
coronary artery plaques
CAD
z
Spectroscopic OCT
Time frequency analysis for wavelength dependent
optical properties
L
z
z
STFT
S(z,) S(z,)
S(,z
+z)
S(,z
)

t
(,z)
I(z)

z
L

Merged

Enviado por

Dados do documento

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Merged

Enviado por

Direitos autorais:

Formatos disponíveis

!"#$"!

B = w| w = !b, for b "B { }

Você também pode gostar