Escolar Documentos
Profissional Documentos
Cultura Documentos
ing
i
cess Vetterl Signal Processing
ro in Digital
al P
t
Sign nd Mar
l
ta
Module 8: Image Processing
Digi doni a 13
n
0
Pra
2
aolo
Module Overview:
Module 8.3:
Module 8.4:
Module 8.5:
ing
i
cess Vetterl
ro in
al P art
2D Fourier Analysis l Sign
dM
ta
Digi doni an 13
Image Filters
an 20
lo Pr
Image Compression
Pao
ing
i
cess Vetterl Signal Processing
ro in Digital
al P
t
Sign nd Mar
l
ta
Module 8.1: Image Processing
Digi doni a 13
n
0
Pra
2
aolo
Overview:
8.1
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
2D signal representations
l
ta
Digi doni a 13
Basic signals and operators ran
0
2
lo P
Pao
Images as multidimensional digital signals
Overview:
8.1
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
2D signal representations
l
ta
Digi doni a 13
Basic signals and operators ran
0
2
lo P
Pao
Images as multidimensional digital signals
Overview:
8.1
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
2D signal representations
l
ta
Digi doni a 13
Basic signals and operators ran
0
2
lo P
Pao
Images as multidimensional digital signals
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.1
Digital images
8.1
ing
i
cess Vetterl
ro in
P
t
indices locate a point on a grid pixel al
Sign nd Mar
l
ta
grid is usually regularly spaced
Digi doni a 13
Pran 20
values x[n1 , n2 ] refer to olo pixels appearance
the
Pa
Digital images
8.1
ing
i
cess Vetterl
ro in
P
t
indices locate a point on a grid pixel al
Sign nd Mar
l
ta
grid is usually regularly spaced
Digi doni a 13
Pran 20
values x[n1 , n2 ] refer to olo pixels appearance
the
Pa
Digital images
8.1
ing
i
cess Vetterl
ro in
P
t
indices locate a point on a grid pixel al
Sign nd Mar
l
ta
grid is usually regularly spaced
Digi doni a 13
Pran 20
values x[n1 , n2 ] refer to olo pixels appearance
the
Pa
Digital images
8.1
ing
i
cess Vetterl
ro in
P
t
indices locate a point on a grid pixel al
Sign nd Mar
l
ta
grid is usually regularly spaced
Digi doni a 13
Pran 20
values x[n1 , n2 ] refer to olo pixels appearance
the
Pa
color images: multidimensional pixel values in a color space (RGB, HSV, YUV, etc)
g
8.1
rli
ssin
roce in Vette
we can consider the single components separately:
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
Pao
color images: multidimensional pixel values in a color space (RGB, HSV, YUV, etc)
g
8.1
rli
ssin
roce in Vette
we can consider the single components separately:
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
Pao
color images: multidimensional pixel values in a color space (RGB, HSV, YUV, etc)
g
8.1
rli
ssin
roce in Vette
we can consider the single components separately:
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
Pao
=
+
+
Image processing
ing
i
cess Vetterl
ro in
something still works
al P
t
Sign nd Mar
l
ta
something breaks down
Digi doni a 13
an 20
something is new
lo Pr
Pao
8.1
Image processing
ing
i
cess Vetterl
ro in
something still works
al P
t
Sign nd Mar
l
ta
something breaks down
Digi doni a 13
an 20
something is new
lo Pr
Pao
8.1
Image processing
ing
i
cess Vetterl
ro in
something still works
al P
t
Sign nd Mar
l
ta
something breaks down
Digi doni a 13
an 20
something is new
lo Pr
Pao
8.1
Image processing
ing
i
cess Vetterl
ro in
linearity, convolution
al P
t
Sign nd Mar
l
ta
Fourier transform
Digi doni a 13
n
0
interpolation, sampling lo Pra
2
Pao
What works:
8.1
Image processing
ing
i
cess Vetterl
ro in
linearity, convolution
al P
t
Sign nd Mar
l
ta
Fourier transform
Digi doni a 13
n
0
interpolation, sampling lo Pra
2
Pao
What works:
8.1
Image processing
ing
i
cess Vetterl
ro in
linearity, convolution
al P
t
Sign nd Mar
l
ta
Fourier transform
Digi doni a 13
n
0
interpolation, sampling lo Pra
2
Pao
What works:
8.1
Image processing
ing
i
cess Vetterl
ro in
Fourier analysis less relevant
al P
t
Sign nd Mar
l
lter design hard, IIRs rare igita
D
ni a
ndo 2013
Pra
linear operators only mildly useful
aolo
P
8.1
Image processing
ing
i
cess Vetterl
ro in
Fourier analysis less relevant
al P
t
Sign nd Mar
l
lter design hard, IIRs rare igita
D
ni a
ndo 2013
Pra
linear operators only mildly useful
aolo
P
8.1
Image processing
ing
i
cess Vetterl
ro in
Fourier analysis less relevant
al P
t
Sign nd Mar
l
lter design hard, IIRs rare igita
D
ni a
ndo 2013
Pra
linear operators only mildly useful
aolo
P
8.1
Image processing
Whats new:
ing
i
cess Vetterl
ro in
lP
t
images are nite-support signals Signa
Mar
l
ta
a d
Digi d their n 13
images are (most often) available inoni entirety causality loses meaning
Pran 20
losignals, designed for a very specic processing system, i.e.
images are very specialized
Pao
new manipulations: ane transforms
the
8.1
Image processing
Whats new:
ing
i
cess Vetterl
ro in
lP
t
images are nite-support signals Signa
Mar
l
ta
a d
Digi d their n 13
images are (most often) available inoni entirety causality loses meaning
Pran 20
losignals, designed for a very specic processing system, i.e.
images are very specialized
Pao
new manipulations: ane transforms
the
8.1
Image processing
Whats new:
ing
i
cess Vetterl
ro in
lP
t
images are nite-support signals Signa
Mar
l
ta
a d
Digi d their n 13
images are (most often) available inoni entirety causality loses meaning
Pran 20
losignals, designed for a very specic processing system, i.e.
images are very specialized
Pao
new manipulations: ane transforms
the
8.1
Image processing
Whats new:
ing
i
cess Vetterl
ro in
lP
t
images are nite-support signals Signa
Mar
l
ta
a d
Digi d their n 13
images are (most often) available inoni entirety causality loses meaning
Pran 20
losignals, designed for a very specic processing system, i.e.
images are very specialized
Pao
new manipulations: ane transforms
the
8.1
ing
i
cess Vetterl
A two-dimensional discrete-space signal:
ro in
lP
t
gna
Sni2 ], nd 1Mar Z
l1 ,
x[n
n ,n
ta
Digi doni a 13 2
an 20
lo Pr
Pao
8.1
10
h[n1 , n2 ]
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
n1
8.1
n2
11
ing
i
cess Vetterl
ro in
lP
t
amplitude may be written along Signa
Mar
l
d
ta
explicitly
Digi doni an 13 0
an 20
example:
lo Pr
Pao
n2
[n1 , n2 ] =
1 if n1 = n2 = 0
0 otherwise.
5
5
n1
8.1
12
510
n2
425 ing
i
cess Vetterl
ro 340 in
al P
t
Sign nd Mar
l to
image values are quantized (usually
ta
Digi doni a 13255
8 bits, or 256 levels)
an
20
lo Pr in space
170
the eye does the interpolation
Pao
85
0
0
85
170
255
340
425
510
n1
8.1
13
Why 2D?
8.1
ing
i
cess Vetterl
ro in
images could be unrolled (printers, fax) al P
n
rt
l Sig nd Ma
ta
but what about spatial correlation? ni a
Digi do
13
Pran 20
lo
Pao
14
Why 2D?
8.1
ing
i
cess Vetterl
ro in
images could be unrolled (printers, fax) al P
n
rt
l Sig nd Ma
ta
but what about spatial correlation? ni a
Digi do
13
Pran 20
lo
Pao
14
2D vs raster scan
20
10
ing
i
cess Vetterl
ro in
10
al P
t
Sign nd Mar
l
20a
t
Digi 20doni a0 13 20
10
10
ran 20
olo P
0
Pa
0
0
8.1
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
10
ing
i
cess Vetterl
ro in
10
al P
t
Sign nd Mar
l
20a
t
Digi 20doni a0 13 20
10
10
ran 20
olo P
0
Pa
0
0
8.1
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
10
ing
i
cess Vetterl
ro in
10
al P
t
Sign nd Mar
l
20a
t
Digi 20doni a0 13 20
10
10
ran 20
olo P
0
Pa
0
0
8.1
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
10
ing
i
cess Vetterl
ro in
10
al P
t
Sign nd Mar
l
20a
t
Digi 20doni a0 13 20
10
10
ran 20
olo P
0
Pa
0
0
8.1
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
10
ing
i
cess Vetterl
ro in
10
al P
t
Sign nd Mar
l
20a
t
Digi 20doni a0 13 20
10
10
ran 20
olo P
0
Pa
0
0
8.1
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
10
ing
i
cess Vetterl
ro in
10
al P
t
Sign nd Mar
l
20a
t
Digi 20doni a0 13 20
10
10
ran 20
olo P
0
Pa
0
0
8.1
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
10
ing
i
cess Vetterl
ro in
10
al P
t
Sign nd Mar
l
20a
t
Digi 20doni a0 13 20
10
10
ran 20
olo P
0
Pa
0
0
8.1
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
10
ing
i
cess Vetterl
ro in
10
al P
t
Sign nd Mar
l
20a
t
Digi 20doni a0 13 20
10
10
ran 20
olo P
0
Pa
0
0
8.1
205
410
615
820
1025
1230
1435
1640
15
[n1 , n2 ] =
1
0
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
if n1 = n2 =al
0
t
0
ia
Digi
otherwise. ndon
013
Pra
2
aolo
P
5
5
8.1
16
rect
n1 n2
,
2N1 2N2
ing
i
cess Vetterl
ro in
P
t
1 if |n1 | < N1 ignal
Mar
and |n | talN
d
2
Digi < d2oni an 13 0
=
an 20
lo Pr
0 Pao
otherwise;
5
5
8.1
17
Separability
ing
i
cess Vetterl
ro in
lP
t
i 2 a x d 1 ]x2 [n
Sng]n= n1[nMar 2]
x[n1 ,
l
ta
Digi doni a 13
an 20
lo Pr
o
Pa
8.1
18
Separable signals
rli
s
roce in Vette
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an
20 n
n2
o
1
ln1 P,r n2 =
rect o
rect
.
rect
Pa 2N1 2N2
2N1
2N2
8.1
19
Separable signals
rli
s
roce in Vette
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an
20 n
n2
o
1
ln1 P,r n2 =
rect o
rect
.
rect
Pa 2N1 2N2
2N1
2N2
8.1
19
Nonseparable signal
x[n1 , n2 ] =
1
0
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
if |n1 | + |n2 |ital
<N
Dig doni a 13 0
otherwise
an 20
lo Pr
Pao
5
5
8.1
20
Nonseparable signal
x[n1 , n2 ] = rect
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
ln n
ta
n1 n2
Digi 1 , a2 13 0
,
rect doni
2N1 2N2
2M 2M2
an 1 20
lo Pr
Pao
5
5
8.1
21
2D convolution
ing
i
cess Vetterl
ro in
P
al art
x[n1 , n2 ] h[n1 , n2 ] = l Sign
x[k , k ]h[n k , n k ]
nd M 1 2 1 1 2 2
igitak =i ka= 3
D
on
rand 201
lo P
Pao
1
8.1
22
ing
i
cess Vetterl
ro
tin
nal P
Sigh1[nnd M]ar
x[n1 , n2 ] h[n1 , n2 ] =ital
k1
x[k1 , k2 ]h2 [n2 k2 ]
1
Dig k =ni a 13 k =
do
Pran ] 20] x[n , n ]).
lo
1 2
Pao = h1 [n1 (h2[n2
1
8.1
23
ing
i
cess Vetterl
ro in
al P
non-separable convolution: M1 M2 operations perart
Sign nd M output sample
tal i a
Digi 2 operations per3
separable convolution: M1 + M ndon
01 output sample!
Pra
2
lo
Pao
8.1
24
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
ing
i
cess Vetterl Signal Processing
ro in Digital
al P
t
Sign nd Mar
l
ta
Module 8.2: Image Manipulations
Digi doni a 13
n
0
Pra
2
aolo
Overview:
Ane transforms
Bilinear interpolation
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.2
25
Overview:
Ane transforms
Bilinear interpolation
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.2
25
Ane transforms
mapping R2 R2 that reshapes the coordinate system:
t1
a11 a12 t1 ssing
d1
i
ce dVetterl
= a
t2
al22 rot2
21
2
P
na
rtin
l Sig nd Ma
ta
Digi doni a 13
an 20
lo Pr
ao
t1
t1
d
=A t
t2
2
8.2
26
Ane transforms
mapping R2 R2 that reshapes the coordinate system:
t1
a11 a12 t1 ssing
d1
i
ce dVetterl
= a
t2
al22 rot2
21
2
P
na
rtin
l Sig nd Ma
ta
Digi doni a 13
an 20
lo Pr
ao
t1
t1
d
=A t
t2
2
8.2
26
Translation
A=
1 0
0 1
d=
d1
d2
ing
i
cess Vetterl
ro in
al P
t
=I
Sign nd Mar
l
ta
Digi doni a 13 0
an 20
lo Pr
ao
8.2
27
Translation
A=
1 0
0 1
d=
d1
d2
ing
i
cess Vetterl
ro in
al P
t
=I
Sign nd Mar
l
ta
Digi doni a 13 0
an 20
lo Pr
ao
8.2
27
Scaling
ing
i
cess Vetterl
a1 0
ro in
A=
al P
t
0 a2
Sign nd Mar
l
ta
Digi doni a 13 0
d=0
an
20
o Pr
lpreserved
if a1 = a2 the aspect ratio is
Pao
0
8.2
28
Scaling
ing
i
cess Vetterl
a1 0
ro in
A=
al P
t
0 a2
Sign nd Mar
l
ta
Digi doni a 13 0
d=0
an
20
o Pr
lpreserved
if a1 = a2 the aspect ratio is
Pao
0
8.2
28
Rotation
A=
d=0
cos
sin
ing
i
cess Vetterl
ro in
al P
t
sin
Sign nd Mar
l
ta
0
cos igi
D
ni a
ndo 2013
Pra
aolo
P
0
8.2
29
Rotation
A=
d=0
cos
sin
ing
i
cess Vetterl
ro in
al P
t
sin
Sign nd Mar
l
ta
0
cos igi
D
ni a
ndo 2013
Pra
aolo
P
0
8.2
29
Rotation
A=
d=0
cos
sin
ing
i
cess Vetterl
ro in
al P
t
sin
Sign nd Mar
l
ta
0
cos igi
D
ni a
ndo 2013
Pra
aolo
P
0
8.2
29
Flips
Horizontal:
A=
d=0
Vertical:
A=
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13 0
an 20
lo Pr
ao
1 0
0 1
1 P0
0 1
d=0
0
8.2
30
Flips
Horizontal:
A=
d=0
Vertical:
A=
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13 0
an 20
lo Pr
ao
1 0
0 1
1 P0
0 1
d=0
0
8.2
30
Shear
Horizontal:
A=
d=0
Vertical:
A=
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13 0
an 20
lo Pr
ao
1 s
0 1
1 P0
s 1
d=0
0
8.2
31
Shear
Horizontal:
A=
d=0
Vertical:
A=
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13 0
an 20
lo Pr
ao
1 s
0 1
1 P0
s 1
d=0
0
8.2
31
ing
i
cess Vetterl
ro in
al P
t
t1
S nn n d r
= lA ig 1 d MaR2 = Z2
ta
tg
n
Di2 i don2i a 13
an 20
lo Pr
ao
8.2
32
8.2
i
t1
m +sdng
li
= A1 roc1es 1 ; etter
t2
l P m2 + dn V
i2
na
rt
l Sig to nd Ma
interpolate from the originaligitapoint a the mid-point
grid
D
ni
ndo 2013
(t1 , t2 ) =lo 1 + 1 , 2 + 2 ),
( Pra
1,2 Z,
ao
P
0 1,2 < 1
33
8.2
i
t1
m +sdng
li
= A1 roc1es 1 ; etter
t2
l P m2 + dn V
i2
na
rt
l Sig to nd Ma
interpolate from the originaligitapoint a the mid-point
grid
D
ni
ndo 2013
(t1 , t2 ) =lo 1 + 1 , 2 + 2 ),
( Pra
1,2 Z,
ao
P
0 1,2 < 1
33
Bilinear Interpolation
x[1 + 1, 2 + 1]
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an
20
lo Pr x[ ,
]
ao
2 + 1
1
8.2
1 + 1
34
Bilinear Interpolation
x[1 + 1, 2 + 1]
ing
i
cess Vetterl
(t , t )ro
al P
tin
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
o
2 + 1
Pa
x[1 , 2 ]
1
8.2
1 + 1
34
Bilinear Interpolation
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi donia 13
an 20
lo Pr
ao
2 + 1
1
8.2
1 + 1
34
Bilinear Interpolation
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
2 + 1
1
8.2
1 + 1
34
Bilinear Interpolation
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi donia 13
an 20
lo Pr
ao
2 + 1
1
8.2
1 + 1
34
Bilinear Interpolation
ing
i
cess Vetterl
ro in
al P
t
y [m1 , m2 ] = (1 1 )(1 ig2 )x[1 , 2 ] ar1 (1 2 )x[1 + 1, 2 ]
S n nd M +
l
igita on a
+ (1 D 1 )2 x[1 , i 2 + 1] + 1 2 x[1 + 1, 2 + 1]
d
13
Pran 20
lo
Pao
8.2
35
Shearing
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.2
36
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
ing
i
cess Vetterl Signal Processing
ro in Digital
al P
t
Sign nd Mar
l
ta
Module 8.3: Frequency Analysis
Digi doni a 13
n
0
Pra
2
aolo
Overview:
8.3
ing
i
cess Vetterl
ro in
DFT
al P
t
Sign nd Mar
l
ta
Magnitude and phase
Digi doni a 13
an 20
lo Pr
Pao
37
Overview:
8.3
ing
i
cess Vetterl
ro in
DFT
al P
t
Sign nd Mar
l
ta
Magnitude and phase
Digi doni a 13
an 20
lo Pr
Pao
37
2D DFT
g j rni k
l
c
ro in Vette
n =0 n =0 l P
na
rt
l Sig nd Ma
ta
Digi doni a 13
an N 20
1
lo Pr N 1 1 X [k , k ]e j n k e j n k
x[n1Pn2o=
, a]
1 2
N1 N2
N1 1 N2 1
X [k1 , k2 ] =
x[n1 , n2 ]e ssin1 1 e
e N1
j 2 n k
2
N2
2 2
2
N1
1 1
2
N2
2 2
k1 =0 k2 =0
8.3
38
2D DFT
g j rni k
l
c
ro in Vette
n =0 n =0 l P
na
rt
l Sig nd Ma
ta
Digi doni a 13
an N 20
1
lo Pr N 1 1 X [k , k ]e j n k e j n k
x[n1Pn2o=
, a]
1 2
N1 N2
N1 1 N2 1
X [k1 , k2 ] =
x[n1 , n2 ]e ssin1 1 e
e N1
j 2 n k
2
N2
2 2
2
N1
1 1
2
N2
2 2
k1 =0 k2 =0
8.3
38
ing
i
cess Vetterl
ro in
al P n k t n k
j
j
wk ,k [n1Sign= e Mar
, n2 ]
e
ital ni and
Dig do
13
Pran 20
olo
2
N1
1 1
2
N2
2 2
P
for n1 , k1 = 0, 1, . . . , N1 1 and n2 , k2 = 0, 1, . . . , N2 1
8.3
39
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
40
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
41
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
42
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
43
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
44
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
45
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
46
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
47
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
48
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
49
2D DFT
g
i
s in
cejs n Vetterln k
o
rn2 ]e in k e j
X [k1 , k2 ] =
x[nP,
al 1
t
Sign nd Mar
n al n =0
=0
t
Digi doni a 13
an 20
1D-DFT along n2 (the columns)
lo Pr
Pao
N1 1 N2 1
8.3
2
N1
1 1
2
N2
2 2
50
2D DFT
ing e i
cess n Vettjrl n k
o
X [k1 , k2 ] =
x[nPr 2 ]e
l , n j k e
gna 1 Martin
n =0 l Si=0
n
igita oni and 3
D
d
1
1D-DFT along n2 (the columns)
Pran 20
lo
Pao
N1 1
8.3
N2 1
2
N2
2 2
2
N1
1 1
50
2D DFT
sin
li
o ]e s k t er
rn2cej inn Vete j
x[nP,
X [k1 , k2 ] =
al 1
t
n =0
n =0
Sign nd Mar
l
ta
Digi doni a 13
an 20
1D-DFT along n2 (the columns)
lo Pr
Pao
N2 1
N1 1
8.3
2
N2
2 2
2
n k
N1 1 1
50
2D DFT
g r
ssin k ttejli
ocej n
X [k1 , k2 ] =
x[nP,rn2 ]e in Ve e
al 1 art
n =0
n =0
Sign
ital ni and M
ig
D
do
13
Pran 20
1D-DFT along n2 (the columns)
lo
Pao
N1 1
8.3
N2 1
2
N2
2 2
2
n k
N1 1 1
50
8.3
essi
terli
et
recall also the N N DFT matrix (ModuleProc
al 4.2): artin V
n
l Sig 1 nd M 1
1 igita
1
1
i a2 3 3 . . .
N1
1 D W 1ndonW 01 W
. . . WN
a
N2
lo Pr 2
2(N1)
6
W4
WN
. . . WN
WN = ao
1 WN
...
2
2(N1)
3(N1)
(N1)
N1
1 WN
WN
WN
. . . WN
51
8.3
essi
terli
et
recall also the N N DFT matrix (ModuleProc
al 4.2): artin V
n
l Sig 1 nd M 1
1 igita
1
1
i a2 3 3 . . .
N1
1 D W 1ndonW 01 W
. . . WN
a
N2
lo Pr 2
2(N1)
6
W4
WN
. . . WN
WN = ao
1 WN
...
2
2(N1)
3(N1)
(N1)
N1
1 WN
WN
WN
. . . WN
51
8.3
essi
terli
et
recall also the N N DFT matrix (ModuleProc
al 4.2): artin V
n
l Sig 1 nd M 1
1 igita
1
1
i a2 3 3 . . .
N1
1 D W 1ndonW 01 W
. . . WN
a
N2
lo Pr 2
2(N1)
6
W4
WN
. . . WN
WN = ao
1 WN
...
2
2(N1)
3(N1)
(N1)
N1
1 WN
WN
WN
. . . WN
51
N1 1
X [k1 , k2 ] =
N2 1
x[n1 , n2 ]e
2
n k
N2 2 2
e j N1 n1 k1
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
n1 =0
n2 =0
8.3
52
N1 1
X [k1 , k2 ] =
N2 1
x[n1 , n2 ]e
2
n k
N2 2 2
e j N1 n1 k1
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
a
Digi
V = WN x doni
n
013
Pra
2
lo
V ao N N
P R
n1 =0
n2 =0
8.3
52
N1 1
X [k1 , k2 ] =
N2 1
x[n1 , n2 ]e
2
n k
N2 2 2
e j N1 n1 k1
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
a
Digi
V = WN x doni
3 X = V WN
n
201
ra
lo P
V ao N N
X RN N
P R
n1 =0
n2 =0
8.3
52
N1 1
X [k1 , k2 ] =
N2 1
x[n1 , n2 ]e
2
n k
N2 2 2
e j N1 n1 k1
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
a
Digi
V = WN x doni
3 X = V WN
n
201
ra
lo P
V ao N N
X RN N
P R
n1 =0
n2 =0
X = WN2 x WN1
8.3
52
8.3
ing
i
cess Vetterl
ro in
P range
t
problem: the range is too big for theignal
S grayscaleMar of paper or screen
l
ita n and |X
try to normalize: |X [n1 , nDig |X [no,ni2 ]|/ max3 [n1 , n2 ]|
2 ]| =
d1
1
Pran 20
lo
but it doesnt work... ao
P
try to show the magnitude as an image
53
107
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
105
103
101
0
8.3
5122
54
sing
li
lP
tin V
remove agrant outliers (e.g. X [0, 0]igna
1
S = nd x[na,rn2])
tal 1/3 a M
Digiy =doni after normalization (x 1)
use a nonlinear mapping: e.g.
x
13
Pran 20
lo
Pao
s
if the image is high dynamic range we need to compress the levels r
ette
roce
8.3
55
sing
li
lP
tin V
remove agrant outliers (e.g. X [0, 0]igna
1
S = nd x[na,rn2])
tal 1/3 a M
Digiy =doni after normalization (x 1)
use a nonlinear mapping: e.g.
x
13
Pran 20
lo
Pao
s
if the image is high dynamic range we need to compress the levels r
ette
roce
8.3
55
510
425
425
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
255
ta
Digi doni a 13
an 20 170
lo Pr
ao
340
340
255
170
85
85
0
0
8.3
85
170
255
340
425
510
85
170
255
340
425
510
56
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.3
57
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.3
58
8.3
rli
roce in Vette
P
edges are points of abrupt change inignal values t
S signals Mar
ital and
edges are a space-domain ig
D feature oninot captured by DFTs magnitude
13
ra d
Pfor n 20edges
lo
phase alignment is important
reproducing
Pao
59
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
ing
i
cess Vetterl Signal Processing
ro in Digital
al P
t
Sign nd Mar
l
ta
Module 8.4: Filtering
Digi doni a 13
n
0
Pra
2
aolo
Overview:
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
Classication
Examples
8.4
60
Overview:
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
Classication
Examples
8.4
60
Overview:
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
Classication
Examples
8.4
60
space invariance
impulse response
frequency response
stability
8.4
linearity
2D CCDE
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
61
n
space-invariant lters process everything in the same iway
ess
g
rli
c
ro in Vette
P
t
but we should process things dierently al
Sign nd Mar
l
ta
edges
Digi doni a 13
an 20
gradients
lo Pr
Pao
textures
...
8.4
62
n
space-invariant lters process everything in the same iway
ess
g
rli
c
ro in Vette
P
t
but we should process things dierently al
Sign nd Mar
l
ta
edges
Digi doni a 13
an 20
gradients
lo Pr
Pao
textures
...
8.4
62
n
space-invariant lters process everything in the same iway
ess
g
rli
c
ro in Vette
P
t
but we should process things dierently al
Sign nd Mar
l
ta
edges
Digi doni a 13
an 20
gradients
lo Pr
Pao
textures
...
8.4
62
n
space-invariant lters process everything in the same iway
ess
g
rli
c
ro in Vette
P
t
but we should process things dierently al
Sign nd Mar
l
ta
edges
Digi doni a 13
an 20
gradients
lo Pr
Pao
textures
...
8.4
62
n
space-invariant lters process everything in the same iway
ess
g
rli
c
ro in Vette
P
t
but we should process things dierently al
Sign nd Mar
l
ta
edges
Digi doni a 13
an 20
gradients
lo Pr
Pao
textures
...
8.4
62
n
space-invariant lters process everything in the same iway
ess
g
rli
c
ro in Vette
P
t
but we should process things dierently al
Sign nd Mar
l
ta
edges
Digi doni a 13
an 20
gradients
lo Pr
Pao
textures
...
8.4
62
n
space-invariant lters process everything in the same iway
ess
g
rli
c
ro in Vette
P
t
but we should process things dierently al
Sign nd Mar
l
ta
edges
Digi doni a 13
an 20
gradients
lo Pr
Pao
textures
...
8.4
62
Filter types
8.4
ing
i
cess Vetterl
ro in
causal or noncausal
al P
t
Sign nd Mar
l
ta
highpass, lowpass, ...
Digi doni a 13
lowpass image smoothing an
0
2
lo Pr
highpass enhancement, edge detection
Pao
IIR, FIR
63
Filter types
8.4
ing
i
cess Vetterl
ro in
causal or noncausal
al P
t
Sign nd Mar
l
ta
highpass, lowpass, ...
Digi doni a 13
lowpass image smoothing an
0
2
lo Pr
highpass enhancement, edge detection
Pao
IIR, FIR
63
Filter types
8.4
ing
i
cess Vetterl
ro in
causal or noncausal
al P
t
Sign nd Mar
l
ta
highpass, lowpass, ...
Digi doni a 13
lowpass image smoothing an
0
2
lo Pr
highpass enhancement, edge detection
Pao
IIR, FIR
63
Filter types
8.4
ing
i
cess Vetterl
ro in
causal or noncausal
al P
t
Sign nd Mar
l
ta
highpass, lowpass, ...
Digi doni a 13
lowpass image smoothing an
0
2
lo Pr
highpass enhancement, edge detection
Pao
IIR, FIR
63
Filter types
8.4
ing
i
cess Vetterl
ro in
causal or noncausal
al P
t
Sign nd Mar
l
ta
highpass, lowpass, ...
Digi doni a 13
lowpass image smoothing an
0
2
lo Pr
highpass enhancement, edge detection
Pao
IIR, FIR
63
8.4
ing
i
cess Vetterl
ro in
al P
t
border eects
Sign nd Mar
l
ita algebra doesnt hold in multiple dimensions!
stability: the fundamental Dig
theorem of ni a
do
13
Pran 20
lo
computability
Pao
nonlinear phase (edges!)
64
8.4
ing
i
cess Vetterl
ro in
al P
t
border eects
Sign nd Mar
l
ita algebra doesnt hold in multiple dimensions!
stability: the fundamental Dig
theorem of ni a
do
13
Pran 20
lo
computability
Pao
nonlinear phase (edges!)
64
8.4
ing
i
cess Vetterl
ro in
al P
t
border eects
Sign nd Mar
l
ita algebra doesnt hold in multiple dimensions!
stability: the fundamental Dig
theorem of ni a
do
13
Pran 20
lo
computability
Pao
nonlinear phase (edges!)
64
8.4
ing
i
cess Vetterl
ro in
al P
t
border eects
Sign nd Mar
l
ita algebra doesnt hold in multiple dimensions!
stability: the fundamental Dig
theorem of ni a
do
13
Pran 20
lo
computability
Pao
nonlinear phase (edges!)
64
A noncomputable CCDE
y [n1 , n2 ] = a0 y [n1 + 1, n2 ] + a1 y [n1 , n2 1] + a2 y [n1 1, n2 ] + a3 y [n1 , n2 + 1] + x[n1 , n2 ];
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
2
y[0, 1]
y[0, 0]
y[1, 0]
y[1, 0]
y[0, 1]
1
2
8.4
2
65
A noncomputable CCDE
y [n1 , n2 ] = a0 y [n1 + 1, n2 ] + a1 y [n1 , n2 1] + a2 y [n1 1, n2 ] + a3 y [n1 , n2 + 1] + x[n1 , n2 ];
ing
i
cess Vetterl
ro in
al P
t
1
Sign nd Mar
l
ta
Digi doni a 13
an 20
0
lo Pr
ao
2
8.4
2
65
8.4
rli
ro in Vette
al P
t
per-sample complexity:
Sign nd Mar
lresponses
ta
M1 M2 for nonseparable impulse
Digi doni a 13
an 0
M1 + M2 for separable impulse responses 2
lo Pr
Pao
66
8.4
rli
ro in Vette
al P
t
per-sample complexity:
Sign nd Mar
lresponses
ta
M1 M2 for nonseparable impulse
Digi doni a 13
an 0
M1 + M2 for separable impulse responses 2
lo Pr
Pao
66
8.4
rli
ro in Vette
al P
t
per-sample complexity:
Sign nd Mar
lresponses
ta
M1 M2 for nonseparable impulse
Digi doni a 13
an 0
M1 + M2 for separable impulse responses 2
lo Pr
Pao
66
8.4
rli
ro in Vette
al P
t
per-sample complexity:
Sign nd Mar
lresponses
ta
M1 M2 for nonseparable impulse
Digi doni a 13
an 0
M1 + M2 for separable impulse responses 2
lo Pr
Pao
66
8.4
rli
ro in Vette
al P
t
per-sample complexity:
Sign nd Mar
lresponses
ta
M1 M2 for nonseparable impulse
Digi doni a 13
an 0
M1 + M2 for separable impulse responses 2
lo Pr
Pao
66
Moving Average
sing n i
cesx[n1 Vek1t,er2l k2]
ro in t
k =N k =N
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
o
n n
1
y [n1 , n2 ] =
1
(2N + 1)2
Pah[n1, n2 ] =
8.4
(2N +
1)2
rect
2N 2N
67
Moving Average
sing n i
cesx[n1 Vek1t,er2l k2]
ro in t
k =N k =N
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
o
1
n n
y [n1 , n2 ] =
1
(2N + 1)2
Pah[n1, n2 ] =
8.4
(2N +
1)2
rect
2N 2N
67
Moving Average
sing
ter
o es
Pr1 c 1 t1n Vet
l
na1
ri
h[n1 , Sig = d 1 a 1
l n2 ] 9 M 1
ta
n
Digi
oni a 1 31 1
d
an
lo Pr
ao
8.4
li
01
68
Moving Average
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
11 11 MA
8.4
51 51 MA
69
Gaussian Blur
ing
i
cess Vetterl
1
r,o |nn, n | < N
h[n1 , n2 ] =
e al P
ti 1 2
2gn
Si2 nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
Pao
with N 3
n2 +n2
1 22
2
8.4
70
Gaussian Blur
h[n1 , n2 ]
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
n1
8.4
n2
71
Gaussian Blur
12
8
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
0
l
ta
Digi doni a 13
4
an 20
lo Pr
Pao
8
4
12
12 8
8.4
4
0
4
= 5, N = 14
12
72
Gaussian Blur
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
= 1.8, 11 11 blur
8.4
= 8.7, 51 51 blur
73
Sobel lter
approximation of the rst derivative in the horizontal direction:
1 0 1
i g
i
so [n1 , n2 ] = 2 ess2n
c 0 1 Vetterl
ro 0 in
1
al P
n
rt
l Sig nd Ma
ta
Digi doni a 13
an 20
lo Pr
ao
8.4
1
so [n1 , n2 ] = 2 1 0 1
1
74
Sobel lter
approximation of the rst derivative in the horizontal direction:
1 0 1
i g
i
so [n1 , n2 ] = 2 ess2n
c 0 1 Vetterl
ro 0 in
1
al P
n
rt
l Sig nd Ma
ta
Digi doni a 13
an 20
lo Pr
ao
8.4
1
so [n1 , n2 ] = 2 1 0 1
1
74
Sobel lter
lo
Pao
8.4
Pr
75
Sobel lter
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
Sobel operator
rli
s
roce in Vette
|x[n1 , n2 ]|2 = |so [n1 , n2 ] gnal1 P 2 ]|2art v [n1 , n2 ] x[n1 , n2 ]|2
x[n , n
+ |s
M
l Si
d
ta
Digi doni an 13
(operator because its nonlinear)
an 20
lo Pr
Pao
8.4
77
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
Sobel operator
8.4
Laplacian operator
ing
i
cess Vetterl
ro in
al P f a t
S,ign =n 2M+ r 2 f
f (t1 t2 ) d 2
tal
2
Digi doni a t13 t2
n
01
ra
2
olo P
Pa
8.4
79
Laplacian operator
approximating the Laplacian; start with a Taylor expansion
f (t + ) =
f (n) (t) n
g
n! ssin
e
rli
roc in Vette
al P
t
Sign nd Mar
l
ita (t
and compute the expansion in Di+ ) andni a ): 3
(t g
do
1
Pran 20
lo
Pao(t + ) = f (t) + f (t) + 1 f (t) 2
f
n=0
2
1
f (t ) = f (t) f (t) + f (t) 2
2
8.4
80
Laplacian operator
by rearranging terms:
dM
tal
Digi doni an 13
an 20
which, on the discrete grid, is lo Pr h[n] = 1 2
the FIR
Pao
8.4
ssin
rli
t
1
oce + f V + te
f (t) = 2 (f (t a) r2f (t) tin (t e ))
l P
Sign
ar
1
81
Laplacian
lo
Pao
8.4
Pr
82
Laplacian
ing
i
cess Vetterl
ro
al P 1 i
g = 1 8 t1n
Si2 ]n 1 Mar 1
h[n1 , n
d
tal
Digi doni an1 131 1
an 20
lo Pr
o
Pa
8.4
83
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
Laplacian operator
8.4
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
ing
i
cess Vetterl Signal Processing
ro in Digital
al P
t
Sign nd Mar
l
ta
Module 8.5: Image Compression
Digi doni a 13
n
0
Pra
2
aolo
Overview:
8.5
ing
i
cess Vetterl
ro in
Redundancy in natural images
al P
t
Sign nd Mar
l
Image coding ingredients Digita
ni a
ndo 2013
Pra
aolo
P
85
Overview:
8.5
ing
i
cess Vetterl
ro in
Redundancy in natural images
al P
t
Sign nd Mar
l
Image coding ingredients Digita
ni a
ndo 2013
Pra
aolo
P
85
A thought experiment
8.5
ing
i
cess Vetterl
ro in
each image is 524,288 bits
al P
t
Sign nd Mar
l
ta
Digi 2doni a 10157,826
total number of possible images: n 524,288 2013
Pra
aolo
P
number of atoms in the universe: 1082
86
A thought experiment
8.5
ing
i
cess Vetterl
ro in
each image is 524,288 bits
al P
t
Sign nd Mar
l
ta
Digi 2doni a 10157,826
total number of possible images: n 524,288 2013
Pra
aolo
P
number of atoms in the universe: 1082
86
A thought experiment
8.5
ing
i
cess Vetterl
ro in
each image is 524,288 bits
al P
t
Sign nd Mar
l
ta
Digi 2doni a 10157,826
total number of possible images: n 524,288 2013
Pra
aolo
P
number of atoms in the universe: 1082
86
A thought experiment
8.5
ing
i
cess Vetterl
ro in
each image is 524,288 bits
al P
t
Sign nd Mar
l
ta
Digi 2doni a 10157,826
total number of possible images: n 524,288 2013
Pra
aolo
P
number of atoms in the universe: 1082
86
take all images in the world and list them in an encyclopedia of images
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
a
on the Internet: M = 50 billiont
Digi doni a 13
an
20
raw encoding: 524,288 bits per image
lo Pr
Pao
to indicate an image, simply give its number
8.5
87
take all images in the world and list them in an encyclopedia of images
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
a
on the Internet: M = 50 billiont
Digi doni a 13
an
20
raw encoding: 524,288 bits per image
lo Pr
Pao
to indicate an image, simply give its number
8.5
87
take all images in the world and list them in an encyclopedia of images
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
a
on the Internet: M = 50 billiont
Digi doni a 13
an
20
raw encoding: 524,288 bits per image
lo Pr
Pao
to indicate an image, simply give its number
8.5
87
take all images in the world and list them in an encyclopedia of images
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
a
on the Internet: M = 50 billiont
Digi doni a 13
an
20
raw encoding: 524,288 bits per image
lo Pr
Pao
to indicate an image, simply give its number
8.5
87
take all images in the world and list them in an encyclopedia of images
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
a
on the Internet: M = 50 billiont
Digi doni a 13
an
20
raw encoding: 524,288 bits per image
lo Pr
Pao
to indicate an image, simply give its number
8.5
87
take all images in the world and list them in an encyclopedia of images
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
a
on the Internet: M = 50 billiont
Digi doni a 13
an
20
raw encoding: 524,288 bits per image
lo Pr
Pao
to indicate an image, simply give its number
8.5
87
Compression
ing
i
cess Vetterl
ro in
nal P M rt
allocate bits for things that matter Sig edges) a
(e.g.
ital ni d
Digto ndooutan 1matters
use psychovisual experiments and
what 3
0
Pr
2
lo
Pao
some information is discarded: lossy compression
Another approach:
8.5
88
Compression
ing
i
cess Vetterl
ro in
nal P M rt
allocate bits for things that matter Sig edges) a
(e.g.
ital ni d
Digto ndooutan 1matters
use psychovisual experiments and
what 3
0
Pr
2
lo
Pao
some information is discarded: lossy compression
Another approach:
8.5
88
Compression
ing
i
cess Vetterl
ro in
nal P M rt
allocate bits for things that matter Sig edges) a
(e.g.
ital ni d
Digto ndooutan 1matters
use psychovisual experiments and
what 3
0
Pr
2
lo
Pao
some information is discarded: lossy compression
Another approach:
8.5
88
Compression
ing
i
cess Vetterl
ro in
nal P M rt
allocate bits for things that matter Sig edges) a
(e.g.
ital ni d
Digto ndooutan 1matters
use psychovisual experiments and
what 3
0
Pr
2
lo
Pao
some information is discarded: lossy compression
Another approach:
8.5
88
Key ingredients
8.5
ing
i
cess Vetterl
ro in
al P
using a suitable transform (i.e., a change of basis) rt
Sign nd Ma
tal
smart quantization
Digi doni a 13
an 20
lo Pr
entropy coding
Pao
compressing at block level
89
Key ingredients
8.5
ing
i
cess Vetterl
ro in
al P
using a suitable transform (i.e., a change of basis) rt
Sign nd Ma
tal
smart quantization
Digi doni a 13
an 20
lo Pr
entropy coding
Pao
compressing at block level
89
Key ingredients
8.5
ing
i
cess Vetterl
ro in
al P
using a suitable transform (i.e., a change of basis) rt
Sign nd Ma
tal
smart quantization
Digi doni a 13
an 20
lo Pr
entropy coding
Pao
compressing at block level
89
Key ingredients
8.5
ing
i
cess Vetterl
ro in
al P
using a suitable transform (i.e., a change of basis) rt
Sign nd Ma
tal
smart quantization
Digi doni a 13
an 20
lo Pr
entropy coding
Pao
compressing at block level
89
8.5
ing
i
cess Vetterl
reduce number bits per pixel
ro in
al P
t
Sign nd Mar
l
equivalent to coarser quantization
ta
Digi doni a 13
in the limit, 1bpp
an 20
lo Pr
Pao
90
8.5
ing
i
cess Vetterl
reduce number bits per pixel
ro in
al P
t
Sign nd Mar
l
equivalent to coarser quantization
ta
Digi doni a 13
in the limit, 1bpp
an 20
lo Pr
Pao
90
8.5
ing
i
cess Vetterl
ro in
al P
reduce number bits per pixel
t
Sign nd Mar
l
ta
equivalent to coarser quantization oni a
Digi d
13
Pran 20
lo
in the limit, 1bpp
Pao
90
8.5
ing
i
cess Vetterl
ro in
al P art
code the average value with 8 bits l Sign
ita n d M
Diggivesdlessi an 13
o
3 3 blocks at 8 bits per block
Pran 20
than 1bpp
lo
Pao
divide the image in blocks
91
8.5
ing
i
cess Vetterl
ro in
al P art
code the average value with 8 bits l Sign
ita n d M
Diggivesdlessi an 13
o
3 3 blocks at 8 bits per block
Pran 20
than 1bpp
lo
Pao
divide the image in blocks
91
8.5
ing
i
cess Vetterl
ro in
divide the image in blocks
al P
t
Sign nd Mar
l
t
code the average value with 8 ibitsa
D gi doni a 13
an
3 3 blocks at 8 bits per block gives less 20
lo Pr
than 1bpp
Pao
91
8.5
ing
i
cess Vetterl
ro in
exploit the local spatial correlation
al P
t
Sign nd Mar
l
ta
compress remote regions independentlyi a
Digi don
13
Pran 20
lo
Pao
92
8.5
ing
i
cess Vetterl
ro in
exploit the local spatial correlation
al P
t
Sign nd Mar
l
ta
compress remote regions independentlyi a
Digi don
13
Pran 20
lo
Pao
92
Transform coding
A simple example:
8.5
ing
i
cess Vetterl
ro in
lP
t
storing the signal requires NR bitsSigna
Mar
l
ta
i d
Digi d like an 13
now you take the DFT and it lookson
an 20
this
lo Pr
Pao
10
15
20
25
30
93
Transform coding
A simple example:
8.5
ing
i
cess Vetterl
ro in
lP
t
storing the signal requires NR bitsSigna
Mar
l
ta
i d
Digi d like an 13
now you take the DFT and it lookson
an 20
this
lo Pr
Pao
10
15
20
25
30
93
Transform coding
A simple example:
8.5
ing
i
cess Vetterl
ro in
lP
t
storing the signal requires NR bitsSigna
Mar
l
ta
i d
Digi d like an 13
now you take the DFT and it lookson
an 20
this
lo Pr
Pao
10
15
20
25
30
93
Transform coding
A simple example:
8.5
ing
i
cess Vetterl
ro in
lP
t
storing the signal requires NR bitsSigna
Mar
l
ta
i d
Digi d like an 13
now you take the DFT and it lookson
an 20
this
lo Pr
Pao
10
15
20
25
30
93
Transform coding
i
sing
cesin a Vetterl
captures the important features of an image ro
block
few coecients
al P
tin
Sign nd Mar
l
is ecient to compute
ta
Digi doni a 13
an 20
lo Pr
answer: the Discrete Pao Transform
Cosine
8.5
94
Transform coding
i
sing
cesin a Vetterl
captures the important features of an image ro
block
few coecients
al P
tin
Sign nd Mar
l
is ecient to compute
ta
Digi doni a 13
an 20
lo Pr
answer: the Discrete Pao Transform
Cosine
8.5
94
Transform coding
i
sing
cesin a Vetterl
captures the important features of an image ro
block
few coecients
al P
tin
Sign nd Mar
l
is ecient to compute
ta
Digi doni a 13
an 20
lo Pr
answer: the Discrete Pao Transform
Cosine
8.5
94
2D-DCT
ing
i
cess Vetterl
ro in
N1 N1
al P
rt
Sign nd 1 +a1 k1 cos
C [k1 , k2 ] =
x[n1 , n2l] cos
nM
ita ni N
2
N
a
n =0 n =0Dig
do 2013
an
lo Pr
Pao
1
8.5
n2 +
1
2
k2
95
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.5
96
Smart quantization
8.5
ing
i
cess Vetterl
ro in
deadzone
al P
t
Sign nd Mar
l
variable step (ne to coarse) gita
Di
ni a
ndo 2013
Pra
aolo
P
97
Smart quantization
8.5
ing
i
cess Vetterl
ro in
deadzone
al P
t
Sign nd Mar
l
variable step (ne to coarse) gita
Di
ni a
ndo 2013
Pra
aolo
P
97
Quantization
x [n]
ing
i
cess Ve1terl
ro in t 00
Standard quantization:
al P
t
Sign nd Mar
l
ita ni a
x = oor(x) + 0.5 Dig
do
013 1 10
Pran 22
lo
Pao
01
x[n]
1
1
11
2
8.5
98
Quantization
x [n]
Deadzone quantization:
ing
i
cess Ve1terl
ro in t
al P
t
Sign nd Mar
l
ta
00
Digi doni a 13
0
an 22
1
lo Pr
ao
x = round(x)
01
x[n]
1
10
8.5
98
Entropy coding
8.5
ng
Ve
l Pro
gna andMartin
associate short symbols to frequentSvalues
vice-versa
l i
igita oni and 3
D
d
01
Pr n
if it sounds familiar its becauseait is... 2
lo
Pao
99
Entropy coding
8.5
ng
Ve
l Pro
gna andMartin
associate short symbols to frequentSvalues
vice-versa
l i
igita oni and 3
D
d
01
Pr n
if it sounds familiar its becauseait is... 2
lo
Pao
99
Entropy coding
8.5
ng
Ve
l Pro
gna andMartin
associate short symbols to frequentSvalues
vice-versa
l i
igita oni and 3
D
d
01
Pr n
if it sounds familiar its becauseait is... 2
lo
Pao
99
Entropy coding
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.5
100
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
ing
i
cess Vetterl Signal Processing
ro in Digital
al P
t
Sign nd Mar
l
ta
Digi doni a Module 8.6: The JPEG Compression Algorithm
13
Pran 20
aolo
Key ingredients
8.6
ing
i
cess Vetterl
ro in
al P
using a suitable transform (i.e., a change of basis) rt
Sign nd Ma
tal
smart quantization
Digi doni a 13
an 20
lo Pr
entropy coding
Pao
compressing at block level
101
Key ingredients
8.6
ing
i
cess Vetterl
ro in
al P
using a suitable transform (i.e., a change of basis) rt
Sign nd Ma
tal
smart quantization
Digi doni a 13
an 20
lo Pr
entropy coding
Pao
101
Key ingredients
8.6
ing
i
cess Vetterl
ro in
al P
t
compute the DCT of each block
Sign nd Mar
l
ta
smart quantization
Digi doni a 13
an 20
lo Pr
entropy coding
Pao
101
Key ingredients
8.6
ing
i
cess Vetterl
ro in
al P
t
compute the DCT of each block
Sign nd Mar
l
ita to a
quantize DCT coecients Dig
accordingonipsycovisually-tuned tables
d
13
Pran 20
lo
entropy coding
Pao
101
Key ingredients
8.6
ing
i
cess Vetterl
ro in
al P
t
compute the DCT of each block
Sign nd Mar
l
ita to a
quantize DCT coecients Dig
accordingonipsycovisually-tuned tables
d
13
Pran coding 20
lo
run-length encoding andoHuman
Pa
101
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.6
102
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.6
102
Smart quantization
8.6
rli
ss
roce in Vette
some coecients are more important thanl others t
a P
Sign nd Mar
al
nd out the critical coecientstby experimentation
Digi doni a 13
an 20
allocate more bits (or, equivalently, ner quantization levels) to the most important
lo Pr
Pao
coecients
103
Smart quantization
8.6
rli
ss
roce in Vette
some coecients are more important thanl others t
a P
Sign nd Mar
al
nd out the critical coecientstby experimentation
Digi doni a 13
an 20
allocate more bits (or, equivalently, ner quantization levels) to the most important
lo Pr
Pao
coecients
103
Smart quantization
8.6
rli
ss
roce in Vette
some coecients are more important thanl others t
a P
Sign nd Mar
al
nd out the critical coecientstby experimentation
Digi doni a 13
an 20
allocate more bits (or, equivalently, ner quantization levels) to the most important
lo Pr
Pao
coecients
103
Smart quantization
8.6
rli
ss
roce in Vette
some coecients are more important thanl others t
a P
Sign nd Mar
al
nd out the critical coecientstby experimentation
Digi doni a 13
an 20
allocate more bits (or, equivalently, ner quantization levels) to the most important
lo Pr
Pao
coecients
103
g
16 11 10 16 24 essin 51 terli
c 40 Vet 61
12 12 14 19 l Pro 58n 60 55
a 26 ti
14 13 l 16gn
Si 24 nd40 ar57 69 56
M
gita
14 i 17 22 ni a 51 87 80 62
D
do 29 2013 109 103 77
Q=
ran
18 P22 37 68
lo 35 55 56 81 104 113 92
o
a24
64
P
8.6
104
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
uniform
8.6
tuned
105
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
uniform
8.6
tuned
106
Ecient coding
8.6
sing terli
et
al P artin V
Si n
use zigzag scan to maximize orderingg
ital ni and M
Dig
quantization will create long seriesdo zeros2013
an of
lo Pr
Pao
most coecients are small, decreasing with indexes
roc
107
Ecient coding
8.6
sing terli
et
al P artin V
Si n
use zigzag scan to maximize orderingg
ital ni and M
Dig
quantization will create long seriesdo zeros2013
an of
lo Pr
Pao
most coecients are small, decreasing with indexes
roc
107
Ecient coding
8.6
sing terli
et
al P artin V
Si n
use zigzag scan to maximize orderingg
ital ni and M
Dig
quantization will create long seriesdo zeros2013
an of
lo Pr
Pao
most coecients are small, decreasing with indexes
roc
107
Zigzag scan
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.6
108
Example
100 60 0 6 0 0 0 0
0
0 0 0 0 0 0 0
0
0 0 0 0 essi0 g0 erli
0 n
t
0
0 0 l 0 ro 0 i0 Vet
P c n
a 0 0 a0rt 0 0
13 1gn
0
0
ital Si 0 nd M
ig0
0 ni0a 0 0 0 0 0
D
0 ando 0 2013 0 0 0
r 0 0 0
lo P
0 0 0 0 0 0 0
Pao 0
8.6
109
Example
100 60 0 6 0 0 0 0
0
0 0 0 0 0 0 0
0
0 0 0 0 essi0 g0 erli
0 n
t
0
0 0 l 0 ro 0 i0 Vet
P c n
a 0 0 a0rt 0 0
13 1gn
0
0
ital Si 0 nd M
ig0
0 ni0a 0 0 0 0 0
D
0 ando 0 2013 0 0 0
r 0 0 0
lo P
0 0 0 0 0 0 0
Pao 0
8.6
109
Runlength encoding
ing
i
cess Vetterl
ro in
nal P M rt
r is the runlength i.e. the numberl ofig
S zerosnbeforeathe current value
d
ta
Digi bits oni a to1encode the value
d needed20 3
s is the size i.e. the number of an
Pr
aolo
c is the actual value P
[(r , s), c]
8.6
(0, 0) indicates that from now on its only zeros (end of block)
110
Runlength encoding
ing
i
cess Vetterl
ro in
nal P M rt
r is the runlength i.e. the numberl ofig
S zerosnbeforeathe current value
d
ta
Digi bits oni a to1encode the value
d needed20 3
s is the size i.e. the number of an
Pr
aolo
c is the actual value P
[(r , s), c]
8.6
(0, 0) indicates that from now on its only zeros (end of block)
110
Runlength encoding
ing
i
cess Vetterl
ro in
nal P M rt
r is the runlength i.e. the numberl ofig
S zerosnbeforeathe current value
d
ta
Digi bits oni a to1encode the value
d needed20 3
s is the size i.e. the number of an
Pr
aolo
c is the actual value P
[(r , s), c]
8.6
(0, 0) indicates that from now on its only zeros (end of block)
110
Runlength encoding
ing
i
cess Vetterl
ro in
nal P M rt
r is the runlength i.e. the numberl ofig
S zerosnbeforeathe current value
d
ta
Digi bits oni a to1encode the value
d needed20 3
s is the size i.e. the number of an
Pr
aolo
c is the actual value P
[(r , s), c]
8.6
(0, 0) indicates that from now on its only zeros (end of block)
110
Example
100 60 0 6 0 0 0 0
0
0 0 0 0 0 0 0
0
0 0 0 0 0 0 g0
in i
0
0 0 0 ocess 0 0 terl
r 0 0 in Vet
P
13 1 0 l 0 0 0 t 0 0
gna Mar
i
0tal S0 0 0d 0 0 0 0
n
igi
D
0 doni0a 0 03 0 0 0
0
ran 0 201 0 0 0
0 0 0
lo P0
Pao
[(0, 7), 100], [(0, 6), 60], [(4, 3), 6], [(3, 4), 13], [(8, 1), 1], [(0, 0)]
8.6
111
Example
100 60 0 6 0 0 0 0
0
0 0 0 0 0 0 0
0
0 0 0 0 0 0 g0
in i
0
0 0 0 ocess 0 0 terl
r 0 0 in Vet
P
13 1 0 l 0 0 0 t 0 0
gna Mar
i
0tal S0 0 0d 0 0 0 0
n
igi
D
0 doni0a 0 03 0 0 0
0
ran 0 201 0 0 0
0 0 0
lo P0
Pao
[(0, 7), 100], [(0, 6), 60], [(4, 3), 6], [(3, 4), 13], [(8, 1), 1], [(0, 0)]
8.6
111
8.6
ing
i
cess Vetterl
ro in
al P
t
in theory, 8 bits per pair
Sign nd Mar
l
ita than others!
some pairs are much moreDig
common oni a
and 13
Prbeing smart20
lo
a lot of space can be saved by
Pao
by design, (r , s) A with |A| = 256
112
8.6
ing
i
cess Vetterl
ro in
al P
t
in theory, 8 bits per pair
Sign nd Mar
l
ita than others!
some pairs are much moreDig
common oni a
and 13
Prbeing smart20
lo
a lot of space can be saved by
Pao
by design, (r , s) A with |A| = 256
112
8.6
ing
i
cess Vetterl
ro in
al P
t
in theory, 8 bits per pair
Sign nd Mar
l
ita than others!
some pairs are much moreDig
common oni a
and 13
Prbeing smart20
lo
a lot of space can be saved by
Pao
by design, (r , s) A with |A| = 256
112
8.6
ing
i
cess Vetterl
ro in
al P
t
in theory, 8 bits per pair
Sign nd Mar
l
ita than others!
some pairs are much moreDig
common oni a
and 13
Prbeing smart20
lo
a lot of space can be saved by
Pao
by design, (r , s) A with |A| = 256
112
Variable-length encoding
great idea: shorter binary sequences for common symbols
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.6
113
Variable-length encoding
8.6
tter
oces
gn
ar
al Si i a d M
itlettersnandnwords (wasteful)
in Morse code, pauses separate
Dig do
13
Pran 20
can we do away with separators?
lo
Pao
e
in English, spaces separate words extral symbol (wasteful)
a Pr
tin V
114
Variable-length encoding
8.6
tter
oces
gn
ar
al Si i a d M
itlettersnandnwords (wasteful)
in Morse code, pauses separate
Dig do
13
Pran 20
can we do away with separators?
lo
Pao
e
in English, spaces separate words extral symbol (wasteful)
a Pr
tin V
114
Variable-length encoding
8.6
tter
oces
gn
ar
al Si i a d M
itlettersnandnwords (wasteful)
in Morse code, pauses separate
Dig do
13
Pran 20
can we do away with separators?
lo
Pao
e
in English, spaces separate words extral symbol (wasteful)
a Pr
tin V
114
Prex-free codes
8.6
sing
al P
tin V
Signnond Mar
can parse a bitstream sequentiallyl with
look-ahead
ta
Digi doni a 13
extremely easy to understand graphically... 20
an
lo Pr
Pao
li
r
no valid sequence can be the beginning of anothersvalid sequence
ette
roce
115
Prex-free codes
8.6
sing
al P
tin V
Signnond Mar
can parse a bitstream sequentiallyl with
look-ahead
ta
Digi doni a 13
extremely easy to understand graphically... 20
an
lo Pr
Pao
li
r
no valid sequence can be the beginning of anothersvalid sequence
ette
roce
115
Prex-free codes
8.6
sing
al P
tin V
Signnond Mar
can parse a bitstream sequentiallyl with
look-ahead
ta
Digi doni a 13
extremely easy to understand graphically... 20
an
lo Pr
Pao
li
r
no valid sequence can be the beginning of anothersvalid sequence
ette
roce
115
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
A
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
AA
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
AAB
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
AABA
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
AABAA
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
AABAAB
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
AABAABA
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
AABAABAD
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
AABAABADC
8.6
116
Entropy coding
ing
i
cess Vetterl
assign short sequences to more frequent symbols
ro in
al P
t
Sign nd Mara set of symbol probabilities
the Human algorithm builds the loptimal code for
ta
Digi doni a 13
in JPEG, you can use a general-purpose 20
an Human code or build your own
lo Pr
(but then you pay a side-information price)
Pao
8.6
117
Entropy coding
ing
i
cess Vetterl
assign short sequences to more frequent symbols
ro in
al P
t
Sign nd Mara set of symbol probabilities
the Human algorithm builds the loptimal code for
ta
Digi doni a 13
in JPEG, you can use a general-purpose 20
an Human code or build your own
lo Pr
(but then you pay a side-information price)
Pao
8.6
117
Entropy coding
ing
i
cess Vetterl
assign short sequences to more frequent symbols
ro in
al P
t
Sign nd Mara set of symbol probabilities
the Human algorithm builds the loptimal code for
ta
Digi doni a 13
in JPEG, you can use a general-purpose 20
an Human code or build your own
lo Pr
(but then you pay a side-information price)
Pao
8.6
117
Example
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l 0.38
igita
Dp(A) =doni a 13 p(B) = 0.32
20
Pran
p(D) = 0.2
lop(C ) = 0.1
o
four symbols: A, B, C, D
probability table:
Pa
8.6
118
Example
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l 0.38
igita
Dp(A) =doni a 13 p(B) = 0.32
20
Pran
p(D) = 0.2
lop(C ) = 0.1
o
four symbols: A, B, C, D
probability table:
Pa
8.6
118
C
0.10 sing
p(B) = 0.32
p(C ) = 0.1
rli
s
roce in Vette
P
t
0.30gnal
Si
Mar
l
d
ta
Digi doni an 0.20 D
1
13
Pran 20
aolo
p(A) = 0.38
8.6
p(D) = 0.2
119
ng
i
e i e
c1 ss VDtterl
ro in
al P
0.62
t
Sign nd Mar
l
ta
Digi doni a 1B
1 0.32 3
ran 20
P
aolo
0
0.30
p(A) = 0.38
8.6
p(B) = 0.32
p(C + D) = 0.3
120
Human Coding
0.38 A
ing
i
cess Vetterl
ro in0
al P art
1.00
C
1 l Sign
n
ita ni a0 d M
Dig do
13
D
1
Pran 0.62 20
lo
Pao
0
p(A) = 0.38
8.6
p(B + C + D) = 0.62
121
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
END OF MODULE 8
P