Quantization: Input-Output Characteristic of A Scalar Quantizer

Quantization
Input-output characteristic of a scalar quantizer
x
Output
Sometimes, this
convention is used:
xq 2
M representative levels
xq 1
xq
tq
-1
t q+1
t q+2
Q x
input signal x
M-1 decision
thresholds
Bernd Girod: EE398A Image and Video Compression
Quantization no. 1
Example of a quantized waveform

Original and Quantized Signal
Quantization Error
Quantization no. 2
Lloyd-Max scalar quantizer
Problem : For a signal x with given PDF f X ( x) find a quantizer with

M representative levels such that
d MSE E X X
min.
Solution : Lloyd-Max quantizer

[Lloyd,1957] [Max,1960]
M-1 decision thresholds exactly

half-way between representative
levels.
1
tq xq 1 xq q 1, 2,, M -1
2
M representative levels in the
centroid of the PDF between two xq

successive decision thresholds.
Necessary (but not sufficient)
conditions
tq1
x f
( x)dx
tq
tq 1
q 0,1, , M -1
f X ( x)dx
tq
Quantization no. 3
Iterative Lloyd-Max quantizer design

1.
2.
Guess initial set of representative levels xq q 0,1, 2,, M -1

Calculate decision thresholds
1
tq xq 1 xq q 1, 2,, M -1
2
3.
Calculate new representative levels

tq1
xq
x f
( x)dx
tq
tq1
q 0,1,, M -1
f X ( x)dx
tq
4.
Repeat 2. and 3. until no further distortion reduction

Quantization no. 4
Example of use of the Lloyd algorithm (I)
X zero-mean, unit-variance Gaussian r.v.

Design scalar quantizer with 4 quantization indices with
minimum expected distortion D*
Optimum quantizer, obtained with the Lloyd algorithm
Decision thresholds -0.98, 0, 0.98

Representative levels 1.51, -0.45, 0.45, 1.51
D*=0.12=9.30 dB
Boundary
Reconstruction
Quantization no. 5
Example of use of the Lloyd algorithm (II)

Convergence
Quantization Function
Initial quantizer B:
decision thresholds , 0,
Initial quantizer A:
decision thresholds 3, 0 3
6
4
2
0
-2
-4
-6
10
15
6
4
2
0
-2
-4
-6
0.2
0.2
0.15
0.4
10
10
SNROUT final = 9.2978
10
0.1
15
5
0
15
0
10
10
15
SNROUT10final = 9.298
15
8
6
Iteration Number
Iteration Number
SNROUT [dB]
Iteration Number
SNROUT [dB]
10
15
Iteration Number
After 6 iterations, in both cases, (D-D*)/D*<1%
Quantization no. 6
Example of use of the Lloyd algorithm (III)
X zero-mean, unit-variance Laplacian r.v.

Design scalar quantizer with 4 quantization indices with
minimum expected distortion D*
Optimum quantizer, obtained with the Lloyd algorithm
Decision thresholds -1.13, 0, 1.13

Representative levels -1.83, -0.42, 0.42, 1.83
D*=0.18=7.54 dB
Threshold
Representative
Quantization no. 7
Example of use of the Lloyd algorithm (IV)

Convergence
Initial quantizer A,
decision thresholds 3, 0 3
10
5
0
-5
-10
Initial quantizer B,
decision thresholds , 0,
10
Iteration Number
12
14
10
5
0
-5
-10
0.2
0.2
6
4
10
15
SNRfinal = 7.5415
0
10
Iteration Number
10
12
14
16
Iteration Number
0.4
0
0
8
16
0.4
SNR [dB]
15
SNR [dB]
0
0
8
6
4
10
15
SNRfinal = 7.5415
0
Iteration Number10
15
After 6 iterations, in both cases, (D-D*)/D*<1%
Quantization no. 8
Lloyd algorithm with training data

xq ; q 0,1, 2,, M -1
1.
Guess initial set of representative levels
2.
Assign each sample xi in training set T to closest representative xq
Bq x T : Q x q
3.
Calculate new representative levels
xq
4.
q 0,1,2,, M -1
1
Bq
q 0,1,, M -1
xBq
Repeat 2. and 3. until no further distortion reduction
Quantization no. 9
Lloyd-Max quantizer properties
Zero-mean quantization error
E X X 0
Quantization error and reconstruction decorrelated
E X X X 0
Variance subtraction property
E X X
2
X
2
X
Quantization no. 10
High rate approximation
Approximate solution of the "Max quantization problem,"

assuming high rate and smooth PDF [Panter, Dite, 1951]
x( x) const
1
3
f X ( x)
Distance between two

successive quantizer
representative levels
Probability density
function of x
Approximation for the quantization error variance:
d E X X
1 3 f ( x)dx
12M 2 X
x
Number of representative levels

Quantization no. 11
High rate approximation (cont.)
High-rate distortion-rate function for scalar Lloyd-Max quantizer
d R 2 X2 22 R
1 3
with f X ( x)dx
12 x
2
X
Some example values for
uniform
Laplacian
Gaussian
9
2
1
4.5
3
2.721
2
Quantization no. 12
High rate approximation (cont.)
Partial distortion theorem: each interval makes an (approximately)

equal contribution to overall mean-squared error
Pr ti X ti 1 E X X ti X ti 1
Pr t j X t j 1 E X X t j X t j 1
for all i, j
[Panter, Dite, 1951], [Fejes Toth, 1959], [Gersho, 1979]
Quantization no. 13
Entropy-constrained scalar quantizer
Lloyd-Max quantizer optimum for fixed-rate encoding, how can we

do better for variable-length encoding of quantizer index?
Problem : For a signal x with given pdf

rate
M 1
f X ( x) find a quantizer with
R H X pq log 2 pq
such that
q 0
d MSE E X X
min.
2
Solution: Lagrangian cost function
J d R E X X
H X min.
2
Quantization no. 14
Iterative entropy-constrained scalar quantizer design

1.
2.
Guess initial set of representative levels xq ; q 0,1, 2,, M -1

and corresponding probabilities pq
Calculate M-1 decision thresholds
tq =
3.
xq -1 xq
2
log 2 pq 1 log 2 pq
2 xq -1 xq
q 1, 2,, M -1
Calculate M new representative levels and probabilities pq

tq1
xq
xf
( x)dx
tq
tq1
q 0,1,, M -1
f X ( x)dx
tq
4.
Repeat 2. and 3. until no further reduction in Lagrangian cost
Quantization no. 15
Lloyd algorithm for entropy-constrained

quantizer design based on training set
1.
2.
Guess initial set of representative levels xq ; q 0,1, 2,, M -1

and corresponding probabilities pq
Assign each sample xi in training set T to representative

2
J
q
x
minimizing Lagrangian cost x i q log 2 pq
xq
Bq x T : Q x q
3.
Calculate new representative levels and probabilities pq
xq
4.
q 0,1, 2,, M -1
1
Bq
q 0,1,, M -1
xBq
Repeat 2. and 3. until no further reduction in overall Lagrangian

2
cost
J J
x Q x
log p
xi
xi
q xi
xi
Quantization no. 16
Example of the EC Lloyd algorithm (I)
X zero-mean, unit-variance Gaussian r.v.

Design entropy-constrained scalar quantizer with rate R2 bits, and
minimum distortion D*
Optimum quantizer, obtained with the entropy-constrained Lloyd
algorithm
11 intervals (in [-6,6]), almost uniform

D*=0.09=10.53 dB, R=2.0035 bits (compare to fixed-length example)
Threshold
Representative level
Quantization no. 17
Example of the EC Lloyd algorithm (II)

Same Lagrangian multiplier used in all experiments
R [bit/symbol] SNR [dB]
4
2
0
4
2
0
-4
-4
-6
-6
10
15
20
25
Iteration Number
30
35
40
SNRfinal = 10.5363
8
6
5
10
15
20
25
30
35
40
Rfinal = 2.0044
1.5
0
-2
-2
10
Initial quantizer B, only 4 intervals

(<11) in [-6,6], with the same length
12
0
2.5
Initial quantizer A, 15 intervals

(>11) in [-6,6], with the same length
10
15
20
25
30
35
40
10
15
20
Iteration Number
12
SNRfinal = 8.9452
10
8
6
0
2.5
10
15
20
Rfinal = 1.7723
2
1.5
1
Iteration Number
10
15
20
Iteration Number
Quantization no. 18
Example of the EC Lloyd algorithm (III)
X zero-mean, unit-variance Laplacian r.v.

Design entropy-constrained scalar quantizer with rate R2 bits
and minimum distortion D*
Optimum quantizer, obtained with the entropy-constrained Lloyd
algorithm
21 intervals (in [-10,10]), almost uniform

D*= 0.07=11.38 dB, R= 2.0023 bits (compare to fixed-length example)
Decision threshold
Representative level
Quantization no. 19
Example of the EC Lloyd algorithm (IV)

Same Lagrangian multiplier used in all experiments
Initial quantizer A, 25 intervals
(>21 & odd) in [-10,10], with the
same length
10
5
0
-5
-10
10
15
20
25
30
35
Initial quantizer B, only 4 intervals

(<21) in [-10,10], with the same length
40
10
5
0
-5
-10
10
10
SNRfinal = 11.407
10
15
20
25
30
35
40
Rfinal = 2.0063
2
1
10
15
20
25
30
35
40
15
20
15
10
SNRfinal = 7.4111
5
0
10
15
20
Rfinal = 1.6186
2
1
Iteration Number
15
Iteration Number
Iteration Number
10
Iteration Number
15
20
Convergence in cost faster than convergence of decision thresholds
Quantization no. 20
High-rate results for EC scalar quantizers
For MSE distortion and high rates, uniform quantizers (followed by

entropy coding) are optimum [Gish, Pierce, 1968]
Distortion and entropy for smooth PDF and fine quantizer interval
2
d
12
2
H X h X log 2
Distortion-rate function
1 2 h X 2 R
d R 2
2
12
e
is factor
or 1.53 dB from Shannon Lower Bound
6
1 2 h X 2 R
D R
2
2
2 e
Quantization no. 21
Comparison of high-rate performance of

scalar quantizers
High-rate distortion-rate function
d R 2 X2 22 R
Scaling factor
Shannon LowBd
Laplacian
6
0.703
e
e
0.865
Gaussian
Uniform
Lloyd-Max
Entropy-coded
9
4.5
2
3
2.721
2
e2
1.232
6
e
1.423
6
Quantization no. 22
Deadzone uniform quantizer
Quantizer
output x
Quantizer
input x
Quantization no. 23
Embedded quantizers
Motivation: scalability decoding of compressed

bitstreams at different rates (with different qualities)
Nested quantization intervals
Q0
Q1
Q2
In general, only one quantizer can be optimum

(exception: uniform quantizers)
Quantization no. 24
Example: Lloyd-Max quantizers for Gaussian PDF
2-bit and 3-bit optimal

quantizers not embeddable
Performance loss for
embedded quantizers
-.98
.98
0 0 0 01 1 1 1
0 0 1 10 0 1 1
0 1 0 10 1 0 1
-1.75 -1.05 -.50
.50 1.05 1.75
Quantization no. 25
Information theoretic analysis
Successive refinement Embedded coding at multiple

rates w/o loss relative to R-D function
E d ( X , X 1 ) D1
I ( X ; X 1 ) R( D1 )
I ( X ; X ) R( D )
E d ( X , X 2 ) D2
Successive refinement with distortions D1 and D2 D1

can be achieved iff there exists a conditional distribution
f X , X
1
( x1 , x2 , x) f X
( x2 , x) f X
X 2
( x1 , x2 )
Markov chain condition
X X 2 X 1
[Equitz,Cover, 1991]
Quantization no. 26
Embedded deadzone uniform quantizers

x=0
8
Q0
Q1
Q2
Supported in JPEG-2000 with general for

quantization of wavelet coefficients.
Quantization no. 27
Vector quantization
Quantization no. 28
LBG algorithm
Lloyd algorithm generalized for VQ [Linde, Buzo, Gray, 1980]
Best representative
vectors
for given partitioning
of training set
Best partitioning
of training set
for given
representative
vectors
Assumption: fixed code word length

Code book unstructured: full search
Quantization no. 29
Design of entropy-coded vector quantizers
Extended LBG algorithm for entropy-coded VQ

[Chou, Lookabaugh, Gray, 1989]
Lagrangian cost function: solve unconstrained problem rather than

constrained problem
J d R E X X H X min.
Unstructured code book: full search for
J xi q xi xq log 2 pq
2
The most general coder structure:

Any source coder can be interpreted as VQ with VLC!
Quantization no. 30
Lattice vector quantization
Amplitude 2
cell
pdf
representative
vector
Amplitude 1
Quantization no. 31
8D VQ of a Gauss-Markov source
SNR [dB]
18
LBG fixed
CWL
12
E8-lattice
LBG variable
CWL
6
scalar
r 0.95
0
0
0.5
1.0
Rate [bit/sample]
Quantization no. 32
8D VQ of memoryless Laplacian source

9
E8-lattice
SNR [dB]
LBG variable
CWL
scalar
3
LBG fixed
CWL
0.5
1.0
Rate [bit/sample]
Quantization no. 33
Reading
Taubman, Marcellin, Sections 3.2, 3.4

J. Max, Quantizing for Minimum Distortion, IEEE Trans.
Information Theory, vol. 6, no. 1, pp. 7-12, March 1960.
S. P. Lloyd, Least Squares Quantization in PCM, IEEE
Trans. Information Theory, vol. 28, no. 2, pp. 129-137,
March 1982.
P. A. Chou, T. Lookabaugh, R. M. Gray, Entropyconstrained vector quantization, IEEE Trans. Signal
Processing, vol. 37, no. 1, pp. 31-42, January 1989.
Quantization no. 34

Quantization: Input-Output Characteristic of A Scalar Quantizer

Enviado por

Dados do documento

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Quantization: Input-Output Characteristic of A Scalar Quantizer

Enviado por

Direitos autorais:

Formatos disponíveis

Quantization

Input-output characteristic of a scalar quantizer

Example of a quantized waveform

Bernd Girod: EE398A Image and Video Compression

Lloyd-Max scalar quantizer

Problem : For a signal x with given PDF f X ( x) find a quantizer with

Solution : Lloyd-Max quantizer

M-1 decision thresholds exactly

M representative levels in the

centroid of the PDF between two xq

Bernd Girod: EE398A Image and Video Compression

Iterative Lloyd-Max quantizer design

Guess initial set of representative levels xq q 0,1, 2,, M -1

Calculate new representative levels

Repeat 2. and 3. until no further distortion reduction

Example of use of the Lloyd algorithm (I)

X zero-mean, unit-variance Gaussian r.v.

Decision thresholds -0.98, 0, 0.98

Bernd Girod: EE398A Image and Video Compression

Example of use of the Lloyd algorithm (II)

SNROUT final = 9.2978

After 6 iterations, in both cases, (D-D*)/D*<1%

Bernd Girod: EE398A Image and Video Compression

Example of use of the Lloyd algorithm (III)

X zero-mean, unit-variance Laplacian r.v.

Decision thresholds -1.13, 0, 1.13

Bernd Girod: EE398A Image and Video Compression

Example of use of the Lloyd algorithm (IV)

After 6 iterations, in both cases, (D-D*)/D*<1%

Bernd Girod: EE398A Image and Video Compression

Lloyd algorithm with training data

Guess initial set of representative levels

Assign each sample xi in training set T to closest representative xq

Calculate new representative levels

Repeat 2. and 3. until no further distortion reduction

Bernd Girod: EE398A Image and Video Compression

Lloyd-Max quantizer properties

Zero-mean quantization error

Quantization error and reconstruction decorrelated

Variance subtraction property

Bernd Girod: EE398A Image and Video Compression

High rate approximation

Approximate solution of the "Max quantization problem,"

Distance between two

Approximation for the quantization error variance:

Number of representative levels

High rate approximation (cont.)

High-rate distortion-rate function for scalar Lloyd-Max quantizer

Some example values for

Bernd Girod: EE398A Image and Video Compression

High rate approximation (cont.)

Partial distortion theorem: each interval makes an (approximately)

[Panter, Dite, 1951], [Fejes Toth, 1959], [Gersho, 1979]

Bernd Girod: EE398A Image and Video Compression

Entropy-constrained scalar quantizer

Lloyd-Max quantizer optimum for fixed-rate encoding, how can we

Problem : For a signal x with given pdf

f X ( x) find a quantizer with

Solution: Lagrangian cost function

Bernd Girod: EE398A Image and Video Compression

Iterative entropy-constrained scalar quantizer design

Guess initial set of representative levels xq ; q 0,1, 2,, M -1

Calculate M-1 decision thresholds

After 6 iterations, in both cases, (D-D)/D<1%

After 6 iterations, in both cases, (D-D)/D<1%