Wave 03

Wavelets: algorithms and applications
Pierre Carrier
Day 1 January 22, 2009

➢Introduction and examples
➢Essential tools of “xv” software for image translations and dimensioning
➢Digital image formats (and Huffman algorithm): image=multidimensional signals
➢Reminder of the essential ingredients of the wavelets theory (Haar- Daubechies)
➢Discrete wavelet transformation in terms of matrices for 1-D (vector) signal
➢Harr, Daubechies-04, Daubechies-06, ..., Daubechies-{L+1}
Day 2 February 5, 2009
➢Structure of algorithm for 1-D Daubechies transformation and inverse

➢Algorithm for 2-D (matrix) Daubechies transformation and inverse
➢Parallelization, reduction of memory using read-write vectors, or parallel algorithm
➢Compression; Edge detection
➢Program: implementation, structure, example runs
➢Coiflets, Biorthogonal filter, jpeg2000.

➢Complex Daubechies and extention to higher dimension filters. WHY?
➢Digression on quantum mechanics and operators (eigenvalue problems)
➢Multiwavelets using Legendre and interpolating polynomials
Daubechies scaling function
Wavelets
Daubechies
Unique solution
for polynomial with
|z| >1 (z complex)
✔Orthogonal
✔Using conditions of Orthogonality (on h and g) + conditions on symbols
✗Cannot have compact support, symmetric and orthogonality at the same
time for finite length finite impulse response (FIR).
e.g., existence of wrapping rows (periodicity of filter)
More symmetric than Daubechies

but still asymmetric
Image from Wolfram
Wavelets
Coiflet Daubechies
✔Orthogonal
✔Conditions on symbols only
✗Compact support but non-symmetric for finite length finite impulse
response (FIR). e.g., existence of wrapping rows (periodicity of filter)
Daubechies' system of equations:
Orthogonality
Symbols
Derivatives of symbols
Coifman's system of equations:
Orthogonality
Symbols
Find a system of equations on the symbols only that replaces orthogonality conditions
Why? get rid of conditions resulting directly from the transform matrix (on the h's)
(Uniformity of the conditions)
In fact, Coifman wavelets (or “Coiflets”) corresponds to one equation that replaces
the set of 4 equations above.
Coiflet filter theorem follows...

Coiflets:
Construction of Coiflet filters on pages 289-300 (van Fleet), is based on:

-orthogonality and finite-length Fourier series
-and on the following trigonometric identity
Remark: The general expression relating high to low pass filter is

This is the condition of

orthogonality using symbols
Example: Coiflets K=1 (length 6)
Same algorithm as Daubechies; replace h and g coefficients

Not yet symmetric
Image from Wolfram
Wavelets
Coiflet Daubechies
✔Orthogonal
✗compact support and non-symmetric for finite length finite impulse
response (FIR). e.g., existence of wrapping rows (periodicity of signal)
Important theorem
(constraint on wavelet construction):
If the wavelet and scaling functions
are orthogonal with compact support,
and at least one of the two functions
is symmetric or antisymmetric, then
it must be the Haar wavelet.
Proof in Ten Lectures on Wavelets
From” Wavelets and Other Orthogonal System”, Walter & Shen
by Daubechies, chapter 8.
Bi-orthogonal Wavelets
Coiflet Daubechies
wavelets
✔Almost Orthogonal
✗compact support and symmetric for finite length finite impulse response
(FIR). Can reduce wrapping rows (by making the signal periodic)
Objective: construct a symmetric finite-length almost orthogonal wavelet
Haar (or Daubechies-02) is the only symmetric Daubechies filter.
Why? Can handle better non-periodic conditions (wrapping rows problem)
Daubechies' system of equations:
Orthogonality
Symbols
Biorthogonal filters:
RELAX THIS CONDITION

Orthogonality
Symbols
Keep Derivatives of symbols
Solution Construct 2 sets with a special connection

between the two. That condition is:
For Coiflets, we saw that the construction is based on orthogonality conditions on H and G:
One can show (Van Fleet, pages 356 and on) that this condition above (for Coiflets) can be
extended to bi-orthogonal wavelets, with
+ 3 similar expression (due to the 4 possible permutations of H, G with its tildes)

Biorthogonal definition:
The condition on g (and its tilde) is related to h (and its tilde) in a similar way
as for Coiflets:
tilde
Remark 1: the 2 sets of approximations h and its tilde can be (or are usually) of different length
[it is similar to (but not equal to) combining Daub-02 with Daub-06]
Remark 2: The length is not necessarily even (unlike Daubechies-02, 04, 06,...)
Remark 3: There exist extensive conditions on the index of the filters (Van Fleet, page 366)
Remark 4: biorthogonality does not automatically imply symmetry (p. 361),
but some biorthogonal wavelets can be made symmetric
Daubechies (SIAM J. Math Anal, 24 499, 1993) suggested to choose
one filter (the tilde) and to deduce the other one from the condition
Given Odd Spline filters (based on Pascal's triangle)
Then deduce h from biorthogonal condition above. The result is,
(Solution given by I. Daubechies)
Find:
+ parity condition (van Fleet pages 286 & 386)

Example: N = N = 2:
Given:
Find:
These are the h coefficients
This example is called “(3,5) biorthogonal wavelet”

Cohen-Daubechies-Fauveau: CDF9/7
JPEG2000 (lossy part) is similar in principle to the “(9,7) biorthogonal wavelet”.

It uses the same orthogonality condition as biorthogonal wavelets:
However, instead of fixing H and then finding H as in the biorthogonal case, in the
CDF9/7 both H and H are assumed to have the form:
Given:
Given:
Then we search for the polynomials p and p such that the following product is satisfied:
Find:
with
DETAIL: Van Fleet, page 400.
Advantage: CDF9/7 is “more” orthogonal than the (9,7)-biorthogonal

see Unser & Blu, IEEE Trans. Image Proc. 12, 1080 (2003)
Cohen-Daubechies-Fauveau: CDF9/7
http://www.jpeg.org/jpeg2000/index.html
Solution to CDF9/7: (This is jpeg2000!)
Remark concerning patents and jpeg:

The litigation concerns U.S. Patent 4,698,672 owned by Compression Labs, Inc
http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=1&f=G&l=50&co1=AND&d=PTXT&s1=4,698,672.PN.&OS=PN/4,698,672&RS=PN/4,698,672
also other litigations with Lucent and Phillips, apparently. This concerns jpeg itself.
However,...
: JPEG2000: “The up and coming JPEG 2000 standard has been prepared along these lines, and
agreement reached with over 20 large organisations holding many patents in this area to allow use of
their intellectual property in connection with the standard without payment of license fees or royalties.”
See further info at http://www.jpeg.org/newsrel1.html

Conclusion: CDF9/7 seems to be available
Computer implementation is similar to Daubechies as described in talk #2,

except that one needs to define h and g properly before applying the algorithm:
The main difference with Daubechies is that Daubechies are even length filters only,
while in the case of biorthogonal filters the length can be odd or even, and both lengths
differ [e.g., (5,3), (9,7), (8,4), etc]
Algorithm is described in Chapter 11 of Van Fleet, page 414 (not verified)
There is also a version(page 435-457) that takes into account the symmetry of the
coefficients so as to avoid periodicity (the wrapping rows).
This is the main reason for inventing (M,N)-biorthogonal filters:
Signal “abcdefg” is symmetrized into “abcdefggfedcb”

Pierre Carrier



Problem:
One cannot have compact support with symmetric and orthogonal wavelet, except for Haar
See K. Fritz, Theorem 1.35 or Daubechies' Ten lectures on wavelets chapter 8
SCALAR wavelets
Coiflet Daubechies
wavelets
Consequence:
One needs to trade the following 3 properties, for generalizing Haar wavelet:
orthogonality + symmetry = compact support

Haar: Yes Yes Yes
Daubechies: Yes No Yes
Coiflet: Yes No (~Yes) Yes
Bi-orthogonal: No Yes Yes
CDF9/7 No (~Yes) Yes Yes
Daubechies wavelets:
Real solution are always non-symmetric;
Imaginary solutions can be made symmetric
Complex Daubechies
Coiflet Daubechies
wavelets
See also
W. Lawton
IEEE trans Signal Proc.,
41, 3566 (1993)
Im{z} y
n
R
Re{z} x
Balanced multiwavelets
Complex Daubechies
Coiflet Daubechies
wavelets
Lawton, Lee, & Shen, An Algorithm for Matrix Extension and Wavelet Construction,
Math. Comput. 65, 723 (1996)
Lebrun & Vetterli, High-Order Balanced Multiwavelets: Theory, Factorization, & Design
IEEE Trans. Signal Proces. 49, 1918 (2001)
On the other hand,...
Multiwavelets
Alpert, Belkin, Harrison,..., have been working on Legendre multiwavelets

applied to integral operators, not on signals processing
Goal: Solve integrodifferential equations
Application: Hamilton equation, an eigenvalue problem,

Navier-Stokes, an advection-diffusion problem
2
B. Alpert, A Class of Bases in L for the Sparse Representation of Integral Operators
SIAM J. Math. Anal. 24, 246 (1993)
It turns out that all the wavelets approaches (applied to signal processing or integration)
are related to the same theory, now called multiwavelets
Remark: multiwavelets can also be made bi-orthogonal multiwavelets,
which is the most general case.
Multiwavelets
Balanced multiwavelets
Complex Daubechies
Coiflet Daubechies
wavelets
✔Orthogonal
✔Compact support FIR
✔Discontinuous (similar to Haar)
✔Lead to sparse matrices with small norm (effectively sparse) SCALAR wavelets
✔Simple construction and flexible (explicitly uses vanishing moments) VECTOR wavelets
✔Scaling function symmetric; multiwavelets (anti-)symmetric
Pierre Carrier



100 years of physics in 1 minute!
Schroedinger equation:
Kinetic opposes potential energy
Pendulum: That's an
Eigenvalue problem
Potential high;
kinetic low Atoms act the same way
as a pendulum
[think of magnetic resonance
Potential low; imaging (MRI) in medicine]
kinetic high
Density functional theory is a simplification of this “many-body” (electrons) problem where you
use the density as the unknown variable. Walter Kohn, Nobel Prize in Chemistry, 1998
How to write this eigenvalue problem into an integral problem?
Schroedinger equation:
Suppose that the eigenvalue E is fixed (known) and equals -B

Next, define:
Schroedinger equation becomes:

Green's function
Why this?: to put potential on the R.H.S.

So that one can use Green functions where
formalism
G is a function that decays with R
The formal solution using Green's function is Fredholm of the second kind:
eigenvector
One can show that by integrating on the center of mass (a projection),

then the condition E = -B becomes general
M. H. Kalos, Monte Carlo Calculations of the Ground State of 3- and 4-body nuclei,
Phys. Rev. 128, 1791 (1962)
The formal solution using Green's function is Fredholm of the second kind:
eigenvector
One can show that by integrating on the center of mass (a projection),

then the condition E = -B becomes general
M. H. Kalos, Monte Carlo Calculations of the Ground State of 3- and 4-body nuclei,
Phys. Rev. 128, 1791 (1962)
This is called the Lippmann-Schwinger integral formulation

Discretization
A discrete eigenvalue problem

kernel
kernel
Daubechies' wavelets are not easily applicable because it does not form a bases;
it's a scalar function, a unique function.
kernel
The bj's are the

multiwavelets functions
Multiwavelets were invented in that context: find an orthogonal basis with vanishing moments.
Pierre Carrier



Fundamental articles, 2 constructions:
2
SIAM J. Math. Anal. 24, 246 (1993)
Using Legendre scaling functions
B. Alpert, G. Belkin, D. Gines, & L. Vozovoi, Adaptive Solution of PDE in multiwavelet bases
J. Comput. Phys. 182, 149 (2002)
Using Interpolating (Lagrange) scaling functions
Fundamental difference between Daubechies' wavelets (scalar) and multiwavelets:
Scalar wavelets (Daubechies, Coiflet, bi-ortho, CDF9/7): Multiwavelets:
Scaling and wavelet functions are unknown. Scaling and wavelet functions are given.
Define “symbol” (Fourier series of h, g) where Use vanishing moments condition and
vanishing moments and ortho. are implicitly satisfied orthogonality explicitly (Gauss-Legendre)
(through the definition of symbols and derivatives),
then deduce h and g, the coefficients then deduce h and g, the coefficients
Define scaling functions using Legendre:
Legendre, image from Wolfram
or Define scaling functions using interpolating basis
Multiwavelet functions must satisfy:
Orthogonality
Vanishing moments
Remember in the case of scalars:
Scaling function
Wavelet function
Remember in the case of scalars:
Scaling function
Wavelet function
Same definition with multiwavelets:
[0,1/2] [1/2,1]
Scaling function
Wavelet function
Notice the difference between wavelets (scalar) and multiwavelets (vectors):
The coefficients h are vectors:
The coefficients h are matrices:

Quadrature Mirror Filters (QMF):
To get the coefficients (matrices) h and g , one multiplies both sides of the equation by the
ij ij
function of interest on the R.H.S. Then integrate and use orthogonality conditions between
each function.
Solution:
Solved using Gauss-Legendre quadrature [see page 159,Alpert et al (2002)]
example:
Remark: using Legendre polynomials of degree 0 gives Haar.

Complete closed scheme: Discrete multiwavelets transformation
Complete closed scheme:
Legendre
scaling function:
Scaling matrix
Legendre
scaling function:
Legendre
scaling function:
Legendre
scaling function:
Multiwavelet matrix
multiwavelet function:
Defined through the scaling functions

Using Gauss-Legendre quadrature

Legendre scaling function:



➢Multiwavelets using Legendre polynomials
Using this basis leads to sparse matrix of operators:
First simple example:
2
SIAM J. Math. Anal. 24, 246 (1993)
➢Multiwavelets using Legendre polynomials
Choose Scaling functions on [0,1] interval:
Evaluate the multiwavelet functions on [0,1] interval:
Constructed using conditions of orthogonality and

vanishing moments.
-scaling functions have compact support

-they are orthogonal
-they are symmetric or anti-symmetric.
-Construction is straightforward and follows polynomial order
Leads to an O(N) algorithm
Harrison's program from ORNL

[NWChem]
is the only program that I found
that is reported to be of O(N),
based on MRA. Taken from Fann et al, IBM J Res & Dev 48, 161 (2004)

Wave 03

Enviado por

Dados do documento

Descrição original:

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Wave 03

Enviado por

Direitos autorais:

Formatos disponíveis

Wavelets: algorithms and applications

Day 1 January 22, 2009

Day 2 February 5, 2009

➢Structure of algorithm for 1-D Daubechies transformation and inverse

Day 3 February 19, 2009

➢Coiflets, Biorthogonal filter, jpeg2000.

Daubechies scaling function

More symmetric than Daubechies

Image from Wolfram

Daubechies' system of equations:

Coifman's system of equations:

Coiflet filter theorem follows...

Construction of Coiflet filters on pages 289-300 (van Fleet), is based on:

-and on the following trigonometric identity

Remark: The general expression relating high to low pass filter is

This is the condition of

Example: Coiflets K=1 (length 6)

Same algorithm as Daubechies; replace h and g coefficients

Not yet symmetric

Image from Wolfram

Objective: construct a symmetric finite-length almost orthogonal wavelet

Haar (or Daubechies-02) is the only symmetric Daubechies filter.

Why? Can handle better non-periodic conditions (wrapping rows problem)

Daubechies' system of equations:

RELAX THIS CONDITION

Keep Derivatives of symbols

Solution Construct 2 sets with a special connection

+ 3 similar expression (due to the 4 possible permutations of H, G with its tildes)

Given Odd Spline filters (based on Pascal's triangle)

Then deduce h from biorthogonal condition above. The result is,

(Solution given by I. Daubechies)

+ parity condition (van Fleet pages 286 & 386)

These are the h coefficients

This example is called “(3,5) biorthogonal wavelet”

JPEG2000 (lossy part) is similar in principle to the “(9,7) biorthogonal wavelet”.

DETAIL: Van Fleet, page 400.

Advantage: CDF9/7 is “more” orthogonal than the (9,7)-biorthogonal

Solution to CDF9/7: (This is jpeg2000!)

Remark concerning patents and jpeg:

See further info at http://www.jpeg.org/newsrel1.html

Computer implementation is similar to Daubechies as described in talk #2,

Algorithm is described in Chapter 11 of Van Fleet, page 414 (not verified)

Signal “abcdefg” is symmetrized into “abcdefggfedcb”

Day 1 January 22, 2009

Day 2 February 5, 2009

➢Structure of algorithm for 1-D Daubechies transformation and inverse

Day 3 February 19, 2009

➢Coiflets, Biorthogonal filter, jpeg2000.

orthogonality + symmetry = compact support

On the other hand,...

Alpert, Belkin, Harrison,..., have been working on Legendre multiwavelets

Goal: Solve integrodifferential equations

Application: Hamilton equation, an eigenvalue problem,

Day 1 January 22, 2009

Day 2 February 5, 2009

➢Structure of algorithm for 1-D Daubechies transformation and inverse

Day 3 February 19, 2009

➢Coiflets, Biorthogonal filter, jpeg2000.

100 years of physics in 1 minute!

Kinetic opposes potential energy

How to write this eigenvalue problem into an integral problem?

Suppose that the eigenvalue E is fixed (known) and equals -B