Escolar Documentos
Profissional Documentos
Cultura Documentos
=
2
] [ ) (
Z n
n
z n x z X
where
n
z
2 1
2 1
n n
z z
. Specializing to the unit sphere, we
have
e j
e z = ) (
2 1
e e j j
e e , hence ) (
e j
e X represents the
Fourier transform of ] [n x .
II. BACKGROUND AND PROBLEM STATEMENT
In this section the original AP demosaicking algorithm is
briefly discussed and the key problems that are to be
addressed in this work are stated explicitly.
A. Iterative Demosaicking Algorithm
Let ] [n s denote the raw sensor image obtained from the
CFA, and let ] [ ], [ ], [ n b n g n r represent the full-resolution red,
green, and blue color channels, respectively. The goal of the
AP algorithm starts by obtaining an estimate of the full
resolution green channel, which is then used in the subsequent
estimation of the missing red and blue pixels. This two step
strategy is justified by the fact that the green channel in the
Bayer CFA has twice the sampling density as that of red or
blue channels (Fig. 1), and, hence, is relatively easier to
reconstruct.
The ultimate goal of this section is to focus on how the AP
algorithm iteratively interpolates the missing red and blue
pixels. For simplicity, the case of red pixels is considered
throughout the paper. The processing steps of blue pixels can
be inferred by symmetry.
The first constraint set introduced by the AP algorithm
exploits the correlation between the detail subbands of the
green and red channels. Consider a generic 2-D filter bank
which decomposes an input image ] [n x into one lowpass
subband
] [n
L
x
and N different highpass subbands ] [n
i
H
x
for N i ,..... 1 = . The set of analysis filters { ) (z
i
A } and
synthesis filters { ) (z
i
S } satisfies the perfect reconstruction
condition,
1 ) ( ) (
i
z
i
S z
i
A (1)
One concrete example of such filter bank is the 2-D
undecimated wavelet transform used in [13]. With one level
decomposition, we obtain one lowpass channel and three
highpass channels. Meanwhile the channel filters are all
separable products of 1-D filters. Based on the observation
that the highpass subbands of the green and red channels of a
natural image tend to be very similar, the AP algorithm
proposes to search for full-resolution red image within the
following detail constraint set,
N i for T n g ai n x
i
a n x
d
C s s s = 1 , | ] )[ ( ] )[ ( :| ] [ {
(2)
where ] [n
i
a is the i
th
highpass analysis filter, ] [n g is the
previously estimated full-resolution green channel, and T is a
threshold indicating how similar the two signals should be. To
enforce the constraint defined in (2), the AP algorithm
employs an update operator, defined in the transform domain
as,
=
+ = =
N
i
z G z
i
A z
i
S z X z A z S z Y n x
d
P n y
1
)) ( ) ( )( ( )) ( ) (
0
)( (
0
) ( ] )[ ( ] [
(3)
The above formula (3) can be simplified as follows. Let
) (z H ) ( ) (
0 0
z A z S (4)
Simplifying (3) we get,
) ( )) ( 1 ( ) ( ) ( ) ( z G z H z X z H z Y + = (5)
Compared with the original definition (3) for P
d
, the proposed
formula in (16) only requires the filtering operation of
) (z X and ) (z G in the lowpass channel of the filter bank.
Consequently the computations previously needed in
November Issue Page 39 of 59 ISSN 2229 5208
International Journal of Computer Information Systems,
Vol. 3, No. 5, 2011
obtaining the three highpass channels of ) (z G can be
eliminated.
The second constraint set that the AP algorithm employs
based on the available sensor measurements ] [n s . Most
specifically, the interpolated red channel should belong to the
following observation constraint set,
o
C } for ], [ ] [ : ] [ {
r
n n s n x n x I e = (6)
where
r
I } all for : ) 1
2
2 ,
1
2 ( { Z e + = k2 k1, k k n
represents the locations of the red pixels in the Bayer CFA as
shown in Fig. 1. To enforce the above constraint, we can
define the observation update operator
o
P as follows
(7)
In words, the operator
o
P replaces the value of ] [n x at the red
pixel locations with the available sensor measurements, but
leaves the rest of the pixels intact.
B. Problem Statement
We first recall the following facts about projection onto
convex sets (POCS).
Definition 1: Let C be a closed convex set in Hilbert
space H , for any x H there exists a unique element y
x
C
such that
|| x - y
x
|| || x z ||, for all z C (8)
The mapping P
c
: H H, P
c
x y
x
is the projection operator
onto C.
Theorem 1: Let C
1
and C
2
be the closed convex sets in a
Hilbert space H, and
1
C
P and
2
C
P the corresponding projection
operators. Suppose that C
1
C
2
. For any x
(0)
e H, the
sequence
0,1,2 = k
(k)
C C
1) + (k 1) + (k
} x P P = x : x {
2 1
.
converges to
the projection of
) 0 (
x onto C
1
C
2
, i.e. ,
||
) (k
x -
) 0 (
2 1
x P
C C
|| = 0 (9)
To invoke the above theorem in the context of the AP
algorithm, one can easily verify that the two constraint sets C
o
and C
d
defined in Section II(A) are closed and convex.
Meanwhile the observation projection operator P
o
in (7) is
indeed the projection onto C
o
. However, this is in general not
the case for the detail update operator P
d
.
Proposition 1.
d
P is a projection operator if and only if
) (
e j
e H = 1
D
(e), (10)
where ) (
e j
e H is the filter defined in (4), and 1
D
(e) is the
indicator function defined on the frequency domain support D
of ) (
e j
e H .
Remark 1. Proposition 1 states that, for P
d
to be a projection
operator, the lowpass filter ) (
e j
e H must be ideal in the
frequency domain. Consequently, ] [n h has to be a sinc-like
filter in the spatial domain, and cannot have a finite impulse
response (FIR).
Example 1. The separable filters used in AP algorithm are
. ] 1 - 2 6 2 1 [ ] 1 - 2 6 2 -1 [
64
1
] [
, ] 1 2 1 [ ] 1 2 1 [
16
1
] [
0
T
o
T
n s
and n a
=
=
Algorithm 1 Interpolate Missing Red/Blue Pixels:
Input: The raw CFA image ] [n s and the estimated green
channel g .
Output: An estimated full-resolution red channel.
Begin
Obtain r
(0)
, b
(0)
, g
(0)
using edge directed interpolation.
Initialize: i
IF
Then
Enforce the detail constraint: r
(i+0.5)
= P
d
r
(i)
Enforce the observation constraint: r
(i+1)
= P
o
r
(i+0.5)
i = i+1
Else
return r
(i)
End
In Fig. 2. the magnitude frequency response
) (
e j
e H ) ( ) (
0 0
e e j
e A
j
e S is shown. Evidently, since
both ] [
0
n a and ] [
0
n s are FIR filters, | ) (
e j
e H | is non ideal
and does not satisfy (10), and thus P
d
is not a projection
operator.
Nevertheless, we know from numerical experiments that the
AP algorithm based on this filter still converges after a small
number of iterations.
Example 2. Another interesting fact about the AP algorithm is
that the final result of the iteration process appears to be
independent of the starting point. To demonstrate this
phenomenon, we apply the AP algorithm to the standard test
November Issue Page 40 of 59 ISSN 2229 5208
International Journal of Computer Information Systems,
Vol. 3, No. 5, 2011
image light house from the Kodak set, with three different
initial estimates for
] [
) 0 (
n r
: bilinear interpolation, all zeros,
and i.i.d. random numbers uniformly distributed from 0 to
255. For each initial estimate, the iteration process as
described in Algorithm 1 and
. (11)
where ] [
) (
n r
k
is the estimated red channel at the k
th
iteration,
] [n r
BL
is the convergence value obtained by choosing bilinear
interpolation as the starting point. The results are shown in
Fig. 3. It is seen that the different choices of the initial
estimates only affect the speed of convergence, but all three
processes eventually converge to the same result ] [n r
BL
.
-pi
0
pi
-pi
0
pi
0
0.2
0.4
0.6
0.8
1
Fig. 2. The magnitude frequency response of the lowpass filter ) (
e j
e H
.
III. CONVERGENCE OF AP ALGORITHM
In this section, a rigorous analysis of the convergence property
of the AP algorithm is provided. The theoretical analysis will
be heavily based on the polyphase representation [14], [15] of
signals. To facilitate readers who are not familiar with this
concept, we start our discussion by having a brief overview of
this useful signal representation.
A. Polyphase Representation
The polyphase representation (transform) is a powerful tool in
multirate signal processing. It also plays a critical role in the
derivations presented in later parts of this paper. For a 2-D
image ] [n x , we can decompose it into four polyphase
components ] [ ], [ ], [ ], [
11 10 01 00
n x n x n x n x defined as
} 1 , 0 { , for ], 2 , 2 [ ] , [
2 1 2 1 ,
e + + = j i j n i n x n n x
j i
(12)
The polyphase components in (12) are simply downsampled
versions of the original signal ] [n x . The sampling locations of
all four polyphase components form a partition. Note that the
mapping between the signal x[n] and its polyphase
components is one-to-one. To go back to the original signal
from its polyphase components, we can easily verify that
) (z X z ) (z X ) + z (z X(z) = X
2
10
1
1
2
01
1
2
2
00
+
). (z X z + z
2
11
1
2
1
1
(13)
The polyphase representation becomes especially handy when
we try to describe the periodic sampling structures of color
filter arrays. To describe the filtering operation in the
polyphase domain, consider two signals x[n], y[n] such that
] )[ ( ] [ n h x n y = (14)
0 5 10 15 20 25 30 35 40
10
-10
10
-5
10
0
10
5
iteration number
M
S
E
bilinear
zero
random
Fig. 3. Convergence of the AP algorithm for different choices of initial
estimates.
for some filter ] [n h .Denote by
T
p
[n]] [n], x [n], x [n], x [x [n] x
11 01 01 00
=
and
T
p
z X z X z X z X z X )] ( ), ( ), ( ), ( [ ) (
11 10 01 00
= (15)
the vector of all four polyphase components of ] [n x , and the
corresponding vector in the transform domain, respectively.
Similarly, we can define [n] y
p
and (z) Y
p
) for ] [n y ] as well.
The filtering operation (14) can be described in the polyphase
domain as
) ( ) ( ) ( z X z = H z Y
p p
(16)
At this point, it may seem that going to the polyphase domain
only makes the filtering operation more complicated. As we
shall see below however, the primary advantage in using the
November Issue Page 41 of 59 ISSN 2229 5208
International Journal of Computer Information Systems,
Vol. 3, No. 5, 2011
polyphase notation is that it can convert the shift-variant
operator P
o
P
d
used in the AP algorithm to a multiple-input-
multiple-output system of shift-invariant (filtering) operations.
The added complexity of dealing with matrix-vector
multiplications are more than compensated for by the
convenience of working with shift-invariant operators.
Definition 2. A mapping T from a Hilbert space H to itself is
called Lipschitz continuous, if there is some real number > 0
such that
H x x x x Tx Tx e s
2 1 2 1 2 1
, all for | | , | | | | | | o (17)
The smallest such value of , denoted by
T
, is called the
Lipschitz constant of T . Furthermore, if
T
< 1, then T is a
contraction mapping.
Theorem 3. [16] Let T : H H be a contraction mapping with
Lipschitz constant 0 <
T
< 1.
1. The mapping T admits one and only one fixed point .
2. For arbitrary x
(0)
e H, the iterated sequence x
(k+1)
= T (x
(k)
)
always converges to x .
3. The speed of convergence is bounded by the following
inequality
|| x
(k)
- x ||s (
T
)
k
/ 1-
T
|| x
(k)
- x ||.
Now we just need to check that the mapping T is indeed a
contraction. If that is the case, then the convergence of the AP
algorithm will be automatically guaranteed by the contraction
mapping theorem stated above. The following proposition
presents a simple way to obtaining the Lipschitz constant of T.
Proposition 2. The Lipschitz constant of the mapping T can be
calculated as
T
=max( o
max
(T(e
je
))),
where
max
( ) denotes the largest singular value of a matrix.
So far, we have used the polyphase representation and the
contraction mapping theorem for the rigorous explanation of
the convergent property, and the corresponding condition for
the filter h[n] used in the iteration process. The convergence
value of the iteration process appears to be unique and does
not depend on the starting points. The following section shows
that we can directly reach the convergence result of the AP
algorithm without going through iterations
IV. NONITERATIVE IMPLEMENTATION OF THE AP ALGORITHM
A. General Schemes
We know from Theorem 3 that, if AP algorithm is convergent,
then it must converge to the unique fixed point of the mapping
T.
Let ) (
z C
v
denote the transform domain representation of the
fixed point. Applying the definition of the mapping T we
obtain,
) ( ) (
) ( )) (
( ) (
z B z C z T z C T z C
v v v
+ = =
where T (z) is a matrix obtained from H(z) after removing its
2
nd
row and 2
nd
column, and
TABLE I. PERFORMANCE COMPARISON OF ORIGINAL AP ALGORITHM
AND THE PROPOSED NONITERATIVE SCHEME
PSNR(dB)
Red Green Blue
AP(15
iterations)
38.47 41.81 38.60
AP(5
iterations)
38.44 41.81 38.48
Proposed
scheme
38.43 41.81 38.47
Fig. 4. The block diagram of the proposed noniterative implementation of the
AP algorithm.
(z)) G (z) (S (z)] , H (z)z , H (z)z [H B(z)
T -
01
01 10
1
2 11
1
2 01
=
The above equality implies
that )) )) ( 1 ( ) (
01
01 10
1
2 11
1
2 01
1
(z G (z) (S (z)] , H (z)z , H (z)z [H z T z v C
T -
=
) (
z C
v
) ( )] ( ), ( ), ( [ 01
01 11 10 00
(z) G (z) S z F z F z F
T
(18)
Consequently (18) means
)) (
01
) (
01
)( ( ) (
z G z S z
ij
F z
v
C =
where } (1,1) (1,0), ), 0 , 0 {( , e j i
In words, the final convergence result of the AP algorithm can
be directly obtained by three filtering operations in the
November Issue Page 42 of 59 ISSN 2229 5208
International Journal of Computer Information Systems,
Vol. 3, No. 5, 2011
polyphase domain. Meanwhile, the remaining polyphase
component, ) (
01
z C , is equal to ) (
01
) (
01
( z G z S .The block
diagram of the proposed Noniterative implementation of the
AP algorithm is shown in Fig. 4.
B. Numerical Experiments
To demonstrate the performance of the proposed scheme,
both the proposed Noniterative algorithm and the original AP
algorithm are applied to the standard Kodak test images. For
the proposed algorithm, the three polyphase filters F
00
(z),
F
10
(z), F
11
(z) defined in (18) are generally not FIR. However
these filters can be well-approximated by their finitely-
truncated versions. For AP algorithm, we test two different
options for maximum number of iterations 15 and 5. The
former will ensure us to obtain the convergence value of the
TABLE II. PERFORMANCE COMPARISON OF ORIGINAL AP ALGORITHM AND
THE PROPOSED NONITERATIVE SCHEME
Time(second)
Initialization
stage
Refinement
stage
Total
AP(15
iterations)
0.388 2.138 2.526
AP(5
iterations)
0.388 6.267 6.656
Proposed
scheme
0.125 0.161 0.286
procedure while the later is commonly used in practice due to
its reduced computational load. Both the algorithms are
implemented in Matlab. Table I and II summarizes the main
results, averaged over 24 test images. The accuracy of the
algorithm is measured in terms of peak signal to noise ratio
(PSNR) of the demosaicked image. Note that the PSNR values
for green channel are same for different algorithms, since both
AP and the proposed scheme follow the same steps in
estimating the green pixels. For the red and blue pixels, the
PSNR values obtained by the proposed scheme are very close
to those reached by the AP after 15 iterations. This indicates
that this Noniterative algorithm can accurately compute the
final convergence values of the AP procedure, even though
the polyphase filters used in this implementation are truncated
approximation.
The main advantage of the proposed algorithm is its
computational efficiency. As shown in table II, the total time
is divided into two parts: initialization stage(estimating the
missing green pixels), refinement stage (estimating the
missing red or blue pixels). More substantial improvements
can be found in refinement stage: the original AP algorithm
approaches the reconstruction problem via iteration, whereas
the proposed scheme achieves the same goal by direct filtering
operations in polyphase domain. Correspondingly the running
time can be dramatically reduced.
V. CONCLUSIONS
This paper presents a detail treatment of a classical color
image demosaicking algorithm based on alternating
projections. On the theoretical side, this analysis provides a
rigorous foundation for the convergence of the AP algorithm,
based on contraction mapping theorem. On practical side, an
efficient Noniterative implementation of AP algorithm in
polyphase domain is proposed. Numerical experiments
confirm that the proposed algorithm can achieve the same
results obtained by AP algorithm at convergence, but
significantly faster than latter.
ACKNOWLEDGMENT
First and foremost we thank Almighty God for his grace to
complete this work. We would like to thank Cambridge
Institute of technology for providing permission and
encouragement throughout this work.
REFERENCES
[1] B. E. Bayer, Color imaging array, U.S. Patent 3 971 065,
1975.Eastman Kodak Company.
[2] P. Longre, X. Zhang, P. B. Delahunt, and D. H. Brainard,
Perceptualassessment of demosaicing algorithm performance, Proc.
IEEE, vol.90, pp. 123132, 2002.
[3] H. S. Hou et al., Cubic splines for image interpolation and digital
filtering, IEEE Trans. Acoust., Speech, Signal Process,, vol. ASSP-26,
no.3, pp. 508517, Jun. 1987.
[4] J. E. Adams, Intersections between color plane interpolation and other
image processing functions in electronic photography, Proc. SPIE,
vol.2416, pp. 144151, 1995.
[5] D. R. Cok, Signal Processing method and apparatus for producing
interpolated chrominance values in a sampled color image signal, U.S.
Patent 4 6 42 678, 1987. Eastman Kodak Company.
[6] J. A. Weldy, Optimized design for a single-sensor color electronic
camera system, Proc. SPIE, vol. 1071, pp. 300307, 1988.
[7] J. E. Adams and J. F. Hamilton Jr., Adaptive color plane interpolation
in single color electronic camera, U.S. Patent 5 506 619, 1996.
[8] J. E. Adams, Design of practical color filter array interpolation
algorithms for digital cameras, Proc. SPIE, vol. 3028, pp. 117125,
1997.
[9] J. F. Hamilton Jr. and J. E. Adams, Adaptive color plane
interpolationin single sensor color electronic camera, U.S. Patent 5 629
734, 1997.
[10] E. Chang, S. Cheung, and D. Y. Pan, Color filter array recovery using a
threshold-based variable number of gradients, Proc. SPIE, vol. 3650,
pp. 3643, 1999.
[11] R. Ramnath and W. E. Snyder, Adaptive demosaicking,
J.Electron.Imag., vol. 12, no. 4, pp. 633642, 2003.
[12] R. Kimmel, Demosaicing: Image reconstruction from CCD
samples,IEEE Trans. Image Process., vol. 8, no. 11, pp. 12211228,
Nov. 1999.
[13] B. K. Gunturk, Y. Altunbasak, and R. M. Mersereau, Color plane
interpolation using alternating projections, IEEE Trans. Image
Process.,vol. 11, no. 9, pp. 9971013, Sep. 2002.
[14] Vaidyanathan, P. P., Multirate Systems and Filter Banks, Prentice-Hall,
Englewood Cliffs, NJ (1993).
[15] Vetterli, M. and Kovacevic, J., Wavelets and Subband Coding, Prentice
Hall, Englewood Cliffs, NJ (1995).
November Issue Page 43 of 59 ISSN 2229 5208
International Journal of Computer Information Systems,
Vol. 3, No. 5, 2011
Authors
Jenita Subash
She did her B.E in ECE from SCAD College of
engineering and technology in 2004, Anna University ,
Chennai. She acquired her M.E in Communication System in
PET Engineering College,Anna University of Technology,
Thirunelveli in 2011.
She worked as a Lecturer in ECE dept. on PSN
Engineering college, Thidiyoor during 2005-2009
Currently she is working as a Lecturer in ECE Dept in
Cambridge institute of technology, K.R Puram, Bangalore
Girish H
He did his B.E in ECE from Adichunchagiri institute of
Technology, chikmagalur and M. Tech in United
Technologies Limited ( VTU extension Center),
Yeshwanthpur, Bangalore. He is currently working as a
Assistant Professor in Cambridge Institute of technology,
K.R.Puram, Bangalore-16.
November Issue Page 44 of 59 ISSN 2229 5208