Abstract Remote sensing satellites are limited by the

data transmission rates. As a result the MS (MS) bands are

transmitted at lower resolution and only the PAN band
(PAN) is transmitted at its full resolution. The information
contained in the MS bands is invaluable tool for land use
mapping, urban feature extraction, etc. However, the limited
spatial resolution of the MS images reduces the value of this
information. This paper introduces an image fusion
technique for achieving high spatial quality of the fused MS
image while retaining the spectral quality of the original MS
image. The proposed fusion technique integrates both the
intensity-hue-saturation (IHS) and the Laplacian pyramid
transform (LPT) techniques for combining the advantages of
both. The quality of the fused images has acceptable spatial
characteristics and preserves the spectral characteristics.
The experimental results show that the proposed technique is
superior to the other known fusion techniques.
Keywords Image fusion, IHS transform, Laplacian
pyramid transform, Remote sensing, Spatial and Spectral
quality
I. INTRODUCTION
Image fusion is the process of combining information
from two or more images of a scene into a single
composite image that is more informative and is more
suitable for visual perception or computer processing. The
objective of image fusion is to reduce uncertainty and
minimize redundancy in the output while maximizing
relevant information particular to an application [14].
In remote sensing applications, the fusion of low
resolution MS and high resolution PAN satellite images is
very important issue for many remote sensing and
mapping applications. To effectively utilize such images,
techniques that can combine high-resolution PAN and
low-resolution MS images into one color image are
demanded.
Many image fusion techniques and software tools have
been developed. The well-known techniques are, for
example, (IHS) transform [2], principal component
analysis (PCA) [4], discrete wavelet transform (DWT) [3]-
[8], discrete wavelet frame transform DWFT [6], LPT
[10]- [12], generalized Laplacian pyramid transform
(GLP) [15] and Fast Fourier transform (FFT)-enhanced
IHS method [1].
The IHS and PCA normally improve the spatial
resolution while distort the color composite so it is still
necessary to investigate how to improve the spatial
resolution of color appearance of the images. Later efforts
take advantage from an underlying multiresolution
analysis, by employing the DWT, DWFT, LPT, and GLP.
Although the DWT improves the quality of the fused
images compared to the other fusion techniques, it suffers
from the underlying down-sampling process so its
multiresolution decompositions and the fusion result are
shift variant. This is particularly undesirable when the
source images are not perfectly registered. The shift
variant problem of the DWT can sometimes cause aliasing
in application like pattern recognition or image fusion.
In this paper, we propose a new image fusion technique
based on IHS and LPT. It is known that the IHS transform
introduces images with high spatial quality, while the LPT
is shift invariant and can avoid ringing and blurring the
edges. Integrating both techniques preserves the spectral
quality and enhances the spatial quality of the fused
image.
The testing images are collected from an IKONOS
dataset with 1m PAN and 4 m MS images [13]. The
results compare the proposed fusion technique to the IHS,
DWT, LPT, and IHS-DWT techniques and show that the
new technique gives better discrepancy and correlation
coefficients and reduces the root mean square error of the
fused image.
II. The Proposed IHS-LPT Fusion technique:
The basic idea behind the IHS-LPT integrated fusion
technique is to modify the intensity component of the
input MS image to look more like the input high-
resolution PAN image. LPT One of the most frequently
studied versions of the pyramid transform. Each level of
the LPT is recursively constructed from its lower level by
the following four basic procedures: blurring (low-pass
filtering), sub-sampling (reduce size), interpolation
(expand in size), and differencing (to subtract two images
pixel-by-pixel) in the order we have given [7]- [11]. IHS
An IHS and Laplacian Pyramid Integrated Fusion Technique to
Improve the Visual Quality of Remote Sensing Satellite Images
Rasha Shoitan*, Zaki Nossair**, Tarek El.Tobely***, Hatem El-Bolok**
* Department of Communications Engineering , Modern University of Technology & Science, Cairo, Egypt
** Department of Electronics, Communications & Computer , Helwan University, Egypt
*** Department of Computer and Automatic Control, Tanta University, Egypt
NxM
) B (A
RMSE
M
1 i
N
1 j
2
j i, K, j i, k,
K

= =

=
transform separates the spatial (intensity) and spectral
information (hue& saturation) in a RGB image [9] then
replace the intensity with PAN image. Instead of using a
total replacement of the intensity component as with the
IHS technique [5], the IHS-LPT integrated fusion
technique uses a partial replacement based on LPT. The
detailed steps of this proposed technique are shown in Fig.
4 and can be summarized as follows:
1) The low resolution MS image is co-registered to the
same area as the high resolution PAN image and
resampled to the same resolution as the PAN image.
2) The three resampled bands of the MS image which
represent the RGB space are transformed into IHS
components.
3) Constructing one level of LPT for the PAN and
intensity (I) image according to the above four procedures
(blurring, sub-sampling, interpolating and differencing) to
generate P
HFC
and I
HFC
images, respectively.
4) substituting the I
HFC
with the P
HFC
5) Adding the new I
HFC
which is obtained from step 3
with I
LFC
to generate new intensity(I
new
)
6) Transforming the new intensity together with the hue
and saturation components back into RGB space.
Fig. 4. The integrated IHS-LPT fusion technique
III. Evaluations metrics
The emphasis of this paper is to produce fused image
that retaining the spectral quality of the original MS
images while achieving a high spatial quality. There are
mainly two ways to evaluate the quality of the fused
image -spectral and spatial quality metrics.
A. Spectral Quality Metrics
There are various spectral quality metrics used to
evaluate the spectral characteristics of the fused images
with respect to the original MS images
- Correlation Coefficient (CC
K
)
The correlation coefficient measures the closeness or
similarity between two Images [10]- [6]. It can vary
between 1 to +1. The formula to compute the correlation
between two images A (original MS image) and B (fused
image), both of size NxM pixels is given by
(1)
Where B , A is the mean of the fused and original MS
image respectively,
j i K
B
, ,
and
j i K
A
, ,
are the pixel values
at position ) , ( j i in the Kth band of the fused and original
MS image, respectively
- Discrepancy (D
K
)
The discrepancy measures the spectral quality of the
merged images band-by-band by taking the average
differences between the fused image and the
corresponding original MS band [6], which can be
described by the formula

= =
=
M
1 i
N
1 j
j i, K, j i, K, K
A B
NxM
1
D
(2)
Where
j i K
B
, ,
and
j i K
A
, ,
are the pixel values at
position ) , ( j i in the Kth band of the fused and original
MS image, respectively. The lower the value of the
discrepancy, the higher the spectral quality of the fused
images
- Root Mean Square Error (RMSE
K
)
The Root Mean Square Error (RMSE) measures the
amount of change per pixel due to the processing [10] and
is described by:
(3)
The lower the value of the RMSE, the higher the
spectral quality of the fused images


= =
= = =

= =

M
1 i
256
1 i
256
1 j
2
j i, k,
N
1 j
2
j i, k,
M
1 i
N
1 j
j i, k, j i, k,
K
) B (B ) A (A
) B )(B A (A
(A/B) Corr CC
B. Spatial Quality Metrics
The spatial quality evaluation of the fused images is
important since the goal is to retain the high spatial
resolution of the PAN image in the fused image.
- High Pass Correlation Coefficient (HPCC
k
)
The HPCC
k
Measures the amount of edge information
from the PAN image that is transferred into the fused
image. The high spatial resolution information missed in
the MS image is present in the high frequencies of the
PAN image. The fusion process injects the higher
frequencies from the PAN image into the multipectral
image [6]. A higher correlation between the two highpass
filtered images implies that the spatial information has
been retained faithfully. The two images were highpass
filtered using the Laplacian kernel which is described by
the following the equation:
(
(
(

=
1 1 1
1 8 1
1 1 1
HP
(4)
Let HP(P) be the high pass filtered PAN image and
HP(B) be the high pass filtered fused image. Then the
HPCC
k
is given by:
)) HP(B / Corr(HP(P) HPCC
K K
=
(5)
IV.Experimental Results and Data Discussion
To assess the quality of the fused images, the evaluation
metrics mentioned in the previous sections are employed
to evaluate the proposed IHS-LPT integrated fusion
technique and compare the results of different fusion
techniques such as the IHS, DWT, LPT , and integrated
IHS-DWT techniques.
It is known that the good fusion scheme should preserve
the spectral characteristics of the source MS image as well
as the high spatial resolution characteristics of the source
PAN image. The resulting fused images from the IHS,
DWT, LPT, the Integrated IHS-DWT and the proposed
integrated IHS-LPT fusion techniques are shown in Fig. 5
for IKONOS satellites. The values of spectral quality and
spatial quality metrics are listed in Table I.
It is clear that the IHS technique generally offers a
satisfactory spatial resolution but it tends to produce
distortion in the spectral characteristics of the original MS
images. The DWT approach preserve the spectral
characteristics of MS images compared to the IHS
technique, though images fused by wavelet-based methods
contain much less spatial information than images fused
by IHS methods. Considering the results of the proposed
method, it has lower values for the D
k
, RMSE, and greater
values for the CC
k
. Hence, the spectral quality of the
resulting images is much better than the spectral quality of
the images that are fused by the other techniques. On the
other side, the values of the HPCC
k
in the proposed
techniques are slightly lower than the corresponding
values in the IHS and LPT techniques, but this difference
is insignificant for the quality of the produced images
considering the gained improvements in the other
parameters for the spectral quality.
V.CONCLUSION
This paper introduces a new fusion method based on
integrating Laplacian pyramid transform and IHS
transform by fusing high spatial resolution PAN
IHS DWT LPT IHS+DWT LPT+IHS
R 0.6438 0.8468 0.9219 0.8784 0.9379
G 0.6038 0.8452 0.9208 0.8633 0.9331
C
C
k
B 0.6453 0.7962 0.9203 0.8906 0.9414
R 0.1865 0.1195 0.0842 0.1041 0.0717
G 0.1991 0.1199 0.0845 0.1107 0.0754
D
K
B 0.1819 0.1420 0.0865 0.1018 0.0705
R
0.2383 0.1523 0.1152 0.1363 0.1025
G 0.2527 0.1521 0.1154 0.1435 0.1058
S
p
e
c
t
r
a
l

q
u
a
l
i
t
y
R
M
S
E
K
B 0.231 0.1766 0.1179 0.1326 0.1005
R 0.9327 0.9417 0.9882 0.8944 0.9436
G 0.9482 0.9413 0.9881 0.9053 0.954
S
p
a
t
i
a
l

q
u
a
l
i
t
y
H
P
C
C
k
B
0.9263 0.9385 0.9856 0.8874 0.9398
Table I
IKONOS Image Fusion Results
images and low resolution MS images.
Different metrics are computed to assess the spectral
and spatial qualities of the images that are fused by IHS,
DWT, LPT, integrated IHS-DWT, and the proposed
fusion technique in this paper. The presented results show
that the new technique enhances both the spatial quality
and spectral quality of the fused images. The computed
metrics for the spectral quality of the fused images are
highly improved in the proposed technique compared to
all known fusion techniques, on the other side the spatial
quality is also improved and its computed metric is almost
equal in IHS, and the proposed technique.
It is concluded that the proposed fusion technique is
superior to all know fusion techniques specially IHS and
DWT techniques, where it can produce images with better
spectral quality than IHS and DWT and with the same
spatial quality characteristic as in these two techniques.
REFERENCES
[1] Yangrong Ling , Manfred Ehlers , E. Lynn Usery,
Marguerite Madden, FFT-enhanced IHS transform
method for fusing high-resolution satellite images,
ISPRS Journal of Photogrammetry & Remote
Sensing, vol. 61, pp. 381392, 2007.
[2] Choi M., A new intensity-hue-saturation fusion
approach to image Fusion with a tradeoff
parameter, IEEE transactions on geosciences and
remote sensing, vol.44, no. 6, pp.1391-1402, 2006.
[3] Yun Zhang, Gang Hong, An IHS and wavelet
integrated approach to improve pan-sharpening
visual quality of natural colour IKONOS and
QuickBird images, Information Fusion 6, pp. 225
234, 2005.
[4] Mara Gonzlez-Audcana, Jos Luis Saleta, Raquel
Garca Cataln, Rafael Garca, Fusion of MS and
PAN images using improved IHS and PCA merges
based on wavelet decomposition, IEEE transactions
on geosciences and remote sensing , vol.42, no. 6,
pp.1291-1299, 2004.
[5] Te-Ming Tu, Shun-Chi Su, Heuen-Chyun Shyu, Ping
S.Huang, A new look at IHS-like image fusion
methods, Information fusion 2, pp.177-186.
[6] Shutao Li, James T. Kwok, Yaonan Wang, Using
the discrete wavelet frame transform to merge
Landsat TM and SPOT PAN images, Information
Fusion 3, pp.1723, 2002.
[7] Peter J. Burt, Edward H., The Laplacian pyramid
transform as a compact image code, IEEE
transactions on communication, Vol. com-3l, no. 4,
1983.
[8] Pajares G., Jesus M., A wavelet-based image fusion
tutorial, Pattern Recognition 37, pp.18551872,
2005.
[9] R.C. Gonzalez and R.E. Woods, Digital Image
Processing, 2nd edition, prentice Hall, 2002.
[10] Pushkar pradhan, Multiresolution based,
Multisensor, MS Image Fusion, PhD thesis,
Mississippi State University, 2005.
[11] Rick S. Blum and Zheng Liu, Multi-sensor image
fusion and its applications, CRC Press Taylor &
Francis Group,2006.
[12] Ying-Ying Shih, Analysis of panchromatic and
Multi-spectrum image fusion, M.Sc thesis, Tatung
university, 2004.
[13] The IKONOS web site,
http://studio.gge.unb.ca/UNB/zoomview/examples.h
tml.
[14] A. Ardeshir Goshtasby, Stavri Nikolov, image
fusion :advances in the state of art, information
fusion 8, pp.114-118, 2007
[15] Bruno Aiazzi, Stefano Baronti, Andrea Garzelli,
Leonardo Santurri, Massimo Selva, spatial
enhancement of TIR aster data via VNIR images and
generalized Laplacian decomposition, Proceedings
of 4th EARSeL Workshop on Imaging Spectroscopy
,Warsaw 2005.
(a) (b) (c) (d) (e) (f) (g)
Fig. 5. IKONOS images. (a) original panchromatic image; (b) original multispectral image with R, G, B bands; (c) IHS fusion result; (d)
DWT fusion result; (e) LPT fusion result; (f) Integrated IHS-DWT fusion result; (g) Integrated IHS-LPT fusion results.