A Novel FPGA Implementation of Adaptive Color Image Enhancement based on HSV Color Space

M. C Hanumantharaju1, M. Ravishankar1, D. R Rameshbabu2, S. Ramachandran3

Department of Information Science & Engineering 2 Department of Computer Science & Engineering Dayananda Sagar College of Engineering, Bangalore, India - 560078 Department of Electronics & Communication Engineering SJB Institute of Technology, Bangalore, India - 560060 Email: {mchanumantharaju, ravishankarmcn, bobrammysore, ramachandr}@gmail.com
Abstract In this paper, a new algorithm and a novel architecture suitable for Field Programmable Gate Array (FPGA) implementation of adaptive color image enhancement based on Hue-Saturation-Value (HSV) color space is presented. The contrast and luminance of the color image is enhanced by using saturation feedback. The saturation component is enhanced by stretching its dynamic range to get rich color display. Hue is preserved in order to avoid color distortion. The adaptive luminance enhancement is achieved by using a simple geometric mean filter since traditional arithmetic mean filter tends to lose image detail such as edges and sharpness when compared to geometric mean filter. An efficient architecture for histogram equalization is also developed in order to evaluate the performance of proposed algorithm. The algorithm is implemented on Xilinx Vertex II XC2V20004ff896 FPGA device. The pipelining and parallel processing techniques have been adapted in order to speed up the enhancement process. The experimental results show that the color images enhanced by the proposed algorithm are clearer, vivid and efficient. Keywords: FPGA, Adaptive Color Image Enhancement, Saturation Feedback, HSV, Geometric Mean Filter.
3 1

color and intensity and hence these spaces reconstruct better images than the RGB space. Therefore, the color image should be transformed from RGB to other space. In the proposed work, HSV color space [4] is chosen since it offers good image enhancement. The algorithm proposed by Gang Song et al. [5] uses the arithmetic mean filter which smoothes local variations of luminance and saturation. The reconstructed quality of image using this scheme is generally not satisfactory. In the proposed method, geometric mean filter has been adopted which achieves very good quality reconstructed images. The algorithm is implemented on Xilinx Vertex II XC2V20004ff896 FPGA device. In this work, pipelining and parallel processing techniques [6] have been adapted in order to speed up the enhancement process. It is the first of its kind in the literature, which FPGA implementation of adaptive color image enhancement based on HSV color space has been proposed. This paper is organized as follows: Section 2 describes the proposed method. Section 3 gives brief details of hardware implementation. Section 4 provides experiment results and comparative study. Finally conclusion is presented in Section 5. II. PROPOSED METHOD This paper proposes a new adaptive color image enhancement based on geometric mean filter in order to overcome the drawbacks of the Gang Song et al. [5] algorithm. Geometric mean filter achieves smoothing comparable to the arithmetic mean filter. In addition, geometric mean filter tends to lose less image detail as compared to arithmetic mean filter. The restored pixel is given by the product of the pixels in the subimage window, raised to the power of (1/mn). The geometric means for luminance and saturation may be expressed by Eqns. (1) and (2):

I. INTRODUCTION Digital Image Enhancement [1] refers to accentuation, sharpening of image features such as edges, boundaries, or contrast to make a graphic display more useful for display and analysis. The color image enhancement can be classified into two categories according to the color space: (i) Color Image Enhancement in RGB color Space and (ii) Color Image Enhancement based on Transformed space The algorithms mainly used in RGB space are histogram equalization, intensity transformations and homomorphic filtering [2]. Although it is possible to enhance a color image by applying existing grey-level image enhancement algorithms to each red, green and blue channel, the resulting image may not be enhanced optimally. The algorithm is applied to individual channels without considering the correlation between R, G, and B component of the image. In transformed space the algorithms [3] transform images from RGB color space into other spaces such as HSV, YCbCr, YUV, Lab, etc. HSV color space discriminates between
___________________________________ 978-1-4244 -8679-3/11/$26.00 2011 IEEE

1 Vw = mn

( i , j )w

V (i, j )

1 mn

(1)

160

1 Sw = mn

( i , j )w

S (i, j )

1 mn

(2)
Fig. 1 Block diagram of the Proposed Adaptive Image Enhancement System.

where m = 3 and n = 3 for 33 window. The local variance for luminance and saturation are now expressed as: (3) and (4)

v 2 ( x, y ) =
s 2 ( x, y ) =

( i , j )w
( i , j )w

[V (i, j ) V w]2
[ S (i, j ) S w]2

(3) (4)

Further, the local correlation coefficient of luminance and saturation follows the Eqn. (5).

modules, four multiplier modules, four divider modules and two adders modules. In order to increase processing speed all modules are pipelined. RGB to HSV conversion needs a total of 18 clocks to complete the conversion process. The digital hardware for RGB to HSV conversion is developed using Eqns. (8) to (10). 43 G B 0+ , Max ( R , G , B ) = R Max ( R , G , B ) Min ( R , G , B )
H = 85 +

(x, y) =

(i, j)w

[V(x, y) Vw(x, y)][S(x, y) Sw(x, y)]

v2(x, y)s2(x, y)

(5)

43 B R , Max ( R , G , B ) = G Max ( R , G , B ) Min ( R , G , B ) 43 R G Max ( R , G , B ) Min ( R , G , B ) , Max ( R , G , B ) = B

The new luminance enhancement with saturation feedback is given by the Eqn. (6).

171 +

Venh( x, y ) = V ( x, y ) + k 1[V ( x, y ) V ( x, y )] k 2[ S ( x, y ) S ( x, y )] ( x, y )

(6)

S = 255

{Max( R, G, B) Min( R, G, B)}

Max( R, G, B)

(8) (9) (10)

The value of k1 and k2 was arrived at a value of 2 after conducting elaborate experiments. In order to improve the whole effect of color image with brighter and richer color besides the luminance enhancement [7], it is also required to enhance saturation. The saturation component is enhanced by stretching its dynamic range to get rich color display. The mathematical model for saturation component enhancement is given by Eqn. (7).

V = Max( R, G, B)

Fig. 2 Proposed Architecture for Adaptive Color Image Enhancement Note:

where S represents original saturation component and Senh is the enhanced saturation component. The gamma is the stretch coefficient which determines the degree of saturation enhancement. When the value of gamma is more, the image looks less saturated and, when the value of gamma is less the image looks more saturated. The value of gamma was arrived at a value of 0.77 after experiment analysis. In this work, hue is preserved in order to avoid color distortion. III. HARDWARE IMPLEMENTATION The Fig. 1 shows the block diagram for FPGA implementation adaptive color image enhancement system [8]. The proposed system has been realized using Register Transfer Level (RTL) compliant, Verilog Hardware Description Language (HDL) of about 7000 lines. In this work, the image size is chosen as 16001200 pixels. The first step in the proposed scheme is RGB to HSV [9] conversion. This module transforms pixels from RGB space to HSV. Hardware implementation of RGB to HSV includes three comparator modules, four subtractor

Senh = S

H = Hue; S = Saturation; V = Value (Intensity); D = Delay; W = Window; P = Power; SQRE = Square; SQRT = Square Root; HENH = Hue Enhanced; SENH = Saturation Enhanced; VENH = Value (Intensity) Enhanced The architecture for adaptive color image enhancement is as shown in Fig. 2. The architecture is developed using Eqns. (6) and (7) described in the previous section. The enhanced pixels are transformed back to RGB space using HSV to RGB converter [10] module. The HSV to RGB converter module takes 23 clock cycles to complete the conversion process. In order to speed up the complete system pipelining and parallel processing technique has been adapted.

161

Fig. 3 Proposed Architecture for Histogram Equalization

The Fig. 3 shows the hardware architecture for histogram equalization for single channel. This module performs contrast enhancement of an image. Histogram equalization module includes 256 comparators of each 8-bit, 256 counters of 16-bit, and a memory 256 words of each word of 8-bits followed by a combinational logic in order to perform mapping. In order to realize the digital hardware for histogram equalization we use Eqn. (11)

Fig. 4 Experimental Results of Man, Tower and Nature

Sk =

( L 1) MN

k j =0

nj

(11)

where Sk is the discrete transformation, L represents intensity levels in the image (L=256 for an 8-bit image), the product MN is the total number of pixels in the image. IV. EXPERIMENT RESULTS AND COMPARATIVE STUDY The proposed FPGA implementation of adaptive color image enhancement based on HSV color space has been coded and tested in Matlab (Version 8.1) first in order to ensure the correct working of the algorithm. Subsequently, the complete system has been coded in Verilog HDL so that it may be implemented on an FPGA or Application Specific Integrated Circuit (ASIC). The system simulation is done using ModelSim (Version SE 6.4) and synthesized using Xilinx ISE 9.2i. The algorithm is implemented on Xilinx Vertex II XC2V2000-4ff896 FPGA device. In the proposed work, window size is chosen as 33 since the image looks better and more colorful than the original image. The experiment was conducted and presented in Fig. (4) by considering three poor quality test images. The first column of fig. (4) shows three different original color images. The second column of fig. (4) shows the image enhanced by Gang Song et al. of Ref. [5] method. The third column of fig. (4) shows the images enhanced by conventional histogram equalization of R, G and B channels in RGB color space. The result shows that histogram equalized images suffer from color distortion. The last column of Fig. 4 shows the images enhanced by proposed luminance enhancement algorithm based on saturation feedback.

First Column: Original Image; Second Column: Image Enhanced based on the algorithm proposed in Ref. [5] using Matlab Software; Third Column: Histogram Equalized Image obtained from the architecture shown in Fig. (3); Fourth Column: Enhanced Images obtained from proposed hardware architecture shown in Fig. 2.
column of Fig. 4 shows the images enhanced by proposed luminance enhancement algorithm based on saturation feedback. Table 1 shows the FPGA device utilization summary of adaptive color image enhancement. The proposed method utilizes fewer resources in order to enhance a color image adaptively.
Table 1 Summary of the FPGA Device Utilization XC2V2000-4ff896

Logic Utilization No. of Slice Flip Flops No. of 4 input LUTs Logic Distribution No. of occupied Slices No. of Slices containing only related logic No. of Slices containing unrelated logic Total No. of 4 input LUTs Total Equivalent Gate Count for Design

Used 4096 1867 4092 4092 0 9547

Available 21504 21504 10752 4792 4792 21504 87629

Utilization 19% 8% 44% 100% 0% 44%

Experimental results show that the color image enhancement based on the proposed adaptive luminance and saturation feedback offers much better enhanced images than that of other methods. The reconstructed images offer richer color, clearer details and higher contrast.

162

In order to evaluate the performance of the proposed method, we present contrast enhancement performance, luminance enhancement performance and Peak Signal to Noise Ratio (PSNR). The contrast enhancement performance C, Luminance enhancement performance L and PSNR is evaluated using the following equations

C= L=

out in in
Iout Iin Iin

(12) (13)

(a)

(b)

PSNR = 10 log 10
p q

2552 MSE
2

(14)

The Mean Square Error (MSE) is given by

MSE =

( E ( x, y ) I ( x, y ) )
pq

(15)

(c)
Fig. 5 Performance Comparison of Proposed Method with Gang Song et al. in Ref. [5] and Histogram Equalization (a) Contrast Enhancement Performance (b) Luminance Enhancement Performance (c) Peak Signal to Noise Ratio (PSNR)

x =1 y =1

where out is variance of the luminance value of the output image, in is variance of the luminance value of the input image, Iout is the mean of the luminance value of the output image, Iin is the mean of the luminance value of the output image, E(x, y) is the enhanced gray pixel at position (x, y), I(x, y) is the original gray pixel at position (x, y) and, p and q denote the size of the gray image. The Fig. 5 shows the performance comparison. It is clear that, the contrast enhancement performance of the proposed method is good as compared to Gang Song et al in Ref. [5] and Histogram Equalization [HE]. The luminance of the histogram equalization has increased; hence the image looks over enhanced leading to color distortion. The proposed method has better PSNR as compared to other methods. V. CONCLUSION This paper presented the adaptive color image enhancement based on geometric mean filter using FPGAs. Geometric mean filter provides better visual effects as compared with arithmetic mean filter. The proposed scheme uses HSV color space since it discriminates between color and intensity. In addition, it offers good image enhancement. In this work, efficient hardware architectures for RGB to HSV, HSV to RGB, Adaptive image enhancement and histogram equalization has been developed. In order to increase the enhancement speed the techniques such as pipeline and parallel processing schemes has been adapted in Verilog coding. The Verilog code developed for the complete system is RTL compliant and works for ASIC design. The picture resolution of 16001200 pixels is chosen for this scheme. The implementation presented in this paper has been realized on an FPGA. The experimental results show that the enhanced images are better in visual quality compared to other methods. Currently research work is under progress for enhancing aerial and medical images.

REFERENCES
[1] Gonzalez R.C. Woods R.E Digital Image Processing Reading , MA: Addison Wesley, 1992. [2] Anil K. Jain, Fundamentals of Digital Image Processing, Englewood Cliffs, NJ: Prentice-Hall, 1989. [3] Thomas, B.A., Strickland, R.N., Heffrey .J, Color Image Enhancement using Spatially Adaptive Saturation Feedback, IEEE International Conference on Image Processing, vol. 3, pp. 30-33, 1997. [4] Gang Song, Xiang-Lei Qiao.: Adaptive Color Image Enhancement based on Human Visual Properties. In. Proceedings of International Congress on Image and Signal Processing, 2008 [5] Gang Song and Xiang-Lei Qiao, Color Image Enhancement based on Luminance and Saturation Components, Proceedings of International Congress on Image and Signal Processing, 2008. [6] Kentaro kokufuta and Tsutomu Maruyama, Real-time Processing of Contrast Limited Adaptive Histogram Equalization on FPGA, in. Proceedings of IEEE International Conference on Field Programmable Logic and Applications, pp. 155-158, 2010. [7] Kuen-Jan He, Chien-Chih Chen, Ching-His Lu and Lei Wang, Implementation of a New Contrast Enhancement Method for Video Images, pp. 1982-1987, 2010. [8] Xiaodong Xie, Zaifeng Shi, Wei Guo and Suying Yao, An Adaptive Image Enhancement Technique based on Image Characteristics, Proceedings of IEEE International Conference on Image and Signal Processing, pp. 1-5, 2009 [9] Ming Z. Zhang, Ming-Jung Seow, and Vijayan K. Asari, A Hardware Architecture for Color Image Enhancement Using a Machine Learning Approach with Adaptive Parameterization, Proceedings of IEEE International Joint Conference on Neural Network, pp. 35-40, 2006. [10] Hau T. Ngo, Ming Z. Zhang, Li Tao, and Vijayan K. Asari, Design of a Digital Architecture for Real-Time Video Enhancement Based on Illuminance-Reflectance Model, Proceedings of 49th IEEE International Midwest Symposium on Circuits and Systems, pp. 286290, 2006.

163