Top-down and Bottom-up Strategies in Lesion Detection of Background Diabetic Retinopathy

Xiaohui Zhang, Opas Chutatape School of Electrical & Electronic Engineering Nanyang Technological University, Singapore

Abstract
Bright lesions in the form of exudates and cotton wool spots while dark lesions consisting of hemorrhages are main evidences of background diabetic retinopathy that require early detection and precise classification. Based on different properties of bright lesions and dark lesions, bottom-up and topdown strategies are applied respectively to cope with the main difficulties in lesions detection such as inhomogeneous illumination. In bright lesion detection, a three-stage, bottom-up approach is applied. After local contrast enhancement preprocessing stage, twostep Improved Fuzzy C-Means is applied in Luv color space to segment candidate bright lesion areas. Finally, a hierarchical SVM classification structure is applied to classify bright non-lesion areas, exudates and cotton wool spots. In hemorrhage detection, a top-down strategy is adopted. The hemorrhages are located in the ROI firstly by calculating the evidence value of every pixel using SVM. Then their boundaries can be accurately segmented in the post-processing stage.

Figure 1. Lesions in color fundus image Previous work of bright lesions detection can be grouped into three main categories: thresholding, region growing and classification. Thresholding: Ward [2] and Phillips [3] applied thresholding method in detection of bright lesions. However, the former required the user to select the threshold manually based on the histogram while the latter required the manually selection of the ROI. Region growing: Sinthanayothin [4], Li [5] applied region growing segmentation method. However, the difficulty in selecting the seed point and the stopping criteria still remained. Classification: Wang [6] applied Bayesian statistical classifier by using color features as feature space. Osareh [7] used Fuzzy C-Means clustering to segment the candidate exudates areas. Then a neural network was applied to classify the exudates from non-exudate areas. However, FCM might be applied in RGB color space, but, in case of non-uniform illumination, the detection accuracy is not very desirable. Moreover, classification of cotton wool spots from exudates was not addressed. Previous work on hemorrhages detection are mainly based on bottom-up strategy. Most of them used local color information alone to segment the hemorrhages. Lee and Wang[8] used image normalization, global thresholding and matched filter to detect hemorrhages. However, due to the non-uniform illumination and interference of similar objects such as blood vessels, to

1. Introduction
Diabetic retinopathy has become a common eye disease in most developed countries. It occurs in 80% of all diabetic cases and is the leading cause of blindness [1]. Regular screening is the most efficient way of reducing the preventable eye damages. There are two kinds of lesions in the diabetic fundus images as shown in figure 1. One is bright lesion that includes exudates and cotton wool spots. The other is dark lesion consists of hemorrhages. Exudates are yellowwhite lesions with relatively distinct margins. They are lipid deposits within the body of the retina. Cotton wool spots are whitish patches that have no welldefined margins. They are not real exudates but degenerating nerve fibers. Hemorrhages have dot and blot configurations in the background diabetic retinopathy with their color similar to the blood vessels.

Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR05) 1063-6919/05 $20.00 2005 IEEE

detect the hemorrhages by using color information alone is not robust enough. Not only the classification accuracy but also the segmented boundaries of hemorrhages are not very desirable. Sinthanayothin[4] applied moat operator to enhance the dark area. Recursive region growing segmentation was then used to extract blood vessels and hemorrhages. Finally, neural network is applied to remove blood vessels by using three criteria as input features. Gardner et al. [9] used an artificial neural network to classify regions having the size of 2020 pixels into hemorrhage and non-hemorrhage areas. In conclusion, the main difficulties in the detection of lesions detection are the non-uniform illumination and interference of similar objects. Based on different properties of bright lesions and dark lesions, this paper describes bottom-up and top-down strategies that are applied respectively and distinctively to solve these difficulties.

deviation of the intensity within W be < f >W and W respectively. Also denote f max and f min as the maximum and minimum intensities of the whole image. Then the adaptive local contrast enhancement transformation is defined by: [W ( f ) W ( f min )] (1) g ( x, y ) = 255

[W ( f max ) W ( f min )]
1

where the sigmoidal function W ( f ) is

W ( f ) = 1 + exp f
W

f ( x, y )

(2)

2.2. Improved FCM segmentation stage in Luv color space

Fuzzy C-Means is an effective approach of segmenting color image. Unlike hard segmentation methods such as K-means that force pixels to belong exclusively to one class, the FCM allows pixels to be classified into multiple classes with varying degree of membership. The objective function of FCM [10] is:
J =
c N p uik x k vi 2

2. Bottom-up strategy in bright lesions detection

There are three types of objects in the retinal images: bright objects, dark objects and retinal background. Bright objects include white or yellow objects such as optic disk, exudates and cotton wool spots. Blood vessels, fovea, hemorrhages belong to dark objects. The color of the retinal background is between the bright objects and dark objects. The method proposed here contains three main stages: firstly, local contrast enhancement is applied as a preprocessing stage. Then, Improved Fuzzy C-Means is proposed and applied in Luv color space to segment all candidate bright-lesion areas. This segmentation stage is conservative and segments all possible bright lesions as well as false positives due to cluster overlapping, non-uniformity of color distribution and noises. The final stage is a hierarchical SVM classification stage that aims to distinguish true bright lesions from non-lesions, and more accurate classification between exudates and cotton wool spots.

(3)

i =1 k =1

where the array [u ik ] = U represents a fuzzy partition matrix. It can be used to show the results of classifying the data X to clusters by interpreting each element [u ik ] as a measure according to which data vector xk belongs to cluster i. {vi }c=1 are the prototypes of the i clusters, p > 1 is the weighting exponent which controls the fuzziness of the resulting clusters. However, FCM only considers the individual pixels. Therefore, it is sensitive to noise. Similar to the MFCM [11] proposed by Mohamed et al, a new improved FCM (IFCM) is proposed as follows:
JI =
c N p u ik x k vi 2

N p u ik Medianik

(4)

i =1 k =1

2.1. Local contrast enhancement preprocessing stage

Local contrast enhancement [4] depends on the mean and variance of the intensity within the local area. If the variation of the intensity in the local area is high, the algorithm does not significantly increase the local contrast. On the other hand, if the intensity variation in the local area is small, the local contrast is substantially increased. Consider a sub-image W of the size M M pixels centered on a pixel. Let the mean and standard

where

Medianik = Median

i =1 k =1

xr vi

xr N k

u ik [0,1]; 1 i c ;
c i =1

1 k N
N

u ik = 1 ; 1 k N ;

u ik > 0 ; 1 i c

k =1

N k stands for the set of pixels that exist in a window centered at xk The effect of the neighbor term is controlled by the parameter . IFCM use the median filter effect instead of average filter effect, thus it can keep the edge while reducing the noises.

Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR05) 1063-6919/05 $20.00 2005 IEEE

The objective function J I can be minimized in a way similar to the standard FCM. 1) Fuzzy partition matrix updating The constrained optimization in (4) can be solved using one Lagrange multiplier c N c (5) FI = ( uikp d ik + uikp i ) + ( 1 uik )
i =1 k =1 i =1

where dik = xk vi 2 , and i Taking the derivative of

FI p p = puik 1d ik + puik 1i uik

= Median

xr vi

.
xr N k

FI with respect to u ik and

=0
* uik = uik

setting the result to zero, we get (6)

regions near the image perimeter. So in this proposed method, IFCM is applied in two stages, first in L that represents luminance, then in u and v that represent chrominance. In L, three types of intensity clusters are classified. By using the local contrast enhancement, the intensities of bright lesions are within the brightest class. However, some bright background regions are also classified into the brightest class. Then in the second stage, two classes are classified using the chrominance components u and v to distinguish bright lesions and bright background. Thus by decomposing intensity and chrominance, the problem of nonuniform illumination was more effectively solved.

Solving for
* uik = c j =1

* u ik we have

1 d ik + i d jk + j
1 ( p 1)

(7)

2.3. Support vector machines classification stage

SVM is a statistical learning method based on structural risk minimization (SRM). It can map the input vector x into a high dimensional feature space by choosing a nonlinear mapping kernel. The optimal separating hyperplane in the feature space is given by [12]:
f ( x) = sgn
l i =1

2) Cluster prototype updating Taking the derivative of FI with respect to vi and setting the result to zero, we can obtain:
N k =1 p uik (xk vi ) + N k =1 p uik (xM vi ) vi = vi*

=0

(8)

yi i K ( xi , x) + b

(11)

vi* =

k =1

p uik (xk + xM ) N p uik k =1

2

where (9)

y i are the labels, K is the kernel

(1 + )

where

xM v = Median i

( (

xr vi

xr N k

Approximately, we use
xM vi
2

= Median

xr vi

xr N k

3) IFCM algorithm Step 1) Select initial class prototypes {vi }c=1 . i Step 2) Update the partition matrix using (7). Step 3) Update the prototypes of the clusters using (9). Repeat Step 2)3) till termination. The termination criterion is as follows: (10) Vnew Vold < where V = [v1 , v 2 ...v c ]T Although the retinal image consists of three types of objects, directly using FCM to segment the image into three classes in RGB color space usually cannot achieve the desired result. The main reason is the nonuniform illumination in retinal images. After investigating the Luv color space, it was found that the u component of Luv is good in classifying three kinds of objects in most regions of the image except the dark

function, b is the bias, i are the Lagrange multipliers. After the two-step IFCM clustering stage, the candidate bright lesion areas are segmented. Due to the influence of cluster overlapping, non-uniformity of color distribution and noises, some non-lesion areas among the candidate areas need to be classified. Moreover, the bright lesion areas that consist of exudates and cotton wool spots also need to be classified. There are several schemes of using binary SVM to deal with multi-class problem such as one versus the rest and one versus one. However, a twolevel SVM classification structure is applied in this paper. The first classification stage aims to classify bright lesions versus bright non-lesion areas while the second classification stage classifies exudates from cotton wool spots. The advantage is that unlike other schemes, user can access the results of each classifier for clinical consideration. Moreover, different subsets of suitable features can be used in different stages to achieve better result. In order to classify three kinds of candidate bright lesion areas: non-lesion areas, exudates and cotton wool spots, relevant features need to be selected properly. 1) Region edge strength

Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR05) 1063-6919/05 $20.00 2005 IEEE

f ( x, y ) =

f x

f y

(12)

2) Color difference between inside region area and surrounding area uinside (13) Cd =
usurrounding

By using above two features, non-lesion areas and lesion areas can be linearly classified. However, in order to further classify exudates and cotton wool spots correctly, the following features are added. 3) Region size 4) u and v of Luv color space All the color values such as u and v we use here are relative values. The standard reference color is the color of optic disk region. It should be indicated that the last two features are not suitable to describe the bright non-lesion areas. These bright non-lesion areas are wrongly segmented due to the cluster overlapping, noise and non-uniform illumination. Therefore, their region size and u and v have some uncertainty. Thus, the last two features are only used in the second classification stage.

The colors of fundus images differ widely in different racial and ethnic group. The variation is correlated to the skin pigmentation and iris color of different person. Histogram specification [13] is applied to normalize different colors of fundus images. It is applied independently to each individual RGB channel.

(a)

(b)

(c)

Figure 2. Example of color normalization (a) Reference image (b) Original image c) Image after color normalization Figure 2 is an example of color normalization.

3.2. Input feature vector

3. Top-down strategy in dark lesions detection

There are differences between hemorrhages detection and bright lesions detection. Firstly, the dark objects are more difficult to be distinguished from background in color space since these two classes have much overlapping. Secondly, the color of hemorrhages is similar to that of blood vessels. However, the segmentation of blood vessels itself is also a difficult task. On the other hand, the hemorrhages of background retinopathy have dot and blot configurations while the shapes of bright lesions are irregular. Based on these facts, a top-down strategy is adopted in hemorrhages detection as follows: After color normalization preprocessing stage, the hemorrhages are located in the ROI firstly by calculating the evidence value of every pixel using SVM. The local minimums of the evidence map under a certain threshold will be selected as centers of ROI. Then boundaries of hemorrhages can be accurately segmented in the post-processing stage. By segmenting the boundary of the hemorrhages in the specific small sized windows rather than in the whole image, the influence of the non-uniform illumination and interference of similar objects is reduced.

Unlike many commonly used SVM method, our approach does not attempt to extract any explicit image features such as color, area size, edge strength, shape, etc. Instead, we use finite image windows as input. We define the input pattern to the SVM classifier to be a small N N pixel window centered at the location of interest. The window is chosen large enough to contain a hemorrhage, but small enough to avoid potential interference from neighboring hemorrhages. By investigating the size of hemorrhages in background diabetic retinopathy, a suitable size of 15 15 pixel window is selected for most of hemorrhages. To facilitate the detection of various sizes of hemorrhages especially some large size hemorrhages, a pyramid of images is generated from the original image by gradually changing the resolution at each level. The hemorrhages are hypothesized at each level and then fused to the original scale. Figure 3 is a two-level image pyramid. In case of feature extraction, since the dimension of input vector is N N 3 in color fundus images which is usually too large based on the size of available training sample data, PCA is therefore applied to reduce the dimension of the input space. Thus, good generalization accuracy is possible for a moderate size sample data.

3.1. Color normalization

Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR05) 1063-6919/05 $20.00 2005 IEEE

mainly pick up the noise, in this sense, PCA or KPCA also performs a de-noise function. Since the number of hemorrhages absent samples are many more than the hemorrhages present samples. This will cause the unbalance problem. To overcome this problem, a small subset of representative hemorrhages absent samples is obtained by a bootstrapping approach [14].

Figure 3. Input image pyramid Given a training set of M patches of N N pixels around the manually-selected hemorrhages regions, eigenlesions can be computed by PCA.

3.4. SVM training hemorrhages

and

localization

of

Figure 4. First 10 eigenlesions Figure 4 shows the first 10 eigenlesions of the hemorrhages training set. In this paper, the first 20 principal components are used as input feature. Kernel PCA [12] is nonlinear extension of PCA. It computes the principal components in highdimensional feature space, therefore it can extract the nonlinear structure of the input space. In this paper, the comparison results of using PCA and KPCA as feature extractor are also given.

In this stage, the type of kernel function, the parameters of the kernel function and the regularization parameter C are determined by m-fold cross validation. By applying the trained SVM classifier in the testing image, the evidence map is then calculated. The local minimums of the evidence map under a certain threshold will be selected as centers of ROI. In real medical application, the threshold can be selected by different clinicians according to their different emphasis on TP(truth positive) and FP(false positive) in different situation. Then boundaries of hemorrhages can be accurately segmented in the post-processing stage by level set method. By segmenting the hemorrhages in ROI rather than in the whole image, the influence of the non-uniform illumination and interference of similar objects is reduced.

4. Experiments
4.1. Experiments on bottom-up bright lesions detection

3.3. Training data set sample selection

and bootstrapping

For each hemorrhages location in a training set image, a window of 15 15 pixels image patch centered at its center is extracted. They are samples of hemorrhages present class ( yi = +1). Similarly,

(a)

(b)

samples of hemorrhages absent class ( yi = 1) are collected randomly from all the hemorrhages absent locations. PCA and KPCA are applied in the hemorrhages present class. Eigenlesions are then calculated. By projecting of each input class onto the first 20 eigenlesions, the first 20 principal components are obtained as input feature vector. The dimension reduction of input feature space makes it possible to achieve good generalization accuracy for a moderate size sample data. Moreover, since the main structure in the data set is captured in the first k principal components, and the remaining components

Figure 5. Example of a result of local contrast enhancement. (a) Original image. (b) Result of local contrast enhancement with a window size of 3232.

(a)

(b)

Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR05) 1063-6919/05 $20.00 2005 IEEE

Figure 6. Comparison of the segmentation results. (a) Application of FCM in RGB color space. (b) Application of two-step IFCM in Luv color space.

Polynomial kernel, d=2, C=6, sensitivity=88%, specificity=84%, Ratio of SVs=24%.

4.2. Experiments on top-down dark lesions detection

In SVM classification stage, a training set is selected from 30 training images. It consists of 262 hemorrhages present and 800 hemorrhages absent selected from 10,000 hemorrhages absent samples by bootstrapping mentioned in section 3.3, a testing set consists 15 testing images.

(a)

(b)

Figure 7. Noise reduction effect of IFCM compared with FCM. (a) Application of FCM in Luv color space. (b) Application of two-step IFCM in Luv color space. Figure 5 shows the result of local contrast enhancement. Figure 6 (a) is the result of applying FCM in RGB color space. It is not satisfactory due to the non-uniform illumination. Figure 6 (b) demonstrates the greatly improved result of two-step IFCM applied in Luv color space. Figure 7 shows the effect of noise reduction of IFCM. The labeled area in figure 7 (a) is noise area. In SVM classification stage, the image data set consists of 30 images from SNEC (Singapore National Eye Center). A training set consists of 983 segmented bright non-lesion areas, 457 exudates and 54 cotton wool spots. A testing set consists of 432 bright nonlesion areas, 213 exudates and 47 cotton wool spots. The model selection including the type of kernel function and the regularization parameter C are carried out by 5-fold cross validation. Figure 8 shows the Receiver Operating Characteristics (ROC) curves of the following two SVM classifiers.

Figure 9. FROC curves comparison of PCA and KPCA as feature extractor used in SVM classifier

(a)

(b)

(c) Figure 8. ROC curves of two SVM classifiers Stage 1: Classification between bright lesions and bright non-lesion Linear kernel, C=10, sensitivity=97%, specificity=96%, Ratio of SVs=17%. Stage 2: Classification between exudates and cotton wool spots Figure 10. Example of hemorrhages detection. (a) Original image. (b) Evidence map calculated by SVM. (c) Localization of hemorrhages. Figure 9 shows the FROC curves of PCA and KPCA respectively. Gaussian kernel with = 1 is used in KPCA. Both PCA and KPCA use first 20 principal components. We can see KPCA achieves better result

Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR05) 1063-6919/05 $20.00 2005 IEEE

than PCA in SVM classifier. When number of FP remains 2 per image, KPCA can achieve 90.6% TP while PCA is 89.1%. Figure 10 is example of hemorrhages detection. Figure 10(a) and figure 10(b) are original testing image and evidence map calculated by SVM respectively. Figure 10(c) demonstrates the ROI of detected hemorrhages.

nonuniformly illuminated medical diagnostic image, Proceedings of Thirty-second Conference on Signal, Systems & Computers, Vol.2, pp.941-943, 1998. [9] G. G. Gardner, D. Keating, T. H. Williamson, A. T. Elliott, Automatic detection of diabetic retinopathy using an artificial neural network: a screen tool, British Journal of Ophthalmology, 80, pp.940-944, 1996. [10] J. C. Bezdek and S. K. Pal, Fuzzy models for pattern recognition, IEEE Press, 1991. [11] N. A. Mohamed, M. Y. Sameh, M. Nevin, A. F. Aly and M. Thomas, A modified Fuzzy C-Means algorithm for bias field estimation and segmentation of MRI data, IEEE Transactions on Medical Imaging, Vol. 21, No. 3, 2002. [12] M. N. Muller, S. Mika, G. Ratsch, K. Tsuda, and B. Scholkopf, An introduction to kernelbased learning algorithms, IEEE Trans. Neural Networks, vol.12, pp.181201, 2001. [13] R. Gonzalez and R. Woods. Digital Image Processing. Addison-Wesley Press, 1993. [14] E. Osuna, R. Freund, and F.Girosi, Training support vector machines: Application to face detecion, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 130-136, 1997.

5. Conclusion
The main difficulties of lesion detection are the non-uniform illumination and interference of similar objects. In this paper, it has been shown how different properties of the bright lesions and dark lesions, together with bottom-up and top-down strategies, are respectively and effectively applied to solve these problems.

6. References
[1] J. C. Javitt, L. P. Aiello, Y. Chiang, F. L. Ferris, J. K. Canner, S. Greenfield, Preventive eye care in people with diabetes is cost saving to the federal government, Diabetes Care, Vol. 17, No. 8, pp.909-917, 1994. [2] N. P. Ward, S. Tomlinson and C. J. Taylor, The detection and measurement of exudates associated with diabetic retinopathy, Ophthalmology, Vol. 96, pp. 80-86, 1989. [3] R. P. Phillips, J. Forrester, P. Sharp, Automated detection and quantification of retinal exudates, Graefes Archive for Clinical and Experimental Ophthalmology, Vol. 231, pp.90-94, 1993. [4] C. Sinthanayothin, Image analysis for automatic diagnosis of diabetic retinopathy, PhD thesis, Kings College London, 1999. [5] H. Li, A model based approach for automated feature extraction in color fundus images, PhD thesis, Nanyang Technological University, 2002. [6] H. Wang, W. Hsu, K.G. Goh, M. L. Lee, An effective approach to detect lesions in color retinal images, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, v2, pp181-186, 2000. [7] A. Osareh, M. Mirmehdi, B. Thomas and R. Markham, Automatic recognition of exudative maculopathy using Fuzzy C-Means clustering and neural networks, Proc. Medical Image Understanding and Analysis Conference, pp.49-52. BMVA Press, July 2001. [8] S. C. Lee and Y. Wang, A general algorithm for recognizing small, vague, and image-alike objects in a

Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR05) 1063-6919/05 $20.00 2005 IEEE