Tex 01

Graz University of Technology
Institute for Computer Graphics

and Vision
Diploma Thesis in Telematics
Digital Image Processing
Quantication of Posterior
Capsule Opacication
after Cataract Surgery
Hannes Siegl
Graz, March 2000
Advisors
ao. Univ. Prof. DI Dr. techn. Axel Pinz, Dr. med. Oliver Findl
Supervisor
ao. Univ. Prof. DI Dr. techn. Axel Pinz
University of Vienna
Department of Ophthalmology
Abstract
Cataract surgery has become a standard process in eye surgery. During this
operation the opaque lens is replaced by an articial one, the intraocular lens (IOL).
The most frequent postoperative complication associated with decreased vision is
the posterior capsule opacication (PCO) caused by lens epithelium cells (LEC)
growing between the capsule and the IOL. The PCO can be treated by YAG laser
capsulotomy which possibly, though infrequently, leads to new complications like
retinal detachment or intraocular pressure rise.
As many eorts are made to inhibit the LEC, it is necessary to nd an objective
way of PCO measurement. Inspired by a project by Spalton and Boyce the AKH
Vienna develops - in cooperation with the TU Graz - an optical measuring technique
which analyzes RGB images obtained from a slit lamp assembled by a CCD camera
as automatically as possible.
First, an approximate detection of the region of interest has been implemented
using KBVision whereby thresholding, morphology and ISR ltering are employed
to segment the pupil. As it is not all of the pupil which is the actual region of
interest, a ltered radial gradient method for circular region extraction, proposed by
Paplinski and Boyce, has been reimplemented and tested in MATLAB. Additionally,
a semi-automated approach to solve this problem is introduced.
At next, a possible semi-automated approach to compute reectionfree PCO
images is outlined. Besides, further automation steps are proposed.
Finally, some methods (segmentation by gray level co-occurrence, segmentation
by conjugate images, LOCOMM) to segment the region of interest have been in-
spected using KBVision and MATLAB and compared to an existing scoring system
(EPCO). LOCOMM seems to be the best method of these to automatically obtain
a segmentation which can be used as input for a grading system.
Zusammenfassung
Die Kataraktoperation ist mittlerweile Routine geworden. Im Zuge dieser Operation
wird dem Patienten die vom Star getr ubte Linse entfernt und durch eine k unstliche
ersetzt. Die haugste postoperative Komplikation ist der sogenannte Nachstar (po-
sterior capsule opacication, PCO), bei dem Hautzellen, die zwischen Linsensack
und Kunstlinse wachsen, die Sicht beeintrachtigen.
Da viele Bestrebungen unternommen werden dieses Zellwachstum einzudammen,
ware auch eine Methode zur objektiven Nachstarbeurteilung notwendig. Angeregt
durch ein Projekt von Spalton und Boyce entwickelt das AKH Wien in Zusam-
menarbeit mit der TU Graz ein optische Messsystem, das RGB-Bilder moglichst
automatisiert auswerten soll.
Diese Diplomarbeit hat drei Hauptzielsetzungen: Zum einen soll die Region
of Interest(ROI) automatisch gefunden werden, dann storende, bei der Aufnahme
entstandene Lichtreexe mittels Fusion entfernt werden und schlielich der Segmen-
tationsalgorithmus von Paplinski und Boyce reimplementiert und getestet werden.
Zwar wurde die Pupille wurde uber Schwellwerte, Morphologie und ISR-Filterung
erfolgreich detektiert, aber die Detektion der eigentlichen ROI konnte nicht vollst andig
automatisiert werden, da sich der von Paplinski und Boyce vorgeschlagene Algorith-
mus als zu wenig robust bzw. die Aufgabenstellung als ausserst komplex erwies.
Die semiautomatische Fusion wurde anhand eines Experiments erl autert und
mogliche Automatisierungsschritte skizziert.
Schlielich wurde das Segmentationverfahren von Paplinski und Boyce reim-
plementiert bzw. einem alteren Verfahren von Boyce und einem eigenen Ansatz
(LOCOMM) gegen ubergestellt. Es zeigte sich, dass LOCOMM die am besten zu
automatisierende Methode war.
Acknowledgements
First I would like to thank my supervisor Dr. Axel Pinz, my advisor Dr. Oliver
Findl and all the sta at the TU Graz and the AKH Vienna for the good cooperation.
Many thanks also go to my family for all the support during my time at university.
Last, but not least I have to thank Nadja for her endless love, patience und
understanding. You really rock !
Annotation
This master thesis was reformatted in March 2002 for PDFLatex.
Contents
1 Introduction 6
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 State of Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3 Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2 Explanation of Medical Terms 9
2.1 Cataract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Cataract Surgery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.3 Posterior Capsule Opacication . . . . . . . . . . . . . . . . . . . . . 10
2.4 YAG Laser Capsulotomy . . . . . . . . . . . . . . . . . . . . . . . . . 11
3 Data 13
3.1 Image Acquisition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.2 Description of the Image Data . . . . . . . . . . . . . . . . . . . . . . 14
3.3 Description of the Test Data . . . . . . . . . . . . . . . . . . . . . . . 17
4 Detecting the Region of Interest 20
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.2 Approximate Extraction of the Region of Interest . . . . . . . . . . . 20
4.2.1 Detection by Color . . . . . . . . . . . . . . . . . . . . . . . . 21
4.2.2 Morphological Operations . . . . . . . . . . . . . . . . . . . . 22
4.2.2.1 Dilation and Erosion . . . . . . . . . . . . . . . . . . 23
4.2.2.2 Opening and Closing . . . . . . . . . . . . . . . . . . 23
4.2.2.3 Experiments and Results . . . . . . . . . . . . . . . . 23
4.2.3 Intermediate Symbolic Representation Filtering . . . . . . . . 24
4.3 Rened Extraction of the Actual Region of Interest . . . . . . . . . . 25
4.3.1 Radial Sampling . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.3.2 Low Pass Filtering . . . . . . . . . . . . . . . . . . . . . . . . 27
4.3.3 Derivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.3.4 Elimination of Spurious Points . . . . . . . . . . . . . . . . . . 28
4.3.5 Experiments and Results . . . . . . . . . . . . . . . . . . . . . 29
4.3.6 Semi-automatic Detection of the ROI . . . . . . . . . . . . . . 29
1
5 Computing of a Reectionfree Image 30
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.2 Registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.3 Semi-automated Registration . . . . . . . . . . . . . . . . . . . . . . 32
5.4 Experiments and Results . . . . . . . . . . . . . . . . . . . . . . . . . 34
5.5 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
6 Segmentation 37
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
6.2 Statistical Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
6.2.1 First-order Statistics . . . . . . . . . . . . . . . . . . . . . . . 38
6.2.2 Second-order Statistics . . . . . . . . . . . . . . . . . . . . . . 39
6.3 The Gray Level Co-occurrence Matrix (COM) . . . . . . . . . . . . . 41
6.3.1 Building of a COM . . . . . . . . . . . . . . . . . . . . . . . . 41
6.3.2 Properties of the Gray Level Co-occurrence Matrix . . . . . . 41
6.3.3 Segmentation by COM . . . . . . . . . . . . . . . . . . . . . . 42
6.4 The Local COM Mean Estimate (LOCOMM) . . . . . . . . . . . . . 45
6.4.1 Why Deriving a Local COM . . . . . . . . . . . . . . . . . . . 45
6.4.2 The Mean Estimate . . . . . . . . . . . . . . . . . . . . . . . . 45
6.4.3 Segmentation by LOCOMM . . . . . . . . . . . . . . . . . . . 45
6.5 Conjugate Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
6.5.1 The Concept of Conjugate Images . . . . . . . . . . . . . . . . 48
6.5.2 Building of Conjugate Images . . . . . . . . . . . . . . . . . . 49
6.5.3 Segmentation by Conjugate Images . . . . . . . . . . . . . . . 52
6.6 Experiments and Results . . . . . . . . . . . . . . . . . . . . . . . . . 53
6.6.1 Ground Truth . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
6.6.2 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
6.6.3 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
6.6.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
7 Conclusion and Outlook 60
7.1 Detection of the Region of Interest . . . . . . . . . . . . . . . . . . . 60
7.2 Computing of a Reectionfree Image . . . . . . . . . . . . . . . . . . 60
7.3 Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
7.4 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
List of Figures
2.1 Cataract (Sketch) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2 Posterior Capsule Opacication (Sketch) . . . . . . . . . . . . . . . . 10
2.3 Types of PCO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.4 Eye after YAG Laser Capsulotomy (Sketch) . . . . . . . . . . . . . . 11
2.5 Eyes before and after YAG Laser Capsulotomy . . . . . . . . . . . . . 12
3.1 Acquisition System (Sketch) . . . . . . . . . . . . . . . . . . . . . . . 13
3.2 RGB Proles and Histograms of a Typical Image . . . . . . . . . . . 15
3.3 Red vs. Green Channel . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.4 Test Data: Patient A to F . . . . . . . . . . . . . . . . . . . . . . . . 18
3.5 Test Data: Patient G to L . . . . . . . . . . . . . . . . . . . . . . . . 19
4.1 Thresholding of IHS Channels (Patient C) . . . . . . . . . . . . . . . 22
4.2 Approximate Detection of the Region of Interest . . . . . . . . . . . . 25
4.3 Radial Sampling/ROI . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.4 Two Radial Signals, their Filtered Versions and their Derivatives . . . 27
4.5 Elimination of Spurious Points . . . . . . . . . . . . . . . . . . . . . . 28
4.6 Semi-automatic Detection of the ROI . . . . . . . . . . . . . . . . . . 29
5.1 Purkinje Spots (Sketch) . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.2 Dierent Directions of View . . . . . . . . . . . . . . . . . . . . . . . 31
5.3 Manual Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
5.4 Manual Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.5 Gray Level Adaption . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
6.1 Three Binary Images and their Co-occurrence Matrices . . . . . . . . 39
6.2 Two Binary Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
6.3 Three Images with Dierent Gaussian Noise (0, 5, 25) and their Gray
Level Co-occurrence Matrices ((x, y) = (1, 0), 256 gray values) . . . 43
6.4 Segmentation of a PCO Image Using Gray Level Co-occurrence . . . 44
6.5 Segmentation by LOCOMM . . . . . . . . . . . . . . . . . . . . . . . 46
6.6 LOCOMM Histograms . . . . . . . . . . . . . . . . . . . . . . . . . . 47
6.7 Segmentation by LOCOMM . . . . . . . . . . . . . . . . . . . . . . . 48
6.8 Conjugate Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
6.9 Co-occurrence Matrices of Conjugate Images . . . . . . . . . . . . . . 52
3
6.10 Segmentation by Conjugate Images . . . . . . . . . . . . . . . . . . . 53
6.11 EPCO Score . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
6.12 EPCO Ground Truth of the Test Data Set . . . . . . . . . . . . . . . 54
6.13 Segmentation (Patient A): Flashlight . . . . . . . . . . . . . . . . . . 56
6.14 Segmentation (Patient A): Automatic Segmentation . . . . . . . . . . 57
6.15 Segmentation (Excerpt Patient I): Giant Pearls . . . . . . . . . . . . 57
6.16 Clinician Gradings Mapped to EPCO . . . . . . . . . . . . . . . . . . 58
6.17 EPCO Ground Truth of the Test Data Set . . . . . . . . . . . . . . . 59
List of Tables
3.1 Data Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.2 The Test Data Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
6.1 EPCO Score . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
6.2 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
5
Chapter 1
Introduction
1.1 Motivation
Cataract surgery has become a standard process in eye surgery. Compared to the
formerly used intracapsular cataract extraction (ICCE), the extracapsular cataract
extraction (ECCE) decreases the risk of complications. During this operation the
opaque lens is replaced by an articial one, the intraocular lens (IOL).
The most frequent postoperative complication associated with decreased vision
is the posterior capsule opacication (PCO) caused by lens epithelium cells (LEC)
growing between the posterior capsule and the IOL. The PCO can be treated by YAG
laser capsulotomy which possibly, though infrequently, leads to new complications
such as retinal detachment, endophthalmitis or intraocular pressure rise. But these
new complications may even annul the accommodation that is attained by IOL
implantation [22].
Additionally, laser treatment poses a cost factor for the health care system which
should not be underestimated; e.g. Medicare spent over 250 million dollars for laser
capsulotomy in 1993 in the USA [2]. In fact, laser capsulotomy already is the
second frequently performed ophthalmic surgical procedure after cataract surgery
in the USA. These enormous costs explain why ECCE with IOL implantation is not
extensively performed in the Third World.
Therefore numerous eorts are made to ght PCO. Strategies to prevent PCO are
e.g. the surgical removal of LEC, toxic inhibition or the inhibition of LEC migration.
But LEC proliferation also depend on IOL material and design. It becomes clear
that a way of measuring LEC proliferation is necessary to assess the eciency of
these strategies.
In most clinical trials the laser capsulotomy rate has been used as the main
outcome measure [20, 32]. Unfortunately, the capsulotomy rate mostly depends on
the clinician who performs the assessment of the PCO. Other factors which aect
these studies are patient demands, nancial situation and access to laser equipment.
There are some attempts to objectify the assessments of the clinicians by com-
puter evaluation of areas marked by the clinicians. But these approaches like LOCS,
6
OPAC or GRID [36] in fact still depend on the assessment of the clinician and thus
are subjective.
Other studies have introduced a subjective slit-lamp scoring system [ 21, 23], the
employment of psychophysical data, such as best corrected visual acuity [ 17] or
contrast sensitivity. However, these methods either rely upon the clinician or the
general condition of the patient.
It becomes clear that only an automated quantication of PCO grants objective
assessment in prospective, randomized and double-blinded clinical trials.
1.2 State of Research
There are some approaches to nd an as far as possible objective and automated
way of PCO assessment.
Morphological scoring system of standardized retroillumination photographs
using a standard photo slit-lamp
Tetz [34] introduced this system which is already commercially available. Two
photos are obtained by the slit slightly decentered toward the temporal and
nasal rim of the IOL optics. Then the photos are scanned for computer analysis
where the amount of opacication in the area behind the optics is evaluated.
The density of opacication is graded by a clinician from 0 to 4. The total
PCO score is nally calculated by multiplication of this grading by the fraction
of capsule area graded to this opacication grading scores.
Because of the used photographic technique there are some problems resulting
from the inhomogeneous illumination of the region of interest. Consequently
the grading is dicult in certain portions of the IOL. But the major drawback
of this approach is the still subjective grading where it is possible that the
grading examiner is not free from bias towards e.g. a special IOL design,
because the shape of the optics and haptics are visible during the grading
process.
Measuring density of opacication in a Scheimpug image
Various studies deal with this approach [18, 13] which uses the correlation of
opacication density and visual acuity under conditions of PCO. Since the
method uses cross-sectional images of the IOL and the posterior capsule, no
morphological data of PCO is supplied. But in fact, it is the longitudinal
analysis of PCO morphology which is an intrinsic part of the assessment of
PCO to nd out the most eective ways of preventing PCO.
Employment of digital high resolution retroillumination imaging using coaxial
illumination
This approach seems to be the most promising one, because it oers morpho-
logical data. The main drawbacks of this system are the reections of the
7
illumination source on IOL and cornea resulting from the recording situation
[24] and the still not entirely objective grading, because, as far as it is known,
a clinician has to do some interaction to perform the segmentation of the
dedicated software analysis program [26].
1.3 Objective
The object of this thesis is to nd an automatic way to assess the images obtained by
a recording system similar to retroillumination imaging by Pande [ 24], as far as this
is possible. In fact, the approach and methods by Boyce and Paplinski [ 25, 27, 26]
should be re-implemented, tested and further automated. Especially their measures
to detect the region of interest with radial gradient ltering and to perform a textural
based segmentation are to be inspected more thoroughly. Additionally, an approach
to eliminate the Purkinje spots and to obtain a reexion-free image by fusion is
outlined in this thesis.
8
Chapter 2
Explanation of Medical Terms
2.1 Cataract
The lens consists mainly of water and protein. The protein chains are arranged to
be transparent and to focus the light on the retina. When some parts of the protein
clump together and start to cloud the lens, this is called a cataract (g. 2.1). There
are many types of cataracts. They can be classied as congenital, age related,
complicated or secondary and traumatic cataract according to their development.
Some typical symptoms of cataract are:
cloudy or distorted vision
reduced vision at night
changes in the appearance of colors
doubling or ghosting of vision
sensitivity to bright sunlight
In case of an early cataract, possible measures to improve vision are the use of
dierent eyeglasses, magnifying lenses or simply a stronger light source for reading.
If these steps do not help any longer or if some secondary problems like macula
degeneration or diabetic retinopathy occur, the opaque lens will have to be removed
surgically.
2.2 Cataract Surgery
There are two types of modern cataract surgery, which both go back to the 18th
century.
9
Vitreous
Posterior Capsule
Retina
Cornea
Iris
Anterior Capsule
Opaque Lens
Figure 2.1: Cataract (Sketch)
Anterior Rhexis
Cornea
Haptics
PCO
Posterior Capsule
IOL
Figure 2.2: Posterior Capsule Opacication (Sketch)
Extracapsular cataract extraction (ECCE) The front of the capsule is opened
and the lens is removed. Sometimes ultra sound waves are used to soften and
break up the lens. Then the lens is removed through a narrow tube. This
technique is called phacoemulsication or phaco.
Intracapsular cataract extraction (ICCE) Here, the entire capsule including
the lens is removed.
ECCE with the following implantation of an IOL is considered the better way,
because this technique reduces the risk of complications and restores vision. But in
some cases, for example if the lens is too hard for phaco, ICCE is still used.
2.3 Posterior Capsule Opacication
The most frequent postoperative (ECCE) complication associated with decreased
vision is the posterior capsule opacication (PCO) caused by lens epithelium cells
(LEC) growing between the posterior capsule and the IOL (g. 2.2).
10
(a) regeneratory PCO plate (thin layer
of LECs)
(b) Elschnig pearls (thick layer of LECs)
Figure 2.3: Types of PCO
2.4 YAG Laser Capsulotomy
PCO can be treated by YAG laser capsulotomy (g. 2.4 and g. 2.5 ). In the
course of capsulotomy, a hole is made in the posterior capsule to let light through.
This measure, possibly, though infrequently, leads to new complications like retinal
detachment, endophthalmitis or intraocular pressure rise. But these new complica-
tions may even annul the accommodation that is attained by IOL implantation [ 22].
That is the reason why a lot of research is done to nd another solution for this
problem, e.g. at the University of Rochester an IOL polishing technique is inspected
[9].
Anterior Rhexis
Posterior Rhexis
Haptics
IOL
Cornea
Figure 2.4: Eye after YAG Laser Capsulotomy (Sketch)
11
(a) diuse small pearl formation (b) eye (g. 2.5(a)) after Nd:YAG capsulo-
tomy
(c) capsule stress folds, peripheral wrinkling
of posterior capsule, regeneratory plate and
pearls
(d) eye (g. 2.5(c)) after Nd:YAG capsulo-
tomy
(e) regeneratory PCO plate invading from
edge which is no rhexis-IOL overlap, pearls
(f ) eye (g. 2.5(e)) after Nd:YAG capsulo-
tomy
Figure 2.5: Eyes before and after YAG Laser Capsulotomy
12
Chapter 3
Data
3.1 Image Acquisition
The Vienna University Department of Ophthalmology has set up a high-resolution
digital retroillumination imaging system inspired by a project of Dr. Spaltons group
at the St.Thomas Hospital in London [24]. The optical system (g. 3.1) actually
consists of an adapted slit-lamp:
The optics of a Zeiss 120 slit-lamp is used for observation and imaging.
Zeiss retrolux illumination module provides illumination by a Zeiss anterior
segment ashpack through a ber optic cable.
The Kodak NC2000 CCD has a geometric resolution of 1268 1012 pixels
and a radiometric resolution of 24 bit (RGB). Its main advantage compared
to the monochrome Kodak DCS200 used at St. Thomas is the higher light
sensitivity and consequently a better signal-to-noise ratio (SNR).
Beam splitters
Slit Lamp
Eye Operator
Illumination Module
CCD
45 Degree Mirrors
Figure 3.1: Acquisition System (Sketch)
13
However, the optical design of this system consists of a coaxial illumination and
imaging path. The light emerging from the illumination module is reected to the
patients eye by a beam splitter. Then these rays are refracted by the eyes optics
and reected back from the retina. As the retina is very well supplied with blood,
it seems as if a virtual red light source would retroilluminate the posterior capsule.
To operate the system, the patient is asked to place his/her chin on a chin rest
and xate on the illumination light. The operator focuses the recording optics on
the posterior capsule under direct vision. After the image acquisition, the image is
inspected on a PC and stored if satisfactory.
3.2 Description of the Image Data
After acquisition, the images are imported to the program Adobe Photoshop and
saved as tagged image le (see table 3.1).
Format TIF
Bit Depth 24 bit (RGB)
Height 1012 Pixels
Width 1268 Pixels
Resolution 16 m
File Size 3852360 Bytes
Table 3.1: Data Format
Because of the retroillumination the red color channel seems to contain much
more information for texture analysis than the blue and green channel (see g. 3.2)
which are quite noisy (blue channel) and less dynamic (blue and green channel).
But in fact, the employment of the red channel for texture analysis has two major
drawbacks caused by the recording situation, because the virtual red light source
leads to some negative eects whose inuence is greater on the red channel than on
the others (see g. 3.2):
The area aected by Purkinje spots (see chapter 4) is larger.
The retroillumination with an almost punctate light source is also aected by
the direction of view, i.e. the illumination is not completely regular. This
illumination artefact interferes above all with the red channel.
All in all, the green channel seems to be the best source of information for the
segmentation tasks described in chapter 6. The red channel is used, together with
the green channel, for the extraction of the region of interest (see chapter 4).
14
(a) red channel (b) green channel (c) blue channel
0 200 400 600 800 1000 1200 1400
0
50
100
150
200
250
(d) prole of the red channel
0 200 400 600 800 1000 1200 1400
0
50
100
150
200
250
(e) prole of the green channel
0 200 400 600 800 1000 1200 1400
0
50
100
150
200
250
(f ) prole of the blue channel
100 200
1000
2000
Clear Colors Exact Map
(g) histogram of the red chan-
nel
100 200
1000
2000
(h) histogram of the green
channel
100 200
1000
2000
(i) histogram of the blue chan-
nel
Figure 3.2: RGB Proles and Histograms of a Typical Image : The gray level
proles show a cross section (starting at the top right corner) of an image with two
Purkinje spots. The prole and the histogram of the red channel indicate that this
channel uses a wider range of gray levels than the other channels. The prole of the
blue channel suggests its higher amount of noise.
15
(a) red channel (b) green channel

0 100 200 300 400 500 600 700
100
150
200
250
(c) prole of the red channel

0 100 200 300 400 500 600 700
50
100
150
200
250
(d) prole of the green channel
(e) Purkinje spots (red channel) (f ) Purkinje spots (green channel)
Figure 3.3: Red vs. Green Channel : The gray level proles (g. 3.3(c) and
3.3(d)) show a cross section (from left to right) of an image with low PCO. The
illumination artefact aects the red channel (g. 3.3(a)) more than the green channel
(g. 3.3(b)). Besides, the area of Purkinje spots is larger on the red channel (g.
3.3(e)) than on the green channel (g. 3.3(f )).
16
3.3 Description of the Test Data
This selection of test images is supposed to give a short and incomplete survey of
the vast variety of possible input data to a PCO grading system (see g. 3.4 and
g. 3.5). All images are labeled with an ID to grant medical data protection.
ID IOL Haptics A B C D Mean Iris Lens Rhexis
A a 1 1 2 2 1.5
B b 4 5 5 5 4.75
C c 5 7 8 8 7
D d 1 2 3 2 2
E a 3 4 4 3 3.5
F d 2 3 2 2 2.25
G b 5 7 8 7 6.75
H a 8 8 9 9 8.5
I c 7 7 8 7 7.25
J a 0 0 0 0 0
K e 5 4 6 5 5
L d 6 6 8 7 6.75
Table 3.2: The Test Data Set
The test data set described in table 3.2 can be categorized by the employed
IOLs. In fact, these are images of ve common types of IOL: Acrylic 3-piece (a),
Silicone Plate haptic (b), PMMA 1-piece (c), Hydrogel 3-piece (d) and Silicone 3-
piece (e). One of the simplest ways to identify these dierent IOL products is looking
at their haptics and at their shapes. Thats why the existence or strictly speaking
the visibility of the haptics is also published in the table.
Another item in this table is a subjective PCO grading of the ROI by a team of
four clinicians (A, B, C, D) at the AKH Vienna. The value Mean is the mean of
the four gradings which goes from 0 to 10 where a completely clear lens is graded 0.
The boundary of the ROIs, which will be the input for the segmentation task, is
formed completely or partially by the iris, the IOL or the rhexis. This composition
is dealt with in the last three columns - iris, IOL and rhexis - of the table.
17
(a) patient a: low-grade regeneratory pco (b) patient b: wrinkling of posterior capsule,
extensive pearl formation
(c) patient c: thick regeneratory plate behind
iol, with small pearl formation
(d) patient d: diuse honeycomb pco with early
pearl formation
(e) patient e: regeneratory pco invading from
edge which is no rhexis-iol overlap, honeycomb
in the center
(f ) patient f: capsule stress folds, honeycomb
pco behind entire iol
Figure 3.4: Test Data: Patient A to F
18
(a) patient g: excessive pearl formation and
thick regeneratory plate
(b) patient h: excessive pearl formation, para-
central region with homogeneous regeneratory
plate
(c) patient i: giant pearls centrally, regenera-
tory plate throughout entire iol region
(d) patient j: anterior and posterior primary
capsulorhexis, no PCO
(e) patient k: capsule stress folds, pearl forma-
tion behind entire IOL optic region
(f ) patient l: relatively thick regeneratory plate
with some pearls
Figure 3.5: Test Data: Patient G to L
19
Chapter 4
Detecting the Region of Interest
4.1 Introduction
As only a part of the acquired image is interesting for further inspection, it is
necessary to dene a region of interest (ROI). This rst segmentation step consists
of two parts:
An operation to obtain the position of an approximate region of interest, the
pupil.
A more sophisticated method to extract the actually interesting part of the
posterior capsule.
Since this fully automatic approach is insuciently robust, the chapter closes
with a description of a more practical semi-automatic method.
4.2 Approximate Extraction of the Region of In-
terest
Boyce and Paplinski [27] introduced an approach with an active contour method.
This active contour, also known as a snake [35], is a very good contour extracting
method for noisy images, but its major disadvantage is that a good initialization of
the snake is required to grant a successful operation.
This initialization could also be done by the proposed extraction method. But,
as the snake just detects the edge between the iris and the pupil in a satisfying way
and not the actual region of interest, it does not seem to be necessary to perform
this step.
In fact, the interesting part of the ideal image has the following characteristics:
The area of the pupil is lighter than the surrounding iris.
The color of the pupil is red because of the reection of the retina.
20
The shape of the pupil is round.
Therefore, the region of interest is extracted by using color information. The
obtained binary image is treated with morphological operations to close holes and
delete small artifacts. Finally, labeling is performed and some ISR features are
computed to lter the remaining regions. Each step is described in detail in the
following sections.
This extraction task could also be performed by almost every approach of blob
detection like pyramid linking [29] or the use of scale space theory. Besides, the
Hough transformation [7, 29] could also be employed to detect the pupil because of
its shape.
4.2.1 Detection by Color
One of the most important segmentation approaches is thresholding [ 7]. However,
as thresholding a single color channel is not very robust, an multi-channel threshold
is employed for the initial segmentation. For this approach the color information
of the image is used instead of intensity. The color red characterizes the optics of
the lens because of the retroillumination technique and allows to dierentiate pupil
from iris. Other possible red objects are small blood vessels or the insides of the
eyelids.
Hence, an algorithm is needed to detect red areas. It is obvious that a transfor-
mation of RGB data to the IHS model - IHS stands for intensity, hue and saturation
- and thresholding the hue channel are a possible approach to this issue. The equa-
tions 4.1 4.4 describe a possible RGB-to-IHS transformation proposed in [ 19].
I =
_
1
3
(R + G + B) (4.1)
H =
_
G > B : cos
1
G < B : 360
o
cos
1
(4.2)
=
1
2
(2R GB)
_
(R G)
2
+ (R B)(GB)
(4.3)
S = 1
3 min(R, G, B)
R + B + G
(4.4)
In fact, each of the resulting channels could be employed for the thresholding
(see g. 4.1).
But for this detection task a simpler approach was used subsequently. As the
images are obtained in the RGB color model, it is obvious that the value in the red
channel has to be higher than in the other channels while the values of the green and
blue channel are approximately equal to give a human observer the impression of
21
(a) intensity (b) hue (c) saturation
(d) g.4.1(a) after threshold-
ing (T = 100)
(e) g.4.1(b) after threshold-
ing (T = 0..5, 240)
(f ) g. 4.1(c) after thresh-
olding (T = 130)
Figure 4.1: Thresholding of IHS Channels (Patient C)
red color. To simplify the calculation, only the red and green channel are observed.
This is possible because of the retroillumination technique (virtual red light source).
Let B be a binary image, T the threshold, R the red, G the green channel
of the original input image. Then the proposed algorithm for the multi-channel
thresholding is:
B =
_
(R G) T : 1
(R G) < T : 0
(4.5)
The threshold of T = 30 was determined experimentally.
Unfortunately, some parts of the region of interest (e.g. the edges of thick plates)
are not classied correctly by this approach. On the other hand, there are other
red objects in the image like blood vessels and the inner side of the lids which are
well supplied with blood. That is why a post-processing becomes necessary. This is
done by morphological operations.
4.2.2 Morphological Operations
Mathematical morphology [7, 29] is a powerful tool for the representation and de-
scription of region shape like boundaries, skeletons and the convex hull. Besides
there are many morphological techniques for pre- or post-processing such as open-
ing, closing, ltering, thinning and pruning.
22
Morphology is based on set theory. Two sets, A and B, are combined by one or
more operations such as translation, reection, complement or dierence. Set A is
usually referred to as the image element, while set B is referred to as the structuring
element.
4.2.2.1 Dilation and Erosion
The two basic morphological operations are dilation(AB) and erosion (AB). In
a binary image A, dilation expands regions of value 1, and erosion reduces regions
of value 1 by the structuring element B. These operations are dened [ 7] as:
A B = {x|(
B)
x
A = {}}. (4.6)
A B = {x|[(
B)
x
A] A}. (4.7)
Actually, these morphological operations correspond to a convolution of an image
A with a convolution mask B. Although dilation and erosion are based on set theory,
whereas convolution has an arithmetic background, the basic idea of replacing the
value of each pixel by a value which is calculated by the values of the neighboring
pixels dened by a window or mask, is the same.
4.2.2.2 Opening and Closing
Opening(A B) and closing(A B), two useful morphological operations, can be
built by combining dilation and erosion as follows:
A B = (A B) B. (4.8)
A B = (A B) B. (4.9)
Both, opening and closing, tend to smooth the contour of an image, but whereas
opening breaks narrow isthmuses and eliminates thin protrusions, closing fuses nar-
row breaks, eliminates small holes and lls gaps in the contour.
4.2.2.3 Experiments and Results
The binary images obtained by the multi-channel-thresholding are treated with the
KBVision-tasks MorphClose and MorphOpen (10 times opening, 10 times closing;
structuring element: 3x3 pixels). Small artifacts are erased. The contours of the
pupil and the eyelids are smoothed (see g.4.2). The gaps between dierent parts
of the pupil or an eyelid are closed. Problems only arise, when an eyelid is very
close to the pupil or even touches it. In fact, this should not happen, because these
images are rejected at the image acquisition.
23
4.2.3 Intermediate Symbolic Representation Filtering
As the prior steps may provide more than one possible region (pupil, eyelids, blood
vessels), it becomes necessary to perform additional steps for a correct segmentation.
As only binary data has to be dealt with at this point of the segmentation, it
seems necessary to gather some information about the remaining regions to nd out
which one is the pupil (see g. 4.2). For that reason, the regions are labeled and
converted into an Intermediate Symbolic Representation (ISR). The ISR [ 6] provides
possibilities to represent image events that are more abstract than pixels, such as
regions, edges or lines.
The data structure for representing these events is known as a token. In this
special case constellation tokens are employed. A constellation token is a represen-
tation for regions. It deals with properties such as location, bounding rectangle,
pixel count, shape of regions etc.
The criteria for the subsequent ltering are the size (pixel-count P) and the
shape of the region (compactness C).
C =
Area
Perimeter
2
(4.10)
The rst one is employed to eliminate small objects like blood vessels, the latter
to eliminate the eye lids. The ltering algorithm is quite simple. Regions are
accepted, if their pixel-count P 3 10
5
(approx. 25 % of the total area) and their
compactness C 0.3.
As KBVision [1] oers possibilities to deal with tokensets, the labeling, the com-
puting of constellation tokens and the ltering were done using this tool box.
24
(a) original image (b) after thresholding
(c) after morphological operations, 4 re-
gions
(d) image with ROI after ISR ltering
Figure 4.2: Approximate Detection of the Region of Interest
4.3 Rened Extraction of the Actual Region of
Interest
The boundary of the actual region of interest is formed by the anterior rhexis.
Sometimes this region of interest is overlapped by the iris, especially if the pupil could
not be widened enough by drugs. It may also occur that the IOL is decentralized
and the edge of the IOL is part of the boundary (g. 4.3). To sum up, it can be
said that the boundary of the region of interest if formed in parts or completely by
the following image objects:
Anterior Rhexis
IOL
25
(a) the basic idea (b) the boundary of the ROI is formed
by the anterior rhexis (dotted line) and
the IOL
Figure 4.3: Radial Sampling/ROI
Iris
Paplinski and Boyce [25] employed radial gradient lters to perform this extrac-
tion. This approach will be inspected more thoroughly. The basic tasks of this
approach are:
The image is scanned along radial lines from the estimated center of the ROI.
The radial intensity signal is determined for a number of angles. It becomes
clear that if the approximate location of the ROI is unknown, this approach will
fail and consequently, the employment of this method requires the approximate
detection step which was mentioned before.
The radial sampling signals are smoothed to remove noise while retaining
information about the image borders.
The derivative of the ltered samples is calculated to detect the edges and to
nd an inner and an outer contour.
Outliers of the contours are removed from the estimated contour by means of
a median lter.
The ltered contour points are used to interpolate a second order curve (an
ellipse) through them. This step makes the approach more robust, but leads to
a relatively unprecise estimation of the boundary, because the anterior rhexis
is actually shaped irregularly.
26
4.3.1 Radial Sampling
The radial sampling is very similar to a transformation of the image from Cartesian
to polar coordinates. By sampling the image from the estimated center radially n
times for each angle an n matrix is obtained.
4.3.2 Low Pass Filtering
Paplinski [25] suggested a 9-th order linear phase Parks-McClellan optimal equiripple
FIR lter (equ. 4.11, 4.12) for this operation (see g. 4.4).
H(z) =
1
1
a
n
z
n
(4.11)
a
1
. . . a
9
= [0.05, 0.042, 0.136, 0.234, 0.276, 0.234, 0.136, 0.042, 0.05] (4.12)
240 260 280 300 320 340 360 380 400 420 440
20
30
40
50
60
70
80
90
100
radius
in
t
e
n
s
it
y
(a) ltered and unltered signal ( =
93.6)
240 260 280 300 320 340 360 380 400 420 440
5
0
5
10
radius
in
t
e
n
s
it
y
(b) derivative of g. 4.5(a)
240 260 280 300 320 340 360 380 400 420 440
10
20
30
40
50
60
70
80
90
radius
in
t
e
n
s
it
y
(c) ltered and unltered signal ( =
187.2)
240 260 280 300 320 340 360 380 400 420 440
10
8
6
4
2
0
2
4
6
8
10
radius
(d) derivative of g. 4.4(c)
Figure 4.4: Two Radial Signals, their Filtered Versions and their Derivatives
27
4.3.3 Derivation
As a next step the ltered radial signals are dierentiated with respect to radius.
The maxima of the derived signals indicate edges. Paplinski says that it is possible
to verify that the inner (rhexis) and outer (IOL) contours coincide with the greatest
and second greatest values of the derivative (see g 4.4).
4.3.4 Elimination of Spurious Points
It is obvious that a biological contour is smooth to a certain extent, but from gure
4.5(d) it is apparent that the raw contour contain spurious points that should be
identied and ltered out. These spurious points will occur, e.g. if an edge of a pearl
or plate is falsely supposed to be the edge of the rhexis or iris, because these points
have relatively big values of the derivative of the radial samples n with respect to
. That is why they can be eliminated easily with a median lter (see g. 4.5).
0 50 100 150 200 250 300 350 400
280
300
320
340
360
380
400
420
440
ra
d
iu
s
degrees
(a) unltered inner and
outer contours
0 50 100 150 200 250 300 350 400
100
50
0
50
100
150
200
degrees
(b) contour and derivative
with respect to of inner
contour
0 50 100 150 200 250 300 350 400
280
300
320
340
360
380
400
(c) median ltered contour
(d) unltered contour lines (e) median ltered contour (f ) tted ellipse
Figure 4.5: Elimination of Spurious Points
Figure 4.5 : The algorithm fails to nd the correct contour, because in this case
the two dominating contours are the edge of the lens and of the iris.
28
4.3.5 Experiments and Results
The algorithm was implemented in MATLAB. In contrast to the assumption of
two separable contours, the test data provide a variety of dierent ROI contour
compositions.
This is the main reason why this approach fails most of the time. On the other
hand, the elimination of spurious points is not stable when the lens is very textured
because of the great number of possible edge candidates. The tting of an ellipse
proposed by Paplinski and Boyce [25] provides more stability (see g. 4.5), but in
fact, the algorithm is still not able to detect reliably the desired region of interest
(see g. 4.3). On the other hand these regions of interest are hard to detect for
human spectators, if they have no medical expert knowledge. These are the reasons
why a semi-automatic approach has been employed to support the clinicians.
4.3.6 Semi-automatic Detection of the ROI
The idea is quite straightforward. As the radial ltering approach fails because of
the vast number of edge candidates in an PCO image, an area containing the border
of the region is dened by drawing a circle or a freehand shape with a very thick line
(see g. 4.6). This manual segmentation step also makes the previously described
approximate detection obsolete. Radial sampling and derivation are performed only
in the dened area (see gs. 4.6(a) and 4.6(d)). The result is a polygon which can
easily be manipulated to correct errors (see g. 4.6). This method is used in clinical
work at the AKH Vienna.
(a) selection by draw-
ing a circle
(b) rened selection
of g. 4.6(a)
(c) contour of g.
4.6(b) after manual
manipulation
(d) selection by draw-
ing an approximate
contour
(e) rened selection
of g. 4.6(d)
(f ) contour of g.
4.6(e) after manual
manipulation
Figure 4.6: Semi-automatic Detection of the ROI
29
Chapter 5
Computing of a Reectionfree
Image
5.1 Introduction
The retroillumination recording technique used at the AKH has one major drawback.
The ashlight which illuminates the retina is reected on the transitions of the light
beam through the cornea and the lens. These reections are called Purkinje spots
(see g. 5.1). Additionally, there is a reection R of Purkinje 1, which is caused
by the recording optics (see g. 5.1). The position of this reection is constant in
relation to Purkinje 1. Unfortunately, these reections are situated in the areas most
interesting for examination. It becomes clear that this bias has to be eliminated.
P 2
P 4
P 3
P 1
IOL
Retina
Cornea
Figure 5.1: Purkinje Spots (Sketch): Purkinje 2 is usually not visible, because it
is directly situated behind Purkinje 1.
One possible approach to solve this problem is the recording of image series and
30
P3
R
P1
P4
(a) looking left
P3
R
P1
P4
(b) looking right
Figure 5.2: Dierent Directions of View: P1,..,P4 denote Purkinje spots. R indi-
cates the reection caused by optics.
fusion, since the Purkinje spots change their position when the patient is asked to
change his direction of view (see g. 5.1).
The following sections give a short overview over the required measures of such
a system.
Registration
nding of points of interest for matching
matching
transformation and resampling
Fusion
detecting and extracting of the Purkinje spots
replacing spots by gray level adaption in the reference image
5.2 Registration
Registration is the process of shifting, rotating, scaling and distorting to perform
a mapping of two images of similar geometry and content [ 10]. In other words,
after registration two corresponding points have the same image coordinates in the
images. Registration may be classied under the following heads:
Registration of Rigid Objects [38, 4]
The objects are rigid and their relative position is constant. These assumptions
allow various simplications to this problem, e.g. the employment of an ane
transformation which is fully dened by three matching points. In addition,
the result can usually be improved by the employment of markers.
31
Registration of Non-Rigid Objects [37]
Medical applications often deal with non-rigid objects, e.g. the female breast.
No universal model exists which fully describes non-rigid deformation. Con-
sequently, the number of required matching points is much higher; this also
complicates the use of markers.
Registration may not only be classied by means of image content, but also by
the used reference image:
Patient-Atlas Registration
The reference image is a standard image, the so-called atlas. As the atlas
represents the normal condition, the dierence between atlas and patient data
usually indicates pathological changes. The drawbacks of this approach are
the following: atlas and image do not have the identical content and normally
a non-rigid registration has to be employed.
Patient-Patient Registration
In most medical applications the reference image and the actual image stem
from the same patient; they are often captured using the same recording sys-
tem. A typical application is the recording of a time series to study the
morphology of pathological changes.
It is obvious that in our case we deal with a patient-patient registration. As
we may assume that the lens and the posterior capsule are rigid objects, if the
optical conditions are quite identical, the simpler rigid registration approach can be
employed. A possible approach to this issue is sketched in the following sections.
5.3 Semi-automated Registration
This experiment shows that the assumption of rigid objects holds true for our data,
if only parts of the image are registered.
First, a feature has to be selected for registration to perform the subsequent
matching. Two typical features for matching are edgels or points of interest. How-
ever, the choice of the feature depends mainly on the matching algorithm used later.
For this experiment the matching is performed manually and points of interest are
used (see g. 5.3).
To improve stability, four non-collinear corresponding points are selected for the
subsequent ane transformation (equ.5.1) and a least square error t is employed
to compute the set of parameters for the transformation.
_
u
v
_
=
_
A B
C D
_
_
x
y
_
+
_
tx
ty
_
(5.1)
32
The ane transformation maps one coordinate system (x, y) to another one
(u, v). It is obvious that equation 5.1 is fully determined by six parameters (three
pairs of corresponding points). If more than three points are used for the transforma-
tion, the system is overdetermined. Hence a least square error method is employed
to obtain the best t, i.e. to nd the values A, B, C, D, tx, ty, that minimize E
r1
and E
r2
(equ. 5.2 and equ. 5.3).
E
r1
=
n
i=1
(A x
i
+ B y
i
+ tx u
i
)
2
(5.2)
E
r2
=
n
i=1
(C x
i
+ D y
i
+ ty v
i
)
2
(5.3)
For E
r1
this minimization problem is solved by building the partial derivatives
with respect to A, B, tx (equ. 5.4 equ. 5.6). At next the partial derivations are
set equal zero. The three equations with three unknown elements can be written
in matrix form. The obtained matrix equation (equ. 5.7) is solved using Cramers
rule.
dE
r1
dA
=
n
i=1
2 A (A x
i
+ B y
i
+ tx u
i
) (5.4)
dE
r1
dB
=
n
i=1
2 B (A x
i
+ B y
i
+ tx u
i
) (5.5)
dE
r1
dtx
=
n
i=1
2 tx (A x
i
+ B y
i
+ tx u
i
) (5.6)
_
_
n
i=1
x
i
x
i
n
i=1
x
i
y
i
n
i=1
x
i
n
i=1
x
i
y
i
n
i=1
y
i
y
i
n
i=1
y
i
n
i=1
x
i
n
i=1
y
i
n
i=1
1
_
_
_
_
A
B
tx
_
_
=
_
_
n
i=1
u
i
x
i
n
i=1
u
i
y
i
n
i=1
u
i
_
_
(5.7)
The same algorithms are used to derive C, D, ty from E
r2
.
In fact, the ane transformation itself is never used for this kind of geometric
transformation, because this usually leads to some undened pixels in the output
image. Hence the value of a pixel of the output image is determined by applying
the inverse ane transformation (equ. 5.8) to its coordinates and by interpolating
the value at the obtained position in the input image.
_
x
y
_
=
1
AD BC
__
D B
C A
_
_
u
v
_
_
D B
C A
_
_
tx
ty
__
(5.8)
Common interpolation methods for this resampling are nearest neighbor, bilinear
interpolation and cubic convolution interpolation [ 15].
33
Nearest Neighbor
The value in the output image is the value in the input image next to the
calculated position
Bilinear Interpolation
The value in the output image is a weighted average of the pixels in a 2 by 2
neighborhood around the calculated position in the input image.
Cubic Convolution Interpolation
The value in the output image is a weighted average of the pixels in a 4 by 4
neighborhood around the calculated position in the input image.
5.4 Experiments and Results
The aim of this process is to compute a reectionfree image. Hence a main image
where the reections have to be replaced by registered image parts has to be se-
lected. As was stated before, the matching is performed manually. Unfortunately
the registration of entire images does not perform well, so only small regions around
the reections to be replaced are registered. Consequently the points of interest are
also selected around the reections (see g. 5.3). The registration is computed using
the algorithms proposed in the previous section (cubic convolution interpolation).
(a) corresponding points
(looking left)
(b) corresponding points
(looking right)
(c) image g. 5.3(a) mapped
to image g. 5.3(b)
Figure 5.3: Manual Matching
Finally, the reections in the main image are replaced by the registered image
parts. At the moment, this fusion process is also performed manually by a simple
cut and paste and by a subsequent manual gray level adaption (see g. 5.4).
5.5 Outlook
There are several problems to solve on the way to an automated system:
34
(a) looking left (support) (b) looking right (main (c) fusion of image g. 5.4(a)
and image g. 5.4(b)
Figure 5.4: Manual Fusion
Detection of the Reections
The reections and consequently the areas to be replaced can be detected by
a simple thresholding and a subsequent morphological opening.
Detection of the Points of Interest
There are a number of known POI operators, such as the Moravec [ 16] or
Foerstner [12] operator, which are measures for successful matching.
Matching
It is possible to use a template matcher, because rotation does not have to be
be dealt with. There are also many methods proposed in literature [ 11] for the
improvement of the accuracy of such an algorithm.
Gray Level Adaption
The main problem of this issue is the reduction of the eect of the ashlight
artefact (chapter 3). An adaption of the gray values of an inserted image part
leads to additional edges in the fusion image when a constant gray level is
added, because in addition to the dierence between the average gray levels
of the images there is also a directional dependency of the ashlight artefact
caused by the dierent directions of view (see g. 5.5). This could be solved
by computing average gray levels for small regions at the border of the image
parts, calculating the dierence of source and destination image and interpo-
lating the change of gray level between the borders.
35
(a) original image (b) image with inserted registered region
(c) manual gray level adaption (inten-
sity +21)
(d) manual gray level adaption (inten-
sity +15)
Figure 5.5: Gray Level Adaption: It is impossible to perform a successful linear
gray level adaption because of the directional dependency of the illumination tech-
nique.
36
Chapter 6
Segmentation
6.1 Introduction
Segmentation can be dened [7, 29] as a process that partitions an image I into n
subregions, R
1
, R
2
, . . . , R
n
by an homogeneity criterion H , such that
i : R
i
I, (6.1)
_
i
R
i
= I, (6.2)
i : R
i
is a connected region, (6.3)
R
i
R
j
= {} i, j : i = j, (6.4)
i : H(R
i
) = True, (6.5)
H(R
i
R
j
) = False if R
i
is connected to R
j
. (6.6)
The conditions 6.1 and 6.2 grant that every pixel must be in a region. According
to condition 6.3, the pixels of one region have to be connected e.g. in the sense of
4- or 8-connectivity [7]. Condition 6.4 requires that the regions are disjoint, while
condition 6.5 and 6.6 deals with the homogeneity criterion that has to be satised
by the pixels of a segmented region.
To sum up, segmentation is the process of splitting images into regions which are
homogeneous with respect to features like uniform gray level or texture. Another
approach to this issue is the generation of edge strength images to split the image
at prominent edges. The problem of nding the dividing line between two regions of
dierent texture is even more dicult than the classication of regions of unknown
texture [31]. Texture seems to be a collection of the picture primitives which are
distributed over an image. The statistics of the gray levels of the pixels are a way
37
to describe texture, if the images are at least to some extent formed by a random
(Markov) process.
In this chapter, statistical features for texture analysis are discussed, and three
statistical methods for area segmentation are introduced, namely segmentation by
gray level co-occurrence by Haddon and Boyce [8], by a mean operator derived from
local co-occurrence matrices and by the employment of the concept of conjugate
images, which has been developed by Paplinski and Boyce [ 26].
6.2 Statistical Features
6.2.1 First-order Statistics
The rst-order statistics like mean, variance, skewness and kurtosis only describe
the shape of the frequency distribution of gray levels (see equ. 6.7 6.11).
Mean is a feature that locates the distribution. i(p) denotes the gray level
of a pixel. N is the pixelcount of the image.
=
1
N
p
i(p) (6.7)
Variance
2
is a feature for the average deviation from the mean .
2
=
1
N
p
(i(p) )
2
(6.8)
Skewness S is a feature for the tiltedness of the distribution.
S =
1
N
p
(i(p) )
3
3
(6.9)
Kurtosis K is a feature for the sharpness of the distribution.
K =
1
N
p
(i(p) )
4
4
(6.10)
Especially the variance is of particular importance for texture description, be-
cause it is a measure of gray-level contrast that can be used to describe relative
smoothness. For example, the feature
R = 1
1
1 +
2
(6.11)
is 0 for areas of constant intensity and approaches 1 for large values of the
variance [7].
38
6.2.2 Second-order Statistics
The weakness of rst-order statistics is that they contain no information about the
relative position of the pixels with respect to each other, e.g. it is not possible
to discriminate the following binary images (see g. 6.1) by means of rst-order
statistics features, e.g. = 0.5 for all three images while their co-occurrence matri-
ces ((x, y) = (1, 0), border pixels ignored) a second-order statistics method, are
dierent.
1 1 0 0
1 1 0 0
1 1 0 0
1 1 0 0
1 1 0 0
1 1 0 0
0 0 1 1
0 0 1 1
1 0 1 0
0 1 0 1
1 0 1 0
0 1 0 1
_
4 4
0 4
_ _
4 2
2 4
_ _
0 6
6 0
_
Figure 6.1: Three Binary Images and their Co-occurrence Matrices
Second-order statistics deal with the probability that two pixels p, p
connected
by a position operator d, e.g. the relative position x, y of the two pixels, have the
gray levels i and j. This can be expressed by the function
P(d, i(p), j(p
)). (6.12)
.
In most cases texture can be discriminated best when x = y = 1 [30]. The
co-occurrence matrix can be used as an estimate of this probability function, e.g.
when the two pixels are adjacent and related by one of four basic directions
(x, y) = (1, 0), (1, 1), (0, 1), (1, 1)
as Chen and Pavlidis [3] mentioned. Although it is possible to employ the co-
occurrence matrix as a stand alone discriminator, there are also some prominent
features (see equ. 6.13 6.17) derived from the co-occurrence matrix e.g. :
maximum probability P
max
which is an indicator of the strongest response to
the position operator,
P
max
= max P
i,j
(6.13)
element dierence probability P
k
of order k, which is low when the high values
39
of the COM are near the main diagonal,
P
k
=
i,j
(i j)
k
P
i,j
(6.14)
the inverse element dierence probability P
k
of order k, which describes the
opposite eect,
P
k
=
i=j
1
(i j)
k
P
i,j
(6.15)
the entropy H, which describes the randomness,
H =
i,j
P
i,j
log P
i,j
(6.16)
the energy E gives an indication of the uniformity.
E =
i,j
P
2
i,j
(6.17)
Other features are contrast, homogeneity, inertia, cluster tendency, cluster promi-
nence or probability of a given run length [10, 5].
A remarkable fact is that statistics of lower order can be derived from statistics
of higher order, i.e. rst-order statistics are embedded in second-order statistics,
e.g. the gray level density distribution or normalized histogram can be derived from
the co-occurrence matrix by simply forming the sums over the columns. Hence it
is not possible to nd images with dierent rst-order and identical second-order
statistics.
1 1 0 0
1 1 0 0
1 1 0 0
1 1 0 0
1 0 1 0
0 1 0 1
1 0 1 0
0 1 0 1
Figure 6.2: Two Binary Images
It is obvious that these images (g. 6.2) have identical rst-order statistics but
dierent second-order statistics. Hence second order-statistics are more discriminat-
ing for texture analysis than mere rst-order statistics. It seems to be unessential
to use statistics of higher order than of order 2, because it has been demonstrated
that it is very hard for humans to discriminate images with identical statistics of
order one and two but dierent higher order [14].
40
6.3 The Gray Level Co-occurrence Matrix (COM)
6.3.1 Building of a COM
As mentioned before, the COM is a second-order statistics. A simple example to
illustrate the idea of the gray level COM: Let I be an image consisting of pixels p
x,y
,
e.g. :
I =
0 0 0 0 1 1
1 1 0 0 1 1
1 1 0 0 0 7
0 0 0 5 7 7
2 1 3 7 7 7
1 2 4 4 1 1
The position operator d is dened by (x, y) = (1, 0). The gray level of a pixel is
i(p
x,y
). The gray level of the neighborhood-pixel is i(d(p
x,y
)). The side length of the
COM S is given by the number of gray levels. Hence the column index corresponds
to the gray level i(p
x,y
), the row index to gray level i(d(p
x,y
)).
The algorithm for the calculation of the gray level COM is:
p
x,y
I : S
i(p
x,y
),i(d(p
x,y
))
= S
i(p
x,y
),i(d(p
x,y
))
+ 1 (6.18)
So we obtain the following matrix:
S =
_
_
_
_
_
_
_
_
_
_
_
_
8 2 0 0 0 0 0 0
2 5 1 0 1 0 0 0
0 1 0 0 0 0 0 0
0 1 0 0 0 0 0 0
0 0 1 0 1 0 0 0
1 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
1 0 0 1 0 1 0 3
_
_
_
_
_
_
_
_
_
_
_
_
If S is normalized by the number of entries, we obtain the joint intensity proba-
bility [8]. In literature the joint intensity probability itself often is called gray level
co-occurrence. This also holds true for the rest of this thesis.
6.3.2 Properties of the Gray Level Co-occurrence Matrix
It is possible to express the COM in terms of the conditional probability for a
pair of pixels to have observed intensities given when their region membership is
known. Hence it can be said that for images composed of regions with intensity
distributions characterized by Gaussian statistics the COM may be decomposed as
a sum of normal distributions [8]. According to this approach, these regions are
41
represented by prominent maxima in the main diagonal of the COM (see g. 6.3.2);
in other words: the main diagonal carries the information about the specic regions
while the boundary information is shifted to the o-diagonal regions.
The COM is a square matrix with a side length corresponding to the number of
gray levels. As each gray level usually is not eciently employed to give a description
of an image [3] , a number of investigators used histogram equalized images with a
reduced number of gray levels. Quite often the reduction is from 256 levels to 8, e.g.
in [30].
6.3.3 Segmentation by COM
There are many approaches to this issue. Chen and Pavlidis [ 3] used a split-and-
merge algorithm based on the threshold of dierence between COMs with dierent
position operators. Conners, Trivedi and Harlow [ 5] developed a split-and-merge
technique combined with a classication task to segment and classify remote sensing
images. They derived six features from the COM to build a uniformity operator.
Haddon and Boyce [8] took advantage of the properties of the COM described above
to segment infrared images. A nearly identical process was employed by Paplinski
and Boyce [26] to obtain a segmentation of PCO images, but they calculated their
COMs with conjugate images obtained by special ltering.
The basic idea of this approach is that the maxima of the main diagonal of a
COM are the centers of the Gaussian distributions which characterizes the dierent
uniformly textured regions of the image. Then the COM itself is divided into regions
which stand either for proper regions or for boundary regions. The regions are
mapped to the original image. After that some relaxation steps are made, e.g. by
using directional information at boundary regions. For full details the reader is
referred to [8]. The main problem using this approach is the stable nding of the
maxima and correct detecting of the main diagonal regions (see g. 6.3.3).
42
(a) image (b) noisy image ( = 5) (c) noisy image ( = 25)
(d) co-occurence matrix of g-
ure 6.3(a)
(e) co-occurence matrix of g-
ure 6.3(b)
(f ) co-occurence matrix of g-
ure 6.3(c)
0 50 100 150 200 250 300 350
0
10000
20000
30000
40000
50000
60000
70000
80000
90000
(g) main diagonal of gure
6.3(d)
0 50 100 150 200 250 300 350
0
500
1000
1500
2000
2500
(h) main diagonal of gure
6.3(e)
0 50 100 150 200 250 300 350
0
10
20
30
40
50
60
(i) main diagonal of gure
6.3(f )
Figure 6.3: Three Images with Dierent Gaussian Noise (0, 5, 25) and their Gray
Level Co-occurrence Matrices ((x, y) = (1, 0), 256 gray values): The image in g.
6.3(a) consists of 5 regions (four rectangles plus background). Hence there are ve
peaks in the main diagonal of the co-occurrence matrix (g. 6.3(g)) corresponding
to these regions. The o-diagonal values in g. 6.3(d) correspond to the region
boundaries. The remaining gures show the eect on the co-occurence matrix caused
by noise.
43
(a) PCO image (patient B, manually
segmented ROI)
100 200
1000
2000
(b) histogram of g. 6.4(a)
(c) gray level co-occurrence matrix of g.
6.4(a)
0 50 100 150 200 250 300 350
0
500
1000
1500
2000
2500
3000
(d) COM prole (main diagonal) of g.
6.4(a)
(e) segmented image (well manually se-
lected thresholds)
(f ) segmented image (badly automati-
cally selected thresholds)
Figure 6.4: Segmentation of a PCO Image Using Gray Level Co-occurrence: The
main problem of this segmentation approach is that in this medical image data the
gray levels are very often approximately normal-distributed. Hence a sensible detec-
tion of thresholds can not be performed automatically in most cases.
44
6.4 The Local COM Mean Estimate (LOCOMM)
6.4.1 Why Deriving a Local COM
The previously discussed second-order features do not make much sense if they are
employed on an entire image of, say, 600 600 pixels. The use itself of a global COM
also leads to a bottleneck of information, because e.g. a 1000 1000 pixels image
is mapped to a 256 256 matrix feature space which is in fact quite sparse. The
derived features are not able to describe the entire image. It seems more sensible to
compute the features only for a small neighborhood. On the other hand we should
not calculate the features of an entire 256 256 COM with the small number of
entries for just a 5 5 window. That is why the reduction of the gray levels of the
COM, or the image respectively, is considered. I simply used a bit-plane slicing for
the reduction. The best results were achieved by using a reduction from 256 to 16
or 8 gray levels, because a further reduction increases the loss of information while
a lesser reduction leads to an unacceptable run-time behavior.
6.4.2 The Mean Estimate
First-order statistics like the mean are embedded in second-order statistics (equ.
6.19), as it was mentioned before [10]. The mean can be calculated from the sum
over the weighted rows or columns of the COM.
i
=
1
N
i,j
iP
i,j

j
=
1
N
i,j
jP
i,j
(6.19)
In fact, this is not exactly the mean of the window obtained by rst-order statis-
tics as a result of the gray level reduction. Even if it was calculated from a 256 256
matrix, there would be a dierence because of the neighborhood function. For the
mean estimate the average of the row and column mean is calculated.
=

i
+
j
2
(6.20)
6.4.3 Segmentation by LOCOMM
The LOCOMM behaves like an averaging lter where, in the gray level density func-
tion, the gray levels accumulate near special gray levels g
i
which is an eect caused
by the reduction of gray levels, because the probability that neighbors have the same
(reduced) gray level is very high, especially in a small (e.g. 3 3) neighborhood (see
g. 6.6).
g
0
= 0 g
i+1
= g
i
+
256
#graylevels
(6.21)
45
This leads to very prominent peaks in the histogram which are easier to detect
robustly than the peaks in the main diagonal of the COM. This step is followed by
segmentation by thresholding (see g. 6.5).
The main disadvantage in comparison to the two other introduced methods is
the additional quantization error because of gray level reduction. When the number
of gray levels is raised to minimize this bias, the peaks are not that prominent any
longer when a constant neighborhood size is used. An additional enlargement of the
inspected neighborhood leads to better results (see g. 6.6), but at the same time
to an unacceptable run-time behavior, as mentioned before.
(a) PCO image
100 200
2000
4000
6000
8000
10000
12000
14000
(b) LOCOMM histogram (c) LOCOMM image
Figure 6.5: Segmentation by LOCOMM
46
100 200
1000
2000
3000
4000
5000
6000
(a) 33 neighbor-
hood, 8 gray levels
100 200
1000
2000
3000
4000
5000
6000
(b) 33 neighbor-
hood, 16 gray levels
100 200
200
400
600
800
1000
(c) 33 neighbor-
100 200
200
400
600
800
1000
1200
1400
(d) 33 neighbor-
100 200
1000
2000
3000
4000
5000
(e) 55 neighbor-
hood, 8 gray levels
100 200
1000
2000
3000
4000
5000
(f ) 55 neighbor-
100 200
200
400
600
800
1000
(g) 55 neighbor-
100 200
200
400
600
800
1000
1200
1400
1600
(h) 55 neighbor-
100 200
1000
2000
3000
4000
(i) 77 neighbor-
hood, 8 gray levels
100 200
1000
2000
3000
4000
(j) 77 neighbor-
100 200
200
400
600
800
1000
1200
(k) 77 neighbor-
100 200
200
400
600
800
1000
1200
1400
1600
(l) 77 neighbor-
100 200
1000
2000
3000
(m) 99 neighbor-
hood, 8 gray levels
100 200
1000
2000
3000
4000
(n) 99 neighbor-
100 200
200
400
600
800
1000
1200
(o) 99 neighbor-
100 200
200
400
600
800
1000
1200
1400
1600
1800
(p) 99 neighbor-
Figure 6.6: LOCOMM Histograms: This gure shows LOCOMM histograms of
gure 6.7(a) obtained by using dierent numbers of gray levels and dierent neigh-
borhoods. A small number of gray levels combined with a small neighborhood leads
to very prominent peaks, as the pixels in a small neighborhood are likely to have
the same reduced gray level (see g. 6.6(a)). Consequently an enlargement of the
inspected neighborhood causes a atter distribution of gray level densities (see g.
6.6(m)) . The mere employment of a high number of gray levels results in peaks
which are hard to separate (see g. 6.6(d)). When a big number of gray levels and a
big number of inspected neighbor pixels are used (see g. 6.6(p)), peaks are easy to
detect and the best segmentation results are obtained (see g. 6.4.3). Unfortunately,
this setting has an unacceptable run-time behavior.
47
(a) patient h (red channel,
excerpt
01in
100 200
200
400
600
800
1000
1200
(b) histogram of g. 6.6(a) (c) ground truth of g.
6.6(a)
(d) LOCOMM segmentation
(33 neighborhood, 8 gray
levels)
(e) LOCOMM segmentation
levels)
(f ) LOCOMM segmentation
levels)
Figure 6.7: Segmentation by LOCOMM : It can be seen from this gure how the
size of neighborhood aects the result of a segmentation by LOCOMM. It is obvious
that the more pixels are used to compute the local co-occurrence matrix, the stronger
is the blurring because of LOCOMMs averaging character.
6.5 Conjugate Images
Paplinski and Boyce [26] proposed the employment of the concept of conjugate
images for the PCO segmentation. This concept is related to the segmentation by
gray level co-occurrence in many respects. The dierence lies in the computation of
the co-occurrence matrices, in the denition of neighborhood and the used statistical
feature, namely the standard deviation.
Gray level co-occurrence uses pairs of pixels in the same image, while the basic
idea of conjugate images is the employment of pixel pairs with the same coordinates
in dierent images built by directional ltering.
6.5.1 The Concept of Conjugate Images
As a simple approach to this concept it could be said: Let x be a pixel in the original
image, R(x) a neighborhood or region of interest of the reference pixel x and f a
48
function, then a conjugate image c(x) could be dened as follows:
c(x, R) = f(R(x)) (6.22)
Actually the use of one stand alone conjugate image does not make much sense.
It is just the use of an array or a so-called family of conjugate images which is the
key to the power of this concept.
c = [c
1
, . . . , c
n
] (6.23)
6.5.2 Building of Conjugate Images
To give a simple example: An array of four conjugate images are formed by applying
a directional lter in the four cardinal directions (0
o
, 90
o
, 180
o
, 270
o
). To keep the
following segmentation unbiased, an equalization of the contributions from pixels in
the neighborhood of the reference pixel x becomes necessary. This could be done
either by employing non-overlapping lter masks to dene the region of interest or
by using lters with a special shape.
Paplinski and Boyce used a two dimensional edge half operator h(u) obtained
from a 1-d lter introduced by Spacek [33] and improved by Petrou [28] to perform
the directional ltering (for further details the reader is referred to [ 26]) in the four
directions (0
o
, 90
o
, 180
o
, 270
o
) by rotating the lter mask.
In percentage values, the upper quarter of this lter is
h(u) =
_
_
_
_
_
_
0 8 11 8 2 0 0
0 16 26 26 13 1 0
0 26 48 52 33 8 0
0 43 80 80 53 17 0
0 68 100 94 61 21 0
_
_
_
_
_
_
(6.24)
As the variance operator is a good possibility of characterizing texture, this
statistical feature is used for segmentation:
c(x, R) = (
uR
(f(x + u)h(u)
R
)
2
)
1
2
(6.25)
c = [c
1
, c
2
, c
3
, c
4
] (6.26)
where
R
is the mean of the region of interest, h(u) the edge half operator an
is a normalization factor. This factor is set such that the pixel intensity of the
conjugate image lies between 0 and 255 (in fact, the used value for was 40). The
result is a so called family of four conjugate images, namely c.
As it is usually the case in the family of conjugate images, it is not possible
to note the dierence between the member images. All images look like a slightly
blurred version of the original image. The relevant information actually is hidden is
49
the displacement of the images, e.g. a variance edge strength map e can be simply
calculated from this family of conjugate images (see g. 6.5.2) by:
e = |c
1
c
3
| +|c
2
c
4
| (6.27)
For each direction, correlation of the image with the edge half-lter in a rectan-
gular region (2l 1) l is the conjugated image c
i
.
50
(a) PCO image B (cropped) (b) variance edge strength map e
(c) conjugate image c
1
(0
o
) (d) conjugate image c
2
(90
o
)
(e) conjugate image c
3
(180
o
) (f ) conjugate image c
4
(270
o
)
Figure 6.8: Conjugate Images: It is often impossible to notice any dierence be-
tween the members of a family of conjugate images, though they are not identical.
This can be proved by computing the variance edge strength map (equ. 6.27).
51
6.5.3 Segmentation by Conjugate Images
The family of conjugate images (equ. 6.26) can now be employed for calculating a
four-dimensional histogram S(y) to perform the segmentation. For practical reasons,
two co-occurrence matrices formed by a pair of conjugate images from opposing
directions are used instead of the S(y). The matrix S
13
([y
1
y
3
]) is formed from the
conjugate images c
1
and c
3
, and S
24
([y
2
y
4
]) is formed from the conjugate images c
2
and c
4
.
0 50 100 150 200 250 300
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
(a) main diagonal of S
13
(see g. 6.5.2)
0 50 100 150 200 250 300
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
(b) main diagonal of S
24
(see g. 6.5.2)
Figure 6.9: Co-occurrence Matrices of Conjugate Images
In contrast to the gray level co-occurrence, where the two-dimensional histogram
is built by the pixels of the original image and their neighbors, the histogram is
formed by the pixels of two derived images. The segmentation of the image s (equ.
6.28) by these matrices actually is quite analogous to the segmentation described in
section 6.3.3, but it also leads to the problem that in fact the thresholds cannot be
detected automatically in a sucient way.
[ht]s =
1
4
|c
1
+ c
2
+ c
3
+ c
4
| (6.28)
52
(a) segmented image B
(well manually selected
thresholds)
(b) segmented image B
(badly automatically se-
lected thresholds)
Figure 6.10: Segmentation by Conjugate Images: Best results using segmentation
by conjugate images are obtained by a manual setting of thresholds.
6.6 Experiments and Results
6.6.1 Ground Truth
In order to perform an evaluation of the segmentation results, ground truth is nec-
essary. As no gold standard ground truth is available, the segmentation results are
compared to the results of an EPCO scoring(the method proposed by Tetz [ 34]) at
the AKH Vienna (see g. 6.6.1 and table 6.6.1).
The idea of EPCO is quite straightfoward. First, the lens is segmented manually
in regions with an EPCO grading from 0 to 4 (see g. 6.6.1).
(a) epco 0 (b) epco 1 (c) epco 2 (d) epco 3 (e) epco 4
(f ) epco 0 (g) epco 1 (h) epco 2 (i) epco 3 (j) epco 4
Figure 6.11: EPCO Score: In the rst row there are examples for areas typically
graded from 0 to 4. In the second row, the color code for segmented areas is shown.
Finally the areas of the regions are weighted with the gradings (see equ. 6.29)
to obtain the score (see table 6.6.1).
EPCO =
4
i=0
p
i
g
i
4
i=0
p
i
(6.29)
p
i
. . . pixelcount of the regions graded g
i
53
(a) patient A (b) patient B (c) patient C (d) patient D
(e) patient E (f ) patient F (g) patient G (h) patient H
(i) patient I (j) patient J (k) patient K (l) patient L
Figure 6.12: EPCO Ground Truth of the Test Data Set: The images were manu-
ally segmented by a clinician and coded with the colors proposed in gure 6.6.1.
6.6.2 Experiments
Some restrictions have been made to obtain results as rich in substance as possi-
ble. The regions of interest were dened manually for all the test images to inspect
only the relevant parts of original images (Patient A to Patient L) before the seg-
mentation step. Consequently, just the original data and no reectionfree images
were segmented. Additionally, only the rst segmentation step was performed (no
relaxation step), because this does not seem to be interesting at the current stage
of examination.
The segmentation was performed with the red and the green color channel.
Segmentation by Gray Level Co-occurrence (COM)
The segmentation by gray level co-occurrence was implemented in KBVision.
For practical reasons the mapping was reduced to a simple thresholding which
54
ID 0 1 2 3 4 EPCO
A 79.10 12.88 8.02 0.00 0.00 0.29
B 10.55 25.94 30.00 20.35 13.16 2.00
C 0.00 9.86 31.09 42.45 16.60 2.66
D 70.15 23.68 5.02 1.15 0.00 0.37
E 29.65 31.96 6.89 18.60 12.90 1.53
F 34.03 65.97 0.00 0.00 0.00 0.66
G 6.12 20.75 35.59 23.44 14.10 2.19
H 7.26 0.00 8.40 50.60 33.74 3.04
I 0.00 23.61 49.66 11.19 15.54 2.19
J 100.00 0.00 0.00 0.00 0.00 0.00
K 23.09 15.11 6.55 48.14 7.11 2.01
L 13.50 59.31 21.42 4.68 1.10 1.21
Table 6.1: EPCO Score: This table shows the percentage of every EPCO region
type of the test data set and the EPCO score.
was performed on a segmentation image s (according to equ. 6.30):
i(s
x,y
) = i(d(p
x,y
)) (6.30)
As it was not possible to obtain satisfying results by an automatic threshold-
ing,the thresholds were set manually.
LOCOMM
The LOCOMM was coded in C. It provides a rough automatic segmentation
by thresholding the LOCOMM image.
Segmentation by Conjugate Images (CONJ)
For the segmentation by conjugate images, the MATLAB code of Paplinski
[26] was re-implemented. But as also holds true for the segmentation by gray
level co-occurrence, an automatic thresholding did not provide good results.
That is why the output of the MATLAB code was segmented manually.
Some sample results on the subsequent pages (see gs. 6.6.2, 6.6.2 and 6.6.2)
demonstrate characteristic problems of the segmentation methods.
6.6.3 Evaluation
As only LOCOMM provides satisfactory results without manual thresholding, the
evaluation is not done for the other methods. LOCOMM is able to segment rather
uniformly textured regions, but unfortunately it does not give much information on
the quality of the regions. For that reason the EPCO grading has to be performed
55
(a) red channel (b) green channel (c) ground truth
(d) com(red) (e) conj(red) (f ) locomm(red)
(g) com(green) (h) conj(green) (i) locomm(green)
Figure 6.13: Segmentation (Patient A): Flashlight: It can be seen from this gure
that the results of the three introduced methods are quite similar. Unfortunately,
they are all aected by the ashlight artefact.
56
(a) conj(red)
100 200
1000
2000
3000
(b) diagonal:
conj(red)
(c) conj(green)
100 200
1000
2000
3000
4000
5000
6000
7000
(d) diagonal:
conj(green)
Figure 6.14: Segmentation (Patient A): Automatic Segmentation: Most of the
times it is impossible to automatically detect thresholds which lead to satisfactory
results using segmentation by gray level co-occurrence or conjugate images.
(a) red channel (b) ground truth (c) locomm(red)
Figure 6.15: Segmentation (Excerpt Patient I): Giant Pearls: Another problem is
the detection of huge Elschnig pearls which indicate very heavy PCO. All proposed
segmentation tasks merely provide the possibility to detect the borders of these pearls
correctly, since the interior of the pearls is textured like the surrounding areas.
57
0
0,5
1
1,5
2
2,5
3
3,5
4
A B C D E F G H I J K L
A
B
C
D
Mean
Figure 6.16: Clinician Gradings Mapped to EPCO: The objective grading
(range: 0 .. 10) of the four clinicians and the average of these gradings is mapped
to the EPCO score range (0 .. 4).
on the quality of the regions. For that reason the EPCO grading has to be
performed manually to be able to compare the results to the EPCO score. In
addition, the subjective grading (see chapter 3) is mapped to the EPCO range
from 0 to 4. These mappings can be seen from gure 6.6.3.
Finally, the mapped gradings, the EPCO ground truth score and the scores
resulting from LOCOMM (green and red channel) are compared (see g. 6.17)
to each other. The EPCO scores of LOCOMM (red and green channel) provide
results which are highly correlated to the EPCO-score by the AKH Vienna (see
table 6.6.3) and comparable to a pure subjective grading.
Method Di(max) Di(mean) Corr Cov
LOCOMM (red) 0.66 0.26 0.90 0.88
LOCOMM (green) 1.14 0.33 0.94 0.76
Clinicians 1.49 0.37 0.91 0.91
Table 6.2: Evaluation: This table shows a statistical evaluation (maximum and
average dierence, correlation, covariance) of a subjective grading and LOCOMM
in relation to the EPCO scoring.
6.6.4 Conclusion
All three methods are a possible way to obtain a score comparable to the EPCO
score. Unfortunately, merely LOCOMM provides a useful automatic segmenta-
tion for a scoring like EPCO, though the segmentation itself does still not equal
an EPCO ground truth segmentation. The main reasons for this behavior are
57
Figure 6.16: Clinician Gradings Mapped to EPCO: The objective grading (range:
0 .. 10) of the four clinicians and the average of these gradings is mapped to the
EPCO score range (0 .. 4).
manually to be able to compare the results to the EPCO score. In addition, the
subjective grading (see chapter 3) is mapped to the EPCO range from 0 to 4. These
mappings can be seen from gure 6.6.3.
Finally, the mapped gradings, the EPCO ground truth score and the scores
resulting from LOCOMM (green and red channel) are compared (see g. 6.17) to
each other. The EPCO scores of LOCOMM (red and green channel) provide results
which are highly correlated to the EPCO-score by the AKH Vienna (see table 6.6.3)
and comparable to a pure subjective grading.
Method Di(max) Di(mean) Corr Cov
LOCOMM (red) 0.66 0.26 0.90 0.88
LOCOMM (green) 1.14 0.33 0.94 0.76
Clinicians 1.49 0.37 0.91 0.91
Table 6.2: Evaluation: This table shows a statistical evaluation (maximum and
average dierence, correlation, covariance) of a subjective grading and LOCOMM
in relation to the EPCO scoring.
6.6.4 Conclusion
All three methods are a possible way to obtain a score comparable to the EPCO
score. Unfortunately, merely LOCOMM provides a useful automatic segmentation
for a scoring like EPCO, though the segmentation itself does still not equal an EPCO
ground truth segmentation. The main reasons for this behavior are the strong
directional dependency on the illumination and the inability to detect structures
58
0
0,5
1
1,5
2
2,5
3
3,5
4
A B C D E F G H I J K L
Clinicians
EPCO
LOCOMM Red
LOCOMM Green
Figure 6.17: EPCO Ground Truth of the Test Data Set
the strong directional dependency on the illumination and the inability to detect
structures like huge Elschnig pearls. Besides, even the LOCOMM regions have to
be graded manually for an EPCO score, because all these segmentation methods
do not provide any classication.
58
Figure 6.17: EPCO Ground Truth of the Test Data Set
like huge Elschnig pearls. Besides, even the LOCOMM regions have to be graded
manually for an EPCO score, because all these segmentation methods do not provide
any classication.
59
Chapter 7
Conclusion and Outlook
As pointed out before there can be no doubt that an automatic and objective PCO
grading system will be an essential step ahead in the ght against LEC proliferation.
One possible approach to this issue is segmentation of an PCO image, e.g. to obtain
automatically an input for the subjective, computer assisted EPCO system by Tetz
[34]. This chapter provides a summery of the experiments made in the course of
this thesis as well as an outlook on possible improvements to such a system.
7.1 Detection of the Region of Interest
As the original input shows not only the region to grade but the entire eye, a
possible way to an automatic ROI detection is discussed. The proposed approach
consists of an approximate and a rened detection step. Unfortunately merely the
approximate detection (multichannel thresholding, morphology and ISR ltering)
provides the expected results. The rened method proposed by Paplinski and Boyce
[26] is insuciently robust for clinical work. That is why a more practical, semi-
automatic detection based on the rened method is also also introduced.
7.2 Computing of a Reectionfree Image
The retroillumination technique used at AKH Vienna causes reections, which have
a negative eect on grading. The discussed semi-automatic fusion process provides
good results. In addition, a possible way to complete automation of this task is
outlined.
7.3 Segmentation
This thesis introduces three segmentation methods based on multi dimensional his-
tograms:
Segmentation by Gray Level Co-occurrence [8]
60
Segmentation by Conjugate Images [26]
Segmentation by LOCOMM
Only the third approach leads to sucient segmentation results without human
interaction. That is why evaluation, i.e. comparison to genuine EPCO scores, is not
performed for segmentation by gray level co-occurrence and conjugate images. The
LOCOMM based EPCO score shows high correlation with the genuine score.
Unfortunately, even LOCOMM is not completely automatic, because the output
of segmentation still has to be graded manually. In addition, the output of all
approaches is aected by a ashlight artefact caused by the recording system. Hence
to improve the result either the artefact has to be removed, or features like entropy
or variance, which are not that easily aected, have to be employed for segmentation.
Another problem is the detection of structures such as giant Elschnig pearls, because
in the center they are textured like an area without PCO possibly surrounding them.
It is obvious that the employment of further features such as shape, size or edge
strength is necessary to obtain satisfying results.
7.4 Outlook
The semi-automatic detection of ROI may be the best compromise on automation,
stability and good results, because the border of the ROI is sometimes very hard to
determine without medical expert knowledge. However, the calculation of reection-
free images still oers many automation possibilities such as automatic detection of
points of interest, matching or gray level adaption.
Concerning the grading itself the proposed segmentation methods actually do not
seem to be the best solution. As the segmentation results do not contain sucient
information for a completely automatic grading, i.e. there is still human interac-
tion necessary, maybe classication using a texture feature vector rather than one
dimensional segmentation should be considered. A training set would therefore be
essential. Possible features for such a vector could be local variance, mean or entropy.
61
Bibliography
[1] Amerinex Articial Intelligence, Inc., The kbvision system programmers refer-
ence manual, 1994.
[2] D. J. Apple, K. D. Solomon, and et al., Posterior capsule opacication, Surv
Ophtalmol 37 (1992), no. 2, 73116.
[3] P. C. Chen and T. Pavlidis, Segmentation by texture using a co-occurance matrix
and a split-and-merge algorithm, CGIP 10 (1979), 172182, CHEN79.
[4] D. L. Collins, P. Neelin, T. M. Peters, and A. C. Evans, Automatic 3d intersub-
ject registration of mr volumetric data in standartized talairach space, Computer
Assisted Tomography 18 (1994), 192205.
[5] R. W. Conners, M. M. Trivedi, and C. A. Harlow, Segmentation of a high-
resolution urban scene using texture operators, Computer Vision, Graphics, and
Image Processing 25 (1984), no. 3, 273310.
[6] B. A. Draper, R. T. Collins, J. Brolio, A. R. Hanson, and E. M. Riseman,
The schema system, International Journal of Computer Vision 2 (1989), no. 3,
209250.
[7] R. C. Gonzales and R. E. Woods, Digital image processing, Addison-Wesley
Publishing Company, 1993.
[8] J. F. Haddon and J. F. Boyce, Image segmentation by unifying region and
boundary information, IEEE Transactions on Pattern Analysis and Machine
Intelligence 12 (1990), no. 10, 929948.
[9] O. E. Hanuch, V. B. Agrawal, S. Papernov, M. del Cerro, and J. V. Aquavella,
Posterior capsule polishing with the neodymium:ylf picosecond laser: Model eye
study., J Cataract Refract Surg 23 (97), no. 10, 156171.
[10] R. Haralick and L. Shapiro, Computer and robot vision, volume 1, Addison
Wesley, 1991.
[11] , Computer and robot vision, volume 2, Addison Wesley, 1993.
62
[12] C. Harris and M. Stephens, A combined corner and edge detector, Proc 4th
Alvey Vision Conf, vol. 1, aug 1988, Manchester, pp. 189192.
[13] K. Hayashi, H. Hayashi, and et al., In vivo quantitative measurement of poste-
rior capsule opacication after extracapsular cataract surgery, Am J Ophthal-
mol 125 (1998), no. 6, 16235.
[14] B. Julesz, Experiments in the visual perception of textures, Scientic American
232 (1975), 3443.
[15] R. G. Keys, Cubic interpolation for digital image processing, IEEE Transactions
on Acoustics, Speech and Signal ASSP-29 (1981), no. 6, 11531160.
[16] L. Kitchen and A. Rosenfeld, Gray-level corner detection, Pattern Recognition
Letters 1 (1982), 95102.
[17] M. Kuchle, A. Amberg, and et al., Pseudoexfolation syndrome and secondary
cataract, Br J Ophthalmol 81 (1997), no. 10, 8626.
[18] M. S. Lasa, M. B. 3rd Datiles, B. V. Magno, and A. Mahurka, Scheimpug pho-
tography and postcataract surgery posterior capsule opacication, Ophthalmic
Surg 26 (1995), no. 2, 1103.
[19] R. S. Ledley, M. Buas, and T. J. Golab, Fundamentals of true-color image
processing, Proceedings 10. ICPR Int. Conf. on Pattern Recognition 1 (1990),
791795.
[20] L. Mastropasqua, L. Lobefalo, and et al., Heparin eyedrops to prevent posterior
capsule opacication, J Cataract Refract Surg 23 (1997), no. 3, 4406.
[21] D. J. Nadler, N. S. Jae, and et al., Glare disablility in eyes with intraocular
lenses, Am J Ophthalmol 97 (1984), no. 1, 437.
[22] O. Nishi, After cataract: The last big hurdle in cataract surgery, Internet Jour-
nal of Ophtalmology 2 (1997), 911.
[23] R. J. Olson and A. S. Cranda, Silicone versus polymethylmethacrylate intraoc-
ular lenses with regard to capsular opacication, Ophthalmic Surg Lasers 29
(1998), no. 1, 558.
[24] M. V. Pande, P. G. Ursell, D. J. Spalton, G. Heath, and S. Kundaiker,
High-resolution digital retroillumination imaging of the posterior capsule after
cataract surgery, J Cataract Refract Surg 23 (1997), no. 10, 15217.
[25] A. P. Paplinski and J. F. Boyce, Circular region extraction using a ltered
radial gradient method, Tech. Report 95-2, Monash University, Department of
Robotics and Digital Technology, 31 January 1995.
63
[26] , Computational aspects of segmentation of a class of medical images
using the concept of conjugate images, Tech. Report 95-06, Monash University,
Department of Robotics and Digital Technology, 26 April 1995.
[27] , An implementation of the active contour method for noisy images using
a local minimisation algorithm, Tech. Report 95-1, Monash University, Depart-
ment of Robotics and Digital Technology, 12 January 1995.
[28] M. Petrou and J. Kittler, Optimal edge detectors for ramp edges, IEEE Trans-
actions on Pattern Analysis and Machine Intelligence 13 (1991), 483491.
[29] A. Pinz, Bildverstehen, Springer-Verlag, 1994.
[30] N. J. Pressman, Optical texture analysis for automatic cytology and histology: A
marcovian approach, Ph.d. dissertation, Univ. of Pennsylvania, 1976, available
as Tech. Report UCRL-52155, Lawrence Livermore Lab.
[31] A. Rosenfeld and A. C. Kak, Digital picture processing, 2nd ed., vol. 2, Aca-
demic Press, 1982.
[32] D. H. Shin and Y. Y. Kim et al., Decrease of capsular opacication with ad-
junctive mitomycin c in combined glaucoma and cataract surgery, Ophtalmology
105 (1998), no. 7, 12226.
[33] L. A. Spacek, Edge detection and motion detection, Image vision Comput. 4
(1986), 4356.
[34] M. Tetz, M. Sperker, G. U. Auahrt, M. Sperker, M. Blum, and et al., Pho-
tographic image analysis system of posterior capsule opacication, J Cataract
Refract Surg 93 (1997), no. 1, 337.
[35] D. Williams and M. Shah, A fast algorithm for active contour and curvature
estimation, CVGIP: Image Understanding 55 (1992), 1426.
[36] J. K. Wolfe and L. T. Jr. Chylack, Objective measurement of cortical and subcap-
sular opacication in retroillumination photographs, Ophthalmic Res 22 (1990),
no. 1, 627.
[37] R. P. Woods, S. T. Grafton, J. D. G. Watson, N. L. Sicotte, and J. C. Mazziotta,
Automated image registration: Ii. intrasubject validation of linear and nonlinear
models, Computer Assisted Tomography 22 (1994), 139152.
[38] R. P. Woods, J. C. Mazziota, and S. R. Cherry, Mri-pet registration with auto-
mated algorithms, Computer Assisted Tomography 17 (1993), 536546.
64

Tex 01

Enviado por

Dados do documento

Descrição original:

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Tex 01

Enviado por

Direitos autorais:

Formatos disponíveis

Graz University of Technology

Institute for Computer Graphics

Você também pode gostar