FACE RECOGNITION USING LINE EDGE MAPS 9

Face Recognition Using Line Edge Maps

Jarjit Tandel, M.S. in Electrical Engineering. Email: jjt1506@rit.edu
Website: www.rit.edu/~jjt1506

process if rather more robust. This is fascinating

philosopher and scientist for years.
Abstract- The recognition of a human face from a
We are interested in finding a computational
set of human faces has always been a challenging
model of human face, as it can contribute to
task for researchers. Various computational
theoretical insights but as well as to practical
techniques have been employed. This paper
applications. Instances where computers are used to
discusses the different techniques for human face
identify faces, for instance criminal identification,
edge detection, the challenges using these
security systems, image and film processing, a
techniques, find the best one and develop a real-
computational model of face play a very important
time system to accurately recognize human faces. It
role. Each presents different constraints in terms of
emphasizes on the fact that human faces are similar
processing requirement. For example, the ability to
in structure with minor differences from person to
model a face and distinguish it from a large number
person. Moreover there are certain factors like
of stored face models would make it possible to
lighting conditions, facial expressions, ageing and
vastly improve the security system in any company.
orientations of the face which increase the
It can also be implemented in photo printing
complexity of face recognition process. A compact
mechanism.
face feature, Line Edge Map (LEM) is developed
However developing a computational model of a
which represents a unique face and is used by the
face is not a trivial process. They are a natural class
two face images to find similarity between them by
of objects. Various techniques have been employed
using Hausdorff distance measurement. Researches
to recognize a face. Edges of a face may be
demonstrate that this technique gives superior
considered to be an important feature for
performances compared to the other techniques.
classification of faces, as it is unique to a person. In
When compared with the eigenfaces approach,
this paper we explore this fact to find and recognize
results were found to be as accurate as the
faces. A novel concept to develop a Line Edge Map
eigenfaces approach. However in certain cases, it
(LEM) to represent a unique face is proposed in
may be considered to be more accurate than
this paper. LEM is developed by first finding the
eigenfaces approach.
edges of the face using a sobel filter, followed by
thinning of the edges and finally applying
I. INTRODUCTION polynomial curve fitting algorithm to account for
the discontinuities in the edged image. LEM is

F ace recognition has always been a major field

of research within the biometric signal
processing. The face is our primary focus of
attention in social intercourse, playing an important
role in conveying identity and emotion. Although
the ability to infer intelligence is a suspect, the
human ability to recognize faces is remarkable. We
see thousands of faces in our lifetime. Though we
can accurately recognize a face irrespective of
change in viewing conditions, ageing, expressions
and distractions such as glasses, change in hair
style, etc. We may say that the face recognition
invariant to lighting condition, varying facial
expression, and varying pose.
Determining the edges of a human face
accurately is very important for accurate face
recognition. In this paper three approaches to find
edges of a face are discussed, skin color
segmentation [2], Vector Gradient Approach [3]
and applying a dynamic two strip curve fitting
algorithm to the edge of the image [4].
On comparing it to the other face recognition
approaches, it is found to perform superior to (or
equally to) the eigenfaces approach, which is
FACE RECOGNITION USING LINE EDGE MAPS
considered to be one of the best face recognition
technique.
In the following, a literature review of face
recognition techniques is given in Section 2, where
a brief idea of other face recognition techniques
like eigenface, template matching, geometric
feature matching techniques and dynamic link
architecture are discussed. Section 3 gives an
overview of a generalized face recognition system
and describes various techniques to detect edge of a
face. Section 4 describes the concepts of face
recognition using line edge map. Section 5 outlines
the Hausdorff Distance method to find the
similarity between two images. The results are
addressed in Section 6 followed by conclusion in
Section 7 and references in Section 8.
II. BACKGROUND AND RELATED WORK
Much of the work in computer recognition of faces
has focused on detecting individual features, such
as the eyes, nose, mouth, and the head outline, and
defining a face model by the position, size and
relationships among these features. This section
overviews the major face recognition techniques
applied to frontal face. Some of the techniques
discussed in this topic are eigenfaces, template
matching, geometric feature matching techniques
and dynamic link architecture. The approaches are
compared in terms of the facial representation,
lighting conditions and change in pose.
Eigenface approach is considered to be one of the
best approach for accurate face recognition. It is
also known as Karhunen - Loeve expansion,
eigenpicture, eigenvector, and principal
components. Scientist believed that human face can
be reconstructed by a small collection of weights
for each face and a standard face picture
(eigenpicture). A mathematical model of the face
features was developed and based on eigen values
and their corresponding eigen vectors, a eigenface
was developed. Each face can be represented by a
linear combination of the eigenfaces. However
this approach has some limitations. Eigenfaces is
sensitive to lighting conditions, size variation and
face orientation.
A simple version of template matching is that a
test image represented as a two-dimensional array
of intensity values is compared using a suitable
9
metric, such as Euclidian distance, with a single
template representing the whole face. In order to
match faces with different viewpoints, more than
one template from different view points has to be
considered. This approach of face recognition is
computationally complex. Moreover defining
templates is a major drawback of this approach. On
the other hand, Geometrical feature matching
techniques are based on the computation of a set of
geometrical features from the picture face. This
approach can be used to recognize a face in an
image with coarse resolution as low as 8*6 pixels
[10]. It can be widely applied to find possible
matches in the large database such as a mug shot
album. The major constraint of this approach is that
it is dependent on the accuracy of the feature
location algorithm.
III. GENERALIZED FACE RECOGNITION
SYSTEM
This section gives an overview of the underlying
face detection algorithm.
The face detection problem can be stated as
follows: given an image compare it with all the
images in the database, find the best match amongst
them the face co-ordinates within an image.
Fig. 1: Generalized Face Recognition System [8]
Initially we concentrate on the task of finding a
single face in an image to avoid complexity.
Henceforth the extension to finding multiple faces
is straightforward. In this paper an edge based face
recognition method is employed. Thus the primary
aim is to find the edge magnitude image with an
edge operator, thin the edges and extract features
by a locally adaptive threshold filter to compensate
variation in illumination. One of the main challenge
in using this approach is to accurately find the
edges of the face. Various techniques have been
FACE RECOGNITION USING LINE EDGE MAPS 9
employed. In this paper, three such techniques are
described, 1) skin color segmentation [2], 2) Vector Fig. 2: Detect edges using skin color segmentation
Gradient Approach [3] and 3) applying a dynamic
two strip curve fitting algorithm to the edge of the Mean Vector:
image [4].
Mean vector is defined as the ratio of sum of data
1) Skin Color Segmentation: points to total number of data points,
Our objective is to develop a classifier skin and 1 n …
x= ⋅ ∑ xi Eq. (2)
non-skin regions in and image. We can classify the n i =1
given set of data into 2 classes: skin and non-skin
regions. Using Bayes’s rule we can develop one Covariance Vector:
such classifier. From Bayes’s rule we can write, Covariance is the measure of how much two
variables vary from each other. It is defined using
( x− x )2 the following equation,
^
1 −
p ^ (ω i | x) = e 2.σ 2 n
…Eq. ∑( x ) ⋅( y )
i − x i − y
…Eq. (3)
ω i|x 2π .σ cov( x ) = i =1

n −1
(1) Multivariate Density Function:
Where, ωi is the class, x is the test sample, σ2 is the The general multivariate normal density in d
variance. Let us define each of these terms in brief. dimension is written as under,

Input Image in RGB Format 1  1 

. exp − ( x − µ ) ∑ −1 ( x − µ )
t
p ( x) = d 1
(2π ) 2
|∑| 2  2 
Convert the image in RGB format to YES format ...Eq. (4)

Where, x is a d-component column vector, d is

Calculate Mean Vector, Covariance Matrix And
Mahalanobis Distance
Calculate True Positive and False Positive
Values For Different Values Of Threshold
the dimension (i.e., number of features. In our case
it is 2), Σ is a d-by-d covariance matrix and μ is d-
component mean vector and | ∑| is the
determinant of covariance matrix.

Plot Receiver Operating Characteristics (ROC)

Curve

Determine Threshold Value

Is pixel value No
within
threshold?

Yes
Label as skin Label as non-skin
region region
FACE RECOGNITION USING LINE EDGE MAPS 9
Classification:
i
Choose Wi when Pr[W K | X g ] = max Pr[Wi | x g ]
p ( x g | Wi ). Pr( Wi )
Pr[ Wi | x g ] =
p( x g )

Ii 1 …. Pixel at g ⊂ Wi
zgj =
0
…. Pixel at g ⊄ Wi
Li
1
∑ ∑ fp
Ii λt , I ij
FPλt j = g
T2 j =1 ( g∈I ij )
Li
1
∑ ∑ tp
I ij λt , I ij
TP λt = g
T1 j =1 ( g∈I ij )

Fig. 3: P.D.F. of skin region

... If z g = 1 and λj ≤ λt
i
I i
λt , I ij 1 j

tp g =
0 .... otherwise
Color Space:
1
…. if z g = 0 and λj ≤ λt
i
A new color space YES is defined in this project. It λ , I ij
fp g t =
I j i

overcomes the limitation of RGB color space, 0

…. otherwise
which is sensitive to luminance. Moreover YES
color space is used because of the following
reasons,
 It reduces chrominance variation,
 It is computationally efficient,
 It is free of singularities.
The conversion between YES and RGB color
spaces is as under,
Y  0.253 0.684 0.063 R 
E  = 0.500 − 0.50 0.000   
   G  … Eq.

S 
  0.250 0.250 − 0.500 

B 

(5)
where, Y represents the luminance and the
chrominance is defined by the two planes E and S. Fig. 4: R.O.C. Curve [11]
Universal Threshold: The above figure illustrates the receiver operating
characteristic curve (R.O.C.), which is a plot of true
The classification is performed over various positive value against false positive value. The
thresholds and based on the true-positive/false- point where the ROC curve intersects the true
positive (ROC curve analysis); we define the value positive value is considered to be an ideal threshold
of universal threshold. The process can be value for the skin region. Thus by thresholding the
summarized by the following sets of equations, image we can extract the skin region. Hence we can
find the edges of the skin region by using an edge
λig < t ai ( h i , t ui ) … Eq. (6) operator. However this approach solely depends on
the values in the training set, which further
λig > t ai ( h i , t ui ) … Eq. (7) complicates the face recognition process.
FACE RECOGNITION USING LINE EDGE MAPS 9
2) Detecting Boundaries in a Vector Field:
Boundary detection in a two-dimensional field with Find maximum eigen value
one attribute, i.e., edge detection in a
monochromatic image is an extensively studied
field. Many types of approaches have been
employed. In one type, change in a single attribute Find the corresponding eigen vector
in the two dimensional field is computed as the
gradient of a scalar field. In this paper a vector
Draw line (critical line) through 2 reference
gradient approach is proposed to detect boundaries
points
of a face. Earlier studies on finding the edge of a
color image were based on finding the edge of each
R, G and B plane separately, followed by clubbing Define 2 boundary points parallel to critical
them into one. Direction of an edge pixel is chosen line and at a distance d from critical line
to be either the direction corresponding to the
maximum component or weighted average of the
three gradient directions. d is also called as the error tolerance
However instead of finding the edge of the three
color planes separately, a single technique should
be employed to identify the edges in all the planes.
Images may have multiple dimensions. Hence in Curve is traversed point by point
general we may define an image as a vector field
that maps m-dimensional (spatial) space to n-
dimensional (color) space.
Is pt.
Input Image outside the No
boundary
region?
Find partial derivative D(x) of the kth
component w.r.t. jth component of x
Yes

Find eigen values of DTD matrix Line Segment is found

2 2 2
 ∂u   ∂v   ∂w 
p =   +  + 
Largest value of matrix D is the gradient &  ∂x   ∂x   ∂x 
corresponding right SVD the direction  ∂u  ∂u   ∂v  ∂v   ∂w  ∂w 
t =  
 +     + 
 

 ∂x  ∂y   ∂x  ∂y   ∂x  ∂y 
Fig. 5: Find edges using vector field
Fig. 6: Two Strip Dynamic Algorithm
For RGB, colored image the matrix D is given as 2 2 2
under,  ∂u   ∂v   ∂w 
q =
 ∂y 
 +
 ∂y 
 +
 ∂y 

∂u / ∂x ∂u / ∂y       
D =
∂v / ∂x ∂v / ∂y 


∂w / ∂x ∂w / ∂y 
 and the largest eigenvalue is given by,
Thus, DTD matrix is given by, 1
λ= ( p + q + ( p + q ) 2 − 4( pq − t 2 ) )
p t 2
DT D = 
t q
 This is one the best approach is accurately
where, identifying the edges of a higher dimensional
FACE RECOGNITION USING LINE EDGE MAPS
objects. To compare the two edged images
developed by this approach, we use the Euclidian
distance. The major limitation of this approach is
that it can only recognize images that are stored in
our database.
9
The approach of the Line Edge Detection
technique can be summarized as in Fig. 8.
V. HAUSDORFF DISTANCE

3) Dynamic Two-Strip Algorithm: Hausdorff distance measurement technique uses the

It this approach we develop an algorithm which
first finds the best fitted left-hand and then finds the
right-hand side strips at each point on the curve.
The orientation and width of the strip can be
adjusted dynamically and the corresponding line
segment can be derived. The general approach to
this algorithm is as follows,
In this approach, we first draw a line between two
reference points, which are the points on the edges
on the image. We name this line as critical line.
Then two boundary lines which are parallel to the
critical lines and at a distance d from the critical
line are defined. These two boundary lines restrict
the curve between 2 lines. Thus smaller the
spatial information of an edge map without
considering the inherent local structural
characteristics inside such a map. Line Edge Map
(LEM), is proposed here to integrate the structural
information with spatial information of a face
image by grouping pixels of face edge map to line
segments.
Line Segment Hausdorff Distance (LHD)
measure is used to match LEMs of faces. LHD has
better distinctive power as it makes use of the
additional structural attributes of line orientation,
line-point association and number disparity in
LEM.
Consider two LEMs
M ={m1 , m 2 ,......... ..., m p } , represent a model
l l l l

distance, greater is the accuracy. A curve is then

LEM in database and T = {t1 , t 2 ,......... , t n } ,
l l l l

traversed through these two points is within the

boundary region. Curve is traversed point by point, represent an input LEM. The distance between two
l l
eventually covering the entire image. line segments mi and t j is defined as,
d (mil , t ;jl ) = d θ2 (mil , t lj ) + d //2 ( mil , t lj ) + d ⊥
2
(m li , t lj )
Boundary line The above equationInput Image
finds the distance between
the two images. The distance between the two
Critical Line images is small, when the two images are similar.
Encode into binary edge map
Thus is a helpful technique for face recognition.
Boundary line
Grouping the pixels of face edge map
Fig 7: Definition of a strip to line segments

Thinning the edge map

IV. LINE EDGE MAP

The faces are first encoded into binary edge maps, Polynomial Curve Fitting Process
using an edge detection algorithm. Then our aim is
to find the distance or difference between the image
Line Edge Map image
under consideration and the image in the training
set, i.e., the database. This paper uses Hausdorff
distance measurement technique to find the Calculate Hausdorff distance
similarity between the two images. Housdorff
distance measurement technique is used as it is less
sensitive to noise. This technique of distance Images
measurement uses only the spatial information of similar
? one Next
an edge map without considering the inherent local image
structural characteristics inside such a map. in the
databas
e?

Recognized image
FACE RECOGNITION USING LINE EDGE MAPS 9

VI. RESULTS

 Result using skin color segmentation

technique,
(c)

Fig 9: a) Original Image, b) Extracted Skin Region,

c) Edge Of The Skin Region

 Result using a vector field in detecting

boundaries:

(a)

(a)

(b)
FACE RECOGNITION USING LINE EDGE MAPS 9
(b)

Fig. 10: a) Original Image, b) Edge of the

original image

 Result using dynamic two-strip algorithm:

(a) (b)
FACE RECOGNITION USING LINE EDGE MAPS
(b)
Fig. 10: (a) Original Image, (b) Edge of the image using Sobel filter, (c) Thinned edges of the image,
(d) applying dynamic two-strip algorithm
VII. CONCLUSION
Figure 9, figure 10 and figure 11 are the outputs of
the different approaches used to detect edges of a
human face. On observation, we may infer that
applying edge detection operator to the extracted
skin region as shown in figure 9, is not an efficient
approach to find the edges of the face, as it depends
on the training set under consideration. Figure 10
shows the output of the edge detection process
using vector field. This can be considered as one of
the best face edge detection algorithm. However
when applied to face recognition, it detects face
that are currently in the database using Euclidian
distance measurement technique. It fails to detect
faces with variation in pose and orientation. On the
other hand, the by using dynamic two-strip
algorithm, we can find the straight line edges,
which makes it computational more efficient and
robust. Thus, is considered an ideal technique to
locate edges of human faces in face recognition
problem. This edge detection technique can be
9
(d)
extended to applications where human face needs
to be recognized which is invariant to illumination,
face orientation, pose, etc.
VIII. REFERENCES
[1] Yougsheng Gao and Maylor K.H.Leung ‘Face
Recognition Using Line Edge Map’, IEEE
Transactions on Pattern Analysis and Machine
Intelligence, VOL.24, NO.6,JUNE 2002.
[2] Eli Saber,A.Murat Tekalp,Reiner Eschbach and
Keith Knox ‘ Automatic Image Annotation Using
Adaptive Color Classification’, Graphical Models
and Image Processing, VOL.58,NO.2, MARCH
1996.
[3] Hsien Che Lee and David R.Cok ‘Detecting
Boundaries in a Vector Field’
IEEE Transactions on Signal Processing, VOL.39,
NO.5, MAY 1991.
[4] Maylor K.Leung and Yee-Hong Yang
‘Dynamic Two-Strip Algorithm in Curve Fitting’,
Pattern Recognition, VOL.23, NO.1, 1990.
FACE RECOGNITION USING LINE EDGE MAPS 9
[5] Baofeng Guo, Kin-Man Lam, Wan-Chi Siu and
Shuyuan Yang ‘Human Face Recognition Using A
Spatially Weighted Hausdorff Distance’, IEEE
2001.
[6] Oliver Jesorsky , Klaus J.Kirchberg and Robert
W Frischholz ‘ Robust Face Detection Using the
Hausdorff Distance’
Third International Conference on Audio and
Video- Based Biometric Person Authentication,
JUNE 2001
[7] Matthew Turk and Alex Pentland ‘Eigenfaces
for Recognition’, Journal of Cognitive
Neuroscience Volume 3, Number 1.
[8] Klaus J.Kirchberg, Oliver Jesorsky and Robert
W Frischholz ‘Genetic Model Optimization for
Hausdorff Distance-Based Face Localization’,
International ECCV 2002 Workshop on Biometric
Authentication, June 2002
[9] Y.Gao ‘Efficiently Comparing Face Image
Using A Modified Hausdorff Distance’, IEE Proc.
Image Signal Processing, VOL.150, NO.6,
December 2003