Using LabVIEW and Vision Development Module to Create a Robust Automated Face Detection System

"We created a system capable of detecting multiple faces using the

LabVIEW graphical development environment and Vision Development
Module to perform complex, tedious tasks. NI products eased the
development process with built-in functions in the basic package as well
as complex imaging functions in the NI Vision Development Module."
- Alex See, Monash University Malaysia

The Challenge:
Creating a reliable face detection system that can accurately detect faces in a cluttered image.

The Solution:
Figure 1: System Hardware

Using NI LabVIEW software and Vision Development Module to develop a fast, automatic face detection system that can run on
any standard computer system without additional hardware requirements.

Author(s):
Alex See - Monash University Malaysia
K.Y. Tan - Monash University Malaysia
Accurate face detection in a video feed has many uses, including facial recognition, closed-circuit television (CCTV) monitoring, and as an autofocusing feature for video
conferencing or web cameras. However, the vast image variations in a cluttered and uncontrolled environment require substantial processing to accurately detect faces with
acceptable speed.
Human face detection in a cluttered environment is a notorious problem for researchers in the computer vision community. The face detection system we created aims to properly
locate the region(s) in the video image consisting of a human face. The first step is effectively reducing the region required for computation. We achieve this with segmentation by
color and background subtraction.
To properly locate an exact, relevant, and appropriate image of the face, the system accurately matches an average face template using normalized cross-correlation. We perform
the correlation on relative gradient images, a novel image processing method suggested by Wei and Lai, offering lighting invariant images. This method overcomes the limitation of
the grey-level image template matching in lighting variations.

Color Segmentation
To segment the skin-colored regions, we use hue-saturation value (HSV) because the skin tones of different ethnic groups actually have a similar HSV, with only the illumination
differing. Hue-saturation plane is suppose to be unaffected by differences in illumination caused by changing lighting conditions.
We use a combination of built-in morphological functions in the Vision Development Module to carry out color segmentation (see Figure 3). We segment the skin-toned pixels using
threshold for the hue and saturation values. The particle filter removes any small, noisy particles that dilate as the function closes holes to condition the large particles for easier
processing later.
The hue-saturation plane is supposedly not affected by illumination differences, however, we found that this is true only to a certain limit of brightness or darkness. To achieve
acceptable system performance, we must segment skin-like regions in various lighting conditions. Note that natural sunlight has differing colors throughout the course of the day and
artificial lighting differs in color planes other than illumination.

Background Subtraction
To meet these varied requirements, the threshold limits should allow for a wider range of hue values, but this results in a large occurrence of error detection of skin-like hue regions in
the image. For this reason, we use background subtraction to aid in the segmentation. By performing background subtraction, we can rule out any skin-tone-colored items in the
background.
Background subtraction uses a combination of basic operators to add, subtract, and divide the pixel values between the background hue and the video frame hue. The hue plane
offers lighting invariance while the saturation plane is not considered to simplify the processing.
Figure 4 shows the results in which intermediate grey values are the actual matching pixels (due to the mathematical manipulation), and the lighter and the darker the deviation from
the grey, the larger the difference between the pixel values. This image is transformed to binary using multiple thresholding and later processed using particle filter and dilation before
combining it with the color segmentation result.

Combined Segmentation
After combining the two segmentation results, we remove more particles and perform particle analysis to use the bounding boxes of the segmented particles for face matching.

Face Matching
Once the color segmentation and the background subtraction results merge, the program uses the information for face matching. Face matching is necessary because the
segmentation result consists of any skin-colored nonbackground items in the video frames, including garments, hands, and neck. The system must accurately locate the face to
obtain proper images containing the facial features with the optimal region of interest.
The grey-level image template matching has high failure rates under lighting conditions that differ from the template image. To counter this problem, we use the relative gradient
image approach suggested by Shou-Der Wei and Shang-Hong Lai [1]. The relative image gradient is given by the following equation [1]:

1/8

www.ni.com

Where I(x,y) is the image intensity function that denotes the gradient operator that takes the partial differentiation across the x and y direction, W(x,y) is a local window centered at
location (x,y), and c is a constant (to avoid division by zero). This equation provides some smoothing effect to the gradient value due to the division with a local maximum resulting
in a possible lighting invariant gradient image.
The relative image gradient of both the average face and the video frames detect the faces in the segmented regions. We use the relative gradient image of the average face as the
template for template matching and perform it using normalized cross-correlation (the pattern matching functions in LabVIEW). We use the incremental pyramidal search for each of
the segmented particlethe size of the images to check for matches start from small to large. We do this to accommodate possible face size variations in video frames.
For faster and more accurate template matching, we shrink and blur the images. We use small templates and shrink the images used to search for faces down to the size of the
template and increase their size through the search. The results show that this approach is effective and even detects faces with glasses on, at different angles, and at varying
distances.
After detecting the faces, they are extracted and reoriented to become straight, using the results from the successful pattern matching.

Conclusion
We created a system capable of detecting multiple faces using the LabVIEW graphical development environment and Vision Development Module to perform complex, tedious tasks.
NI products eased the development process with built-in functions in the basic package as well as complex imaging functions in the NI Vision Development Module. This system is
useful in various surveillance activities. With the minimal use of expensive industrial grade hardware, such as a CCD camera, we created a lowcost but robust face detection
system.
Author Information:
Alex See
Monash University Malaysia
No 2 Jalan Kolej, Bandar Sunway
Selangor 46150
Malaysia
Tel: +60 56360600
Fax: +60 56329134
alex.see@eng.monash.edu.my

2/8

www.ni.com

Figure 1: System Hardware

3/8

www.ni.com

4/8

www.ni.com

Figure 2: Program Flow

Figure 3: (a) Image (b) After Thresholding (c) After Particle Filter (d) After Dilation

Figure 4: Before Threshold

5/8

www.ni.com

Figure 5: Color Segmentation

Figure 6: Background Subtraction

6/8

www.ni.com

Figure 7: Relative Gradient Images in Different Lighting

Figure 8: The Template Image and Examples of Detection at Various Angles

Figure 9: The Main Program Front Panel

7/8

www.ni.com

Figure 10: The Main Program Front Panel Showing the Reoriented Detected Face

Legal
This case study (this "case study") was developed by a National Instruments ("NI") customer. THIS CASE STUDY IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND AND SUBJECT
TO CERTAIN RESTRICTIONS AS MORE SPECIFICALLY SET FORTH IN NI.COM'S TERMS OF USE ( http://ni.com/legal/termsofuse/unitedstates/us/).

8/8

www.ni.com