A Robust License-Plate Extraction Method under Complex Image Conditions

Sunghoon Kim, Daechul Kim, Younbok Ryu, and Gyeonghwan Kim

Dept. of Electronic Engineering
Sogang University
CPO Box 1142, Seoul 100-611, Korea

doncopy, s950812, roy21, gkim@sogang.ac.kr

Abstract rectly affects system’s overall accuracy. To make the ex-

A robust approach for extracting car license plate from
images with complex background and relatively poor qual-
ity is presented in this paper. The approach focuses on deal-
ing with images taken under weak lighting condition. The
proposed method is divided into two steps: 1) searching
candidate areas from the input image using gradient infor-
mation, and 2) determining the plate area among the candi-
dates and adjusting the boundary of the area by introducing
a plate template. A set of experiments has been performed
to prove the robustness and accuracy of the approach. For
many images collected from a large underground parking
place the result shows that 90% of them are correctly seg-
mented.
1. Introduction
As the ITS(Intelligent Transport Systems) becomes get-
ting bigger attention, automatic identification of a vehicle
has turned out to be an important research issue [5]. In
general, there are two ways in identifying a vehicle: one
approach is to use of electronic devices to identify each in-
dividual car. The RF ID tag, which has been attached to a
car for the purpose, is a popular example. The unique infor-
mation from the tag, when microwave signal arrives from
an external antenna, is decoded and used for identifying the
car. Another way is to retrieve the information on the li-
cense plate. In the approach, a camera captures the image
of a car and a computer processes the image and recognizes
the information on the plate by applying various image pro-
cessing and optical pattern recognition techniques [6].
Recognition-based approaches reported in previous re-
search generally consist of several procedures, including ex-
tracting a license plate region, segmenting characters from
the region, and recognizing each character. Among the pro-
cedures, accurate extraction of the license plate region from
a variable of scenes is of crucial importance because it di-
traction process successful, many difficult problems should
be dealt with, such as poor image quality due to various am-
bient lighting conditions and image distortion stems mainly
from various combination of visual angles between the cam-
era and the car. The image capturing process is largely
depending on several aspects, including camera angle, dy-
namic range, ambient lighting condition, background com-
plexity, existence of reflection and plate fracture [2]. There-
fore, it is necessary to make the extraction process robust so
that it can work on any situation in the real world environ-
ment.
For extracting the plate region, techniques such as edge
extraction [1][7], Hough transform [8], histogram analy-
sis [4] and morphological operators [10] have been applied.
An edge-based approach is normally simple and fast. How-
ever, it is too sensitive to the unwanted edges, which may
happen to appear in the front part of a car. Therefore, this
method cannot be used independently. Hough transform for
line detection gives positive effect on images with a large
plate region where it can be assumed the shape of license
plate is defined by lines. However, it needs a large mem-
ory space and considerable amount of computing time. The
histogram based approach does not working properly on the
image with noises and the image with tilted plate. Morphol-
ogy has been known to be strong to noise signals, but it is
rarely used in real time system because of its slow opera-
tion.
The approach presented in this paper is to extract the
region of the license plate from images taken from indoor
parking lots, which suffer from various real world problems.
We have encountered many images with various problems,
including poor ambient lighting, diverse plate locations, tilt
of the plate, and influence from strong light beams. The
proposed approach consists of two parts: 1) searching can-
didate areas using a combination of gradient features ob-
tained from the input image, and 2) determining the plate
area among the candidates and adjusting the boundary of
the area by introducing a plate template.

1051-4651/02 $17.00 (c) 2002 IEEE

 Figure 2. The variance is plotted based on

for a plate image

Figure 1. The overview of the algorithm

2. Outline of the approach

A two-stage search process is proposed in the approach:

the global search in which candidates of the plate region are
sought using the combination of the gradient features, and Figure 3. Feature images formed using  (left)
the local search where the correct plate region is selected and  (right) for two plate images
from the candidates and the boundary of the region is ad-
justed using a plate template. Details on each step shown in
Figure 1 are described in this section. from windows of size 1¢9 pixels using Eq.(2). The
window size is determined experimentally. Figure 2
2.1 Global search shows the changes of variance in a plate image.



 




In the beginning part of the stage, histogram stretching



  
       
  (2)
is performed as a pre-processing, using Eq.(1), in order to  
increase the contrast of the image and protect the vital edges
from subsequent operations. where
 represents the gradient at

   and  is


  

 (1)
where,  represents the intensity value of the input image,
 is the variance of the image, and  is a constant (set to 30
in this case).
Step 1: Obtaining three statistical features
Conventional approaches, which rely only on the variance
of the gradient while ROIs being searched, could result in
the number of pixels involved while the variance is ob-
tained.
¯ Feature2 - the density of edges: Plate regions tend to
have a high density of edges. The density is measured
in a block of size 1¢9 pixels by summing all edge pix-
els using Eq.(3).


   


  

   (3)
 
  
limited success when the image under consideration con-
tains a tilted plate and/or complicated background. There-
fore three statistical features are introduced in the proposed where 

   represents the edge magnitude at

  ,
approach and combined by a simple neural network.
¯ Feature1 - the gradient variance: The gradient of the
image is obtained using the Sobel operator. Based on
the observation that gray scale changes are more fre-
quent in the plate area, local variances are obtained
and  is the number of pixels involved while the fea-
ture is obtained.
Bottom left image in Figure 3 is formed using the fea-
ture described above for the upper left image in the
figure.

1051-4651/02 $17.00 (c) 2002 IEEE

 ¯ Feature3 - the density variance: The feature is based
on the concept that if a candidate area indeed contains
a plate, the foreground pixels are distributed evenly
comparing to the areas with simple structures [3].
Therefore, this feature can be used to discriminate text
regions from background regions. To obtain the fea-
ture, a block of size 9¢9, centered at

  , is divided
into nine equal-sized sub-blocks. For each sub-block

, let  denote the total number of edge points in the
sub-block. Then the feature, the density variance, is


defined by Eq.(4).

 
¿

   

 (4)


where 
 is the average of   s.
Bottom right image in Figure 3 is formed using the
feature described above for the upper right image in
the figure.
Those three features are combined with a neural network
to determine if a particular pixel under examination could
belong to the plate region. To train the network, around
300,000 patterns were collected from 50 images. The out-
put of the network forms a map, which is called the feature
map hereafter.
Step 2: Finding candidate areas
In the feature map, areas with higher variance are grouped
and labeled using chaincode-based contour follower [9].
While the processing is being performed, careful examina-
tion is carried out to apply certain limitation on the size of
candidates. The examination prevents from creating candi-
dates with too big or small aspect ratio comparing to that of
the license plate.
Step 3: Selecting an ROI
Candidate areas are represented by bounding boxes and a
plate region may consist of one or more than one boxes as
shown in Figure 4(left). To merge the boxes, a simple neural
network is employed and it determines whether two boxes
under examination need to be merged or not. As inputs to
the neural network, corner points of the boxes, aspect ratios,
and box filling ratios are provided.
Figure 4. Merging bounding boxes: before
(left) and after (right)
1051-4651/02 $17.00 (c) 2002 IEEE
Figure 5. The search procedure
After the merge is completed, the boxes with both the as-
pect ratio of less than 1/2 and horizontal length of less than
200 pixels are used for the next step. A region of interest
is determined by another back-propagation neural network
which inputs box diagonal indices, average value of inten-
sities and box filling ratio.
2.2 Local search
Once an ROI is selected, it is necessary to make sure
that the box contains the plate. The similarity transform is
performed to the plate template so that it fits into the ROI
selected. Then, projection profiles on both directions are ex-
amined to judge whether the ROI contains the plate or not.
When the parameters collected from the projection profiles
satisfy the conditions shown in Figure 5, the ROI is regarded
as the correct plate region. If the ROI does not satisfy either
of the conditions, the process is repeated until the selected
ROIs are consumed.
Figure 6 illustrates intermediate results of some steps ex-
plained in this section for an input image.
3 Experiments
Many number of images with various types of vehicles,
not only passenger cars but also vans and trucks, were col-
 (a) (b)
(c) (d)
(e) (f)
Figure 6. Processing steps: (a) an input im-
age, (b) the feature map, (c) bounding boxes,
(d) selected ROI, (e) initial setting for the local
search, and (f) the local search result
lected from a large indoor parking place with different an-
gles and different lightening conditions. The image acqui-
sition system consists of a CCD camera, a photo sensor to
detect presence of a car, and a personal computer with a im-
age grabber in it. Size of the image is 640492. 1,000 of
them were used for the evaluation purpose.
The experimental result shows that the proposed algo-
rithm locates correctly the plate area for 90% of the images
used. Sources of the failure can be classified into three ma-
jor categories: 1) existence of other text blocks, 2) bounding
box containing the plate is merged into other box(es), and
(3) weak gradient information from the plate area.
As shown in Figure 7, by combining the features using
the neural network the system becomes more robust against
the noise and tilt comparing to the other method [5].
4 Summary and Conclusion
A robust approach for extracting license plate from a car
image is presented in this paper. The two-step approach is
designed to deal with images taken under various real world
conditions. In the first step, the whole image is searched and
candidate areas are located based on the gradient features.
In the second stage, the candidates are examined to verify
1051-4651/02 $17.00 (c) 2002 IEEE
Figure 7. The gradient variance method (left)
and the proposed method (right)
whether it contains the license plate by introducing a tem-
plate of the license plate.
Based on the experimental result, the performance of the
proposed approach is promising. Comparing to the conven-
tional edge-based approach, the overall processing speed
is a little bit slower, but the approach is more robust in
terms of accuracy. Ways to make the approach more ro-
bust and faster are being sought by combining a couple of
techniques, including the active contour algorithm.
References
[1] D. H. Ballard. Computer Vision. Prentice-Hall Inc., 1991.
[2] M. W. Burke. Image acquisition. Chapman & Hall, 1996.
[3] W.-Y. Chen and S.-Y. Chen. Adaptive page segmentation
for color technical journals’ cover images. Image and Vision
Computing, 16(12-13):855–877, 1998.
[4] D. U. Cho and Y. H. Cho. Implementation of preprocessing
independent of environment and recognition of car number
plate using histogram and template matching. The Journal of
the Korean Institute of Communication Sciences, 23(1):94–
100, 1998.
[5] D. Gao and J. Zhou. Car License Plate Detection from Com-
plex Scene. In Proceedings of International Conference on
Signal Processing, pages 1409–1414, 2000.
[6] P. Hu, Y. Zhao, J. Zhu, and J. Wang. An Effective Automatic
License Plate Recognition System. In The proceedings of
CISST2000, pages 80–84, 2000.
[7] K. Kanayama, Y. Fujikawa, K. Fujimoto, and M. Horino.
Development of vehicle-license number recognition sys-
tem using real-time image processing and its application to
travel-time measurement. In Proceedings of IEEE Vehicular
Technology Conference, pages 798–804, 1991.
[8] K. M. Kim, B. J. Lee, K. Lyou, and G. T. Park. The auto-
matic recognition of the plate of vehicle using the correla-
tion coefficient and Hough transform. Journal of Control,
Automation and System Engineering, 3(5):511–519, 1997.
[9] A. Rosenfeld and A. C. Kak. Digital Picture Processing.
Academic Press, 1982.
[10] M. Shridhar, J. W. Miller, G. Houle, and L. Bijnagte. Recog-
nition of license plate images: Issues and perspectives.
In Proceedings of International Conference on Document
Analysis and Recognition, pages 17–20, 1999.