California Car License Plate Recognition System

ZhengHui Hu Advisor: Dr. Kang

Introduction
A

License Plate Recognition System (LPRS) is a system to automatically detect, recognize and identify a vehicle plate. It involves low-level image processing techniques with higher level artificial intelligence techniques.

Applications
Mainly

for monitoring, surveillance and security. For example,

Entrance/Exit monitoring for parking lot

structures Part of surveillance system for gated communities Control gateways for vehicle passage Security Systems for high traffic
Law

Enforcement

Challenges
Image

Capturing

Vehicle speed Lighting condition Occlusion

Processing

speed accuracy

Heavy traffic

Recognition

High correctness

Recognition Stages
Plate
Plate Localization

Localization Segmentation

Locate plate region out of

car and/or background

Character
Character Segmentation

Segment each

character/number out of plate

Character
Character Recognition

Recognition

Recognize each character

on the plate Similar to OCR process

Previous Work
Many

difference solutions have already been proposed for each stage of recognition Plate localization
Use edge statistics to locate the plate Fuzzy clustering algorithms

Previous Work
Character Segmentation Vertical/horizontal projection Adaptive Clustering Optical Character Recognition Template matching Neural network Feature analysis

Approach
Target:

California State License

Plates
The word California appears at the top-center on the plate in red and italic. The plate number starts with a digit(0-9), followed by three English characters (AZ) and three more digits (0-9). The plate background is a light shade of gray while its characters are of dark blueish color.

Input Images
Captured

using a digital camera

Different distance Different lighting

conditions Different angles

Original

size 2048X1536 Resized to 800X600 for faster process

Base Knowledge
No

assumption on size nor the possible location. Helpful knowledge

light background color and dark foreground rectangular shape with same width and height proportion high in edge concentration

Plate Localization
Noise Filtering and Brightness normalization Extract Edge Information Filter using Color and Edge Information Connected Component Analysis Candidate(s) Found Continue to Character Segmentation > Filter threshold Failed

Noise Filtering and Brightness Normalization

Noise

Filtering by application of 3X3 Gaussian filter

Brightness normalization throgh histogram equalization on Intensity level

Extract Edge Information

Create

a difference image using equation

img =
threshold(close(src)-src)

High

frequency features are enhanced by the operation

Filtering
Select

plate background color using two criteria

Next to edge pixels

detected in previous step A light shade of gray

Connected Component Analysis

Create

connected components Find candidate regions

High concentration

of edge pixels Width/Height ratio similar to plate

Candidate Regions
Multiple

candidate regions can be found at this stage All of them will be submitted to next stage

Character Segmentation
region unprocessed ? Filter and Threshold Find components of similar size Crop Picture <2 > 2 and < 7 Count Components 7 Resize components Continue to Neural Network Recognition Interpolate boxes No Failed

Filter and Threshold

Color filter the plate to remove foreign elements Apply inverse binary threshold using the following function
f img ij=255, if img ij threshold 0, otherwise

Character Extraction

Find a series of boxes with similar shape and size. These are the individual characters If number of boxes found is less than 7 then an interpolation is performed

Character Recognition

Two neural networks are used.

One for recognition of digits (0-9) One for characters (A-Z)

Both networks were trained by using a hybrid method combining traditional Back-propagation algorithm with a Simulated Annealing process

Artificial Neural Network

Artificial Neural Networks (ANN) are modeled after the human brain Network of processing units called Neurons Good for solving classification problems

Input Layer

Hidden Layer

Output Layer

Neurons

Neurons process information by receiving and firing signal according to internal function Two different types are used

Threshold based step function

f =1, if wij threshold 0, otherwise

Sigmoid function
1 f= 1et

ANN Training

ANN must be trained by example before use. Supervised Training

ANN receive set of Input data, and output ANN is adjusted according to the error produced Repeat with different set of data

Back propagation (B-P) algorithm is a classical supervised training method

Limitation of B-P training

Same limitation of any gradient descent algorithm

Global Optimum

Lengthy flat plateau travel Local minima trap

Flat Plateau

Local maximum

Simulated Annealing (SA)

Probabilistic heuristic for locating global optimum in large search space Invented by Kirkpatrick, S., Gelatt, C.D., and Vecchi, M.P. in 1983 Inspired by metallurgic annealing process in which metal is cool down gradually to get the best configuration Inner random selection allows it to escape from local minima trapping

Hybrid B-P SA Training

Create new ANN, as training candidate Train candidate < threshold OR T > 0 Find neighbors Train neighbors NN with least error is new candidate End Training

reduce T

Experimental Results
Hardware

used

Pentium 4 1.0 GHz processor and 512

MB of RAM
Images
50 Images were acquired using a digital

camera

Original image size: 2048X1536 Test image size: 800X600

Experimental Results

Plate Localization
Total Images 50 Plate located 48 Failed to locate 2 Success rate 96%

Character Segmentation
Total Images 48 Character segmented 45 3 93.75% Failed Success rate Cumulative success rate 90%

Experimental Results

Optical Character Recognition Training

5 images as basis for each digit Create 9 variations by adding noise, altering column/rows, distortion, etc..
80% 60%

Recognition rate of digits

Recognition rate on training data Recognition on actual images extracted from LPRS

Experiment Results
Processing

time

Average processing time from image

input to result: ~ 300ms

Conclusion
Preprocessing
Proposed algorithm's performance are

satisfactory
Neural

network training

New method combining Back-

Propagation algorithm with Simulated Annealing process

Future Work
Extend

current system to also recognize uncharacteristic plates and additional character set. Improve recognition ratio by using alternative ANN configurations

References
1.William K. Pratt, Digital Image Processing, Third Edition, John Wiley & Sons, 2001 2.R. Parisi, E.D.Di Claudio, G.Lucarelli, and G. Orlandi, Car Plate Recognition by neural networks and image processing, Proceedings of the 1998 IEEE International Symposium on Circuits and Systems, (ISCAS '98). 3.Leonard G. C. Hamey, Colin Priest, Automatic Number Plate Recognition for Australian Coditions, Proceedings of the Digital Imaging Computing: Techniques and Applications (DICTA 2005) 4.Bai Hongliang, Liu Changping, A Hybrid License Plate Extraction Method Based on Edge Statistics and Morphology, Proceedings of the 17th International Conference on Pattern Recognition(ICPR'04) 5.Choudhury A. Rahman, Wael Badawy, Ahmad Radmanesh, A Real Time Vehicle's License Plate Recognition System, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance (AVSS'03)

References
6.Takashi Naito, TOshihiko Tsukada, Keiichi Yamada, Kazuhiro Kozuka, Shin Yamamoto, Robust License-Plate Recognition Method for Passing Vehicles Under Outside Environment, IEEE Transactions on Vehicular Technology, 2000 7.Rodolfo Zunino, Stefano Rovetta, Vector Quantization for License Plate Location and Image coding, IEEE transactions on Industrial Electronics, Vol 47, No. 1, 2000 8.Mi-Ae Ko, Young-Mo Kim, License Plate Surveillance System Using Weighted Template Matching , Proceedings of the 32nd Applied Imagery Pattern Recognition Workshop (AIPR' 03) 9.Feng Yang, Zheng Ma, Vehicle License Plate Location Based on Histogram and Mathematical Morphology, Proceedings of the Fourth IEEE Workshop on Automatic Identification Advanced Technologies (AutoID'05) 10.Shyang-Lih Chang, Li-Shien Chen, Yun-Chung Chung, Sei-Wan Chen, Automatic License Plate Recognition, IEEE transactions on Intelligent Transportation Systems, Vol 5, No. 1, 2004

References
11.Juntanasub, R., Sureerattanan, N., Car license plate recognition through Hausdorff distance technique, Proceedings of the 17th IEEE International Conference on Tools with Artificial Intelligence, (ICTAI' 05) 12.Timothy Masters, Advanced Algorithms for neural networks?, John Wiley & Sons, 1995 13.Cornelius T. Leondes (Editor), ?Algorithms and Architectures (Neural Networks Systems Techniques and Applications), Academic Press, 1998 14.David E. Rumelhart, Geoffrey E. Hinton & Ronald J. Williams, Learning representations by back-propagating errors, Nature 323, 533 - 536 (09 October 1986) 15.Kirkpatrick, S., Gelatt, C.D., and Vecchi, M.P., Optimization by Simulated Annealing, Science, Volume 220, Number 4598, 13 May 1983, pp. 671680.