PSYCH 221 Automatic License Plate Recognition

Max Guise Stephan Goupille

Team Members
Max Guise
Electrical Engineering
Technical Staff, Sandia National Labs

Stephan Goupille
MS&E, Electrical Engineering Growth Analyst, Facebook

Project Aims
Image
Balancing

First, the several color transformations and balancing schemes will be examined, including adaptive histogram equalization and using the different color spaces (CIELAB).

Plate Extraction

Image patches that contain license plates will be extracted, based on morphological operations, oriented gradients, edge detection, filtering, and erosion.

Finally, character recognition will be performed on these regions of interest and decision criteria will be devised to choose when to Character output a result. Recognition

Project Aims - Adapted

Character recognition particularly difficult and requires a massive database of license plates Focus on license plate extraction:

Tools
Motorola Droid Phone
Great for snapping a quick picture Originally thought RGB values could be exported directly (function hasnt been implemented)

MATLAB
One of the best, fastest mathematical analysis tools Many built-in image processing functions

Other ALPR Systems

Most systems in actual use benefit from a controlled scenario:
Lack of background objects
Higher quality imaging Better flash (license plates are retroreflective) Minimal scaling and rotation

Ease of access to large database from which to build a model

Typically use a machine learning algorithm (SVM, LDA, etc.) to make final decisions

Our goal to accomplish task using basic properties of license plates no machine learning necessary

Methods
Pre-Processing
KMeans

Adaptive Histogram Equalization

Image Transformation
Histogram of Oriented Gradients Harris Corners Hough Transform Edge Detection Filtering

Decision-Making
Thresholding Erosion

KMeans
Unsupervised Nearest Centroids in 5-Space (X,Y,R,G,B)
Variable number of centers can be chosen Can weight X,Y differently from R, G, B

Goal: License plate shows up as one block

KMeans
Issues:
Optimal number of centers to use varies with size of license plate
As a result, cant guarantee good separation of plate from bumper, or might split license plate region into multiple parts

Can be computationally expensive either in itself or later in the pipeline

Histogram Equalization
Image pixels may be far from uniformly spread across the display range
Changing the distribution can add contrast

Can be done across whole image, or adaptively

Tiling across image might split up an individual license plate region
10 x 10
5

Original
6 5

x 10

Equalized
10

x 10

Adaptive

4 6 3 4 2 2 2 4 6

100

200

300

100

200

300

100

200

300

Histogram Equalization
200 400 600 800 1000 1200 1400 1600 1800 500 1000 1500 2000 2500

10

Histogram Equalization
200 400 600 800 1000 1200 1400 1600 1800 500 1000 1500 2000 2500

11

Histogram Equalization
200 400 600 800 1000 1200 1400 1600 1800 500 1000 1500 2000 2500

12

Histogram of Oriented Gradients

Dalal Triggs 05 paper Objects can be described by edge orientation
Compute basic gradient

13

Since we have data in two directions, can calculate a direction for each pixel

Also snap to regular grid 45 degree angles

Create histogram of these directions for a given window size Computation limitation here would need to search at various scales

Biggest problem here is that edges of plates themselves not always well defined
Illumination changes, license plate holders, glare, etc.

Bottom line: need machine learning technique here to make this work

Harris Corner Detector

Detects interest points in image
Based on eigenvalues of sensitivity matrix

14

Our idea was to search the detected Harris corners for rectangles of approximately the right aspect ratio
Turned out to be an incredible computational burden with the detector sensitivity high enough to reliably detect license plate corners
100 200 300 400 500 600 700

10 20 30 40 50 60 70

800

80
900 200 400 600 800 1000 1200

20

40

60

80

100

120

140

Hough/Radon Transform
Canny edge detection to get edge map For each edge pixel, record the set of all lines that could pass through it
Parameterized by distance from origin and angle

15

Resulting image has peaks at (,) pairs representing prominent lines in the image
Idea was to search for peaks corresponding to the approximate aspect ratio of license plates

Again, edges of plates not always well defined

Meant that tuning the thresholds resulted in a detector that delivered an incredible amount of false positives

Hough/Radon Transform
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

16

Simplest is best
Underlying insight is that license plates have a lot of vertical edges (letters) and a specific aspect ratio
Take horizontal gradient Filter with matched filter Designed with aspect ratio, approximate license plate size in mind Threshold Bit of an art ties in closely with the erosion

17

Erode
Region label Throw out regions that are too small Cascaded false positive rejection

Rely on context of license plate (car bumper, license plate holder etc.)

Horizontal Gradient
100

18

200

300

400

500

600

700

800

900

200

400

600

800

1000

1200

200 400 600 800 200 1000 400 1200 600 1400 800 1600 1000 1800 1200
1400 1600 1800

Matched Filtering

19
1 0.5 0 -0.5 -1 40 30 20 10 0 0 20 40 60

500

1000

1500

2000

2500

500

1000

1500

2000

2500

100 200 300 400 100 500 200 600 300 700 400 800 500 900 600
700 800

200

400

600

800

1000

1200

Threshold, Erode, Label

20

False Positive Rejection

Expand candidate license plate window and look for prominent horizontal lines
Reject if we dont find any bumper or license plate holder typically shows up, and we interpret prominent too stringently

21

Conclusions
Detection without learning = hard
Lots of issues: background clutter, contrast, scale/rotation invariance, computation, etc.

22

Color space changes didnt seem to help

Tried various transformations of RGB and LAB coordinates as well as scale-by-max color balancing

Next steps would be window refinement, exploration of scale and rotation capabilities (and, of course, character segmentation and recognition) Thanks!

References
Dalal, N. and Triggs, B. 2005. Histograms of Oriented Gradients for Human Detection. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cvpr'05) - Volume 1 - Volume 01 (June 20 - 26, 2005). CVPR. IEEE Computer Society, Washington, DC, 886-893. Vahid Abolghasemi, Alireza Ahmadyfard. An edge-based color-aided method for license plate detection. Image and Vision Computing, Volume 27, Issue 8, 2 July 2009, pp. 1134-1142. Stokman, Harro and Gevers, Theo. Selection and Fusion of Color Models for Image Feature Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 29.3 (2007): 371-381.

23

Questions?

24