Eye/gaze tracking in video; identify

the users focus of attention

oMihaela Romanca Technical University of Cluj-Napoca

oPeter Robert - Technical University of Cluj-Napoca
oVilius Matiukas - Vilnius Gediminas Technical University
oBrigitta Nagy University of Debrecen
Introducing the team

SSIP 2009 2
 Mihaela Romanca
Student from Technical
University of Cluj-
Napoca

Hobbies: Sports and

ecology
E-mail:
mihar_bv@yahoo.com

SSIP 2009 3
 Robert Peter

Masters student from

Technical University of
Cluj-Napoca

Hobbies: PC games,
football and
movies/music
E-mail:
p_robi86@yahoo.com

SSIP 2009 4
 Vilius Matiukas
PhD student from Vilnius
Gediminas Technical
University, Faculty of
Electronics, Department
of Electronic Systems

Hobbies: Image Processing

and fishing
E-mail:
vilius.matiukas@el.vgtu.lt

SSIP 2009 5
 Brigitta Nagy
Student from University
of Debrecen, Faculty of
Informatics

Hobbies: Image processing,

Wing-Tsun Kung-fu, Reading
and Puzzles
E-mail:
stefanie867@yahoo.co.uk

SSIP 2009 6
 Test subject
Uneducated peace of paper

Hobbies: Staring at the

same direction.
Address: computer
laboratory

SSIP 2009 7
 Problem Description

Input: video of a user sitting in front

of the computer
Goal: Detect the focus of attention
and the modification of the region of
interest of the user.

SSIP 2009 8
Equipment and software
Genius Slim 321c webcamera.
Language: C#
IDE: Microsoft Visual Studio 2005
EMGU CV: Wrapper for C# of
OpenCV

SSIP 2009 9
 Tasks to do
1. Face Detection
2. Detection of the eyeregion
3. Pupil Detection
4. Eye Corner Detection
5. Determine the focus of attention

SSIP 2009 10
 1. Face Detection
We used Haar-like features for
face detection.
Haar-like features are digital image
features used in object recognition.

Then we reduced the face region

and split it to region of eyes.

SSIP 2009 11
 Example
Our test subject

SSIP 2009 12
2. Detection the region of the
eye
To detect the region of eyes we
used also Haar-like features.
Then contrast enchancement on
the detected eye region was
applied.

SSIP 2009 13
 Example
The test subject

Approximation With Haar-like features

SSIP 2009 14
 3. Pupil detection
Circular Hough transformation was
applied for detection of the pupil.
The Hough transform is a feature
extraction technique.
The classical Hough transform was
concerned with the identification of
lines in the image, but later the Hough
transform has been extended to
identifying positions of arbitrary shapes,
most commonly circles or ellipses.

SSIP 2009 15
4. Eye Corner Detection
We choose the one closest
corner to the nose.

SSIP 2009 16
 Calibration for gaze
detection
Wait until the user sits in a position, where
80% of the frames detect the iris center
and the corner also.
Put circles in the center and the four
extremities of the screen, and wait until at
least 15 pupil and eye corners are detected
in both region of eye.
Calculate the average of eye corner and
center coordinates in all the positions
(center, topleft, topright).

SSIP 2009 17
5. Focus of attention
As the users moves the eyes the
mouse cursor moves in the
corresponding direction.

SSIP 2009 18
 Statistics
Variance Variance Variance Variance Variance
Spot (pixel^2) (pixel^2) (pixel^2) (pixel^2) (pixel^2)
Test1 Test2 Test3 Test4 Test5
Left pupil 437 309 292 300 314
Left corner 577 555 567 570 556
Right pupil 346 288 341 294 296
Right corner 256 293 274 178 220

The numbers represent the variance of coordinates

during calibration.

SSIP 2009 19
 Future development
Imitating left and right mouse
clicks with blink detection
Recognition even when face is in
different angle
Expression detection for
different focus regions
Higher precision for full control
for people with disabilities
SSIP 2009 20
Thank you!

SSIP 2009 21