VISVESVARAYA TECHNOLOGICAL UNIVERSITY

Belgaum, Karnataka

Bachelor of Engineering
In
Electronics and Communication

Blue Eye Technology

Seminar Report Submitted By

Avinash.R (4JC05EC015 )
Bhuvan.N (4JC05EC019)

Abstract

Bharath.G (4JC05EC018)
Vikas.M.K (4JC05EC122)

Blue Eye Technology

As the human-computer interaction continues to be enhanced --especially with a targeted commercial rollout
in at least five years--another major stride in facing a new frontier in computer technology has been
achieved.
Is it possible to create a computer, which can interact with us as we interact each other? Blue Eyes
technology is just one of such marvels achieved due to this enhanced interaction and aims at creating
computational Machines with perceptual and sensory abilities like those of human beings but without being
ignorant
Blue Eyes system is thus a versatile system, which can be modified to cater to the Working environment. It
can be applied in every working environment requiring permanent operators attention for it and also
provides technical means for monitoring and recording human-operator's physiological condition. "It will be
capable of working with a number of different types of systems including nonlinear editors, playout stations,
ingest stations. It uses sensing technology to identify a user's actions and to extract key information.
Its just an advanced type of programming made to calculate the emotion (mood) of the person.
In this paper we throw light upon how the blue eye technology and few of its prototypes work as well as
how it has brought about a revolution in this scientific and technological world. This presentation will help
provide a sneak peak into the future of computing.
Introduction
Animal survival depends on highly developed sensory abilities. Likewise, human cognition depends on highly
developed abilities to perceive, integrate, and interpret visual, auditory, and touch information. Human error
is still one of the most frequent causes of catastrophes and ecological disasters. The main reason is that the
monitoring systems concern only the state of the processes whereas human contribution to the overall
performance of the system is left unsupervised. Since the control instruments are automated to a large
extent, a human operator becomes a passive observer of the supervised system, which results in weariness
and vigilance drop. It therefore is crucial to assure that the operator's conscious brain is involved in an active
system supervising over the whole work time period. Without a doubt, computers would be much more
powerful if they had even a small fraction of the perceptual ability of animals or humans.
How can we make computers emotional?
At IBM research a Blue Eye project is being carried out to enhance computers emotional capabilities.
Sensors in the mouse sense the physiological attributes, which are correlated to emotions using a correlation
model. The correlation model derived from a calibration process in which a baseline attribute-to-emotion
correlation is rendered based on statistical analysis of calibration signals generated by users having emotions
that are measured or otherwise known at calibration time. A vector in N dimensions, representative of a
subject user's emotions, is output or subsequent subject users whose emotions are sought to be known, with
the baseline being the reference in the N-dimensional space of the vector. Blue Eyes seeks attentive
computation by integrating perceptual abilities to computers wherein non-obtrusive sensing technology, such
as video cameras and microphones, are used to identify and observe operators actions.

What is Blue Eyes Technology?

The blue eyes is a technology, which aims at creating computational machines that have perceptual and
sensory abilities like those of human beings. In the term Blue eyes, blue stands for Bluetooth, which enables
reliable wireless communication and the Eyes because the eye movement enables us to obtain a lot of
interesting and important information.
2

Blue Eye Technology

The basic idea behind this technology is to give computer human power. For example, we can understand
humans emotional state by his facial expressions. If we add these perceptual abilities to computers, we
would enable them to work together with human beings. It provides technical means for monitoring and
recording human-operators physiological condition. Blue eyes dont predict nor interfere with operators
thoughts. It cannot force the operator to work
Need for blue eyes
Human error is still one of the most frequent causes of catastrophes (calamity) and ecological disasters,
because human contribution to the overall performance of the system is left unsupervised. The control
instruments within the machine have automated it to large extent, thus Human operator becomes a passive
observer of the supervised system, resulting in weariness and vigilance drop, but the user needs to active
because he may not notice important changes of indications causing financial or ecological consequences,
which is a threat to human life. Thus, its crucial that operators brain is involved in an active system
supervising over the whole work time period.
Design of a blue eyes system
The design of blue eyes technology consists of a mobile data acquisition system, a reliable real-time
buffering, processing , mechanisms for creating custom data analyzers, to ensure a possibility of distributing
the processing among a number of computers.
This system consists of 2 major parts:
1. Data Acquisition Unit
2. Central System Unit
DATA ACQUISITION UNIT

Figure1.1: Block diagram of DAU

Components of DAU
Components of data acquisition unit
Atmel 89C52 microcontroller - system core: Carries out various controlling functions as well as helps in data
reception transmission, storage.

Bluetooth module: To connect DAU and CSU

HD44780 - small LCD display: To display the pin code typed

3

Blue Eye Technology

24C16 - I2C EEPROM (on a removable ID card)

Jazz Multisensor interface

Beeper and LED indicators: To indicate if device is on Or off. The LEDs will glow accordingly. Also used for
communication

6 AA batteries and voltage level monitor: This provides the power required by the unit power to work.

5 key keyboard: To carry out communication.

13 bit PCM codec: It is the board onto which the DAU systems are placed.

Figure1.2: DAU equipment

FUNCTIONS OF DATA ACQUISITION UNIT

It is a mobile part of blue eye system.

It maintains Bluetooth connections.
Gets information from the sensor.
Sends information over the wireless connection.
Delivers the alarm messages sent from the Central System Unit (CSU) to the operator.
Handles personalized ID cards.

ID cards

Blue Eye Technology

Figure 2: Id card design etched out on ST16C24 EPROM

We can make the data acquisition unit personalized by means of ID card. The simple board contains
ST16C24 EEPROM on to which the basic design was to be etched as shown in fig 2.A simple programmer
is developed to program the ID cards and it starts with the PCB. This contains a smaller microcontroller
running at 11.0592 MHz, MAX232 for RS232 communication and ID card interface. When the original
mechanical interface was connected to the DAU, it was impossible to get another one and a substitute had to
be used like a few blocking elements along with an ID card reader.
Jazz Multisensor:

Figure 3: Jazz Multisensor

Its an eye movement sensor, to provide necessary physiological data in Data Acquisition Unit (DAU). It
supplies raw digital data regarding eye position, the level of blood oxygenation acceleration along horizontal
and vertical axes and ambient light intensity. Eye movement can be measured using direct infrared
occulographic transducers. The user has to wear this device on his/her forehead.

CENTRAL SYSTEM UNIT:

Blue Eye Technology

Connection
Manager

Visualization
Module

Bluetooth
Data
Analysis

Data
Logger

Figure 4: Central System Unit

The box contains a Bluetooth module (based on ROK101008) and a PCM codec for voice data
transmission. The module is interfaced to a PC using a parallel, serial and USB cable. The audio data is
accessible through standard mini-jack sockets over view of central system unit to program operator's
personal ID cards we developed a simple programming device. The programmer is interfaced to a PC using
serial and PS/2(power source) ports. Inside, there is Atmel 89C2051 microcontroller, which handles UART
transmission and I2C EEPROM (ID card) programming.

Functions of Central System Unit

Central System Unit hardware is the second peer of the wireless connection.
Maintains blue tooth connections in the other side.
Buffers incoming sensor data.
Performs on-line data analysis.
Records the conclusion for further exploration.
Provides visualization interface.

Future improvements
DAU

Small CMOS camera to monitor the operators point of gaze

Single PCB (SMD technology)
6

Blue Eye Technology

CSU

Low voltage ICs Li IO batteries power

Data mining algorithms

Advanced database encryption using e.g. AES algorithm

Types of users
The working of blue eyes depends on three types of users and the users are classified into following three
categories:
Operators
Supervisors
System administrators
Operator
Operator is a person whose physiological parameters are supervised. The operator wears the DAU.
Communication with the operator is carried on using a simple 5-key keyboard, a small LCD display and a
beeper. When an exceptional situation is detected the device uses them to notify the operator. Voice data is
transferred using a small headset, interfaced to the DAU with standard mini-jack plugs. The only functions
offers to the operator are Authorization to the system and receiving alarm alerts.
Authorization: Operator has to enter his personal PIN into DAU, if PIN is accepted, authorization is said to
be complete.
Receiving Alerts: This function supplies the operator with the most important alerts about his and his coworkers condition and mobile device state
Supervisor
He is the person responsible for analyzing operators condition and performance. The supervisor receives
tools for inspecting present values of the parameters (on-line browsing) as well as browsing the results of the
long-term analysis (off-line browsing).
Blue Eyes hardware
A standard 555 timer IC every 10 seconds crosses the middle-button microswitch, which is fully equal to a
"manual" click. To be able to reset properly the Bluetooth module a serial-parallel combo cable was made.
The serial part provides an RS232 communication channel while the parallel part is used to reset the module
and to drive the status LEDs. Another cable is the USB one for powering the device. And two more cables
for audio input and output

Blue Eyes Software

Blue Eye Technology

Figure 5: Software
Blue Eyes software's main task is to look after working operators' physiological condition. To assure instant
reaction on the operators' condition change the software performs real time buffering of the incoming data,
real-time physiological data analysis and alarm triggering. The Blue Eyes software comprises several
functional modules:
1. System Core facilitates the data flow between other system modules (e.g. transfers raw data from the
Connection Manager to data analyzers, processed data from the data analyzers to GUI controls, other data
analyzers, data logger etc.).
2. Connection Manager is responsible for managing the wireless communication between the mobile Data
Acquisition Units and the central system. To increase the reliability of the Connection Manager we have
implemented our own Bluetooth micro-stack which covers a limited set of Bluetooth communication
facilities. The Connection Manager handles:
Communication with the CSU hardware
Searching for new devices in the covered range
Establishing Bluetooth connections
Connection authentication
Incoming data buffering
3. Data Analysis module performs the analysis of the raw sensor data in order to obtain information about
the operators physiological condition. The separately running Data Analysis module supervises each of the
working operators. The module consists of a number of smaller analyzers extracting different types of
information. The most important analyzers are
Saccade detector - monitors eye movements in order to determine the level of operator's visual
attention
Pulse rate analyzer - uses blood oxygenation signal to compute operator's pulse rate
Custom analyzers - recognize other behaviors than those, which are built-in, the system. The new
modules are created using C4.5 decision tree induction algorithm

4. Visualization module provides a user interface for the supervisors. It enables them to watch each of the
working operators physiological condition along with a preview of selected video source and related sound
stream. All the incoming alarm messages are instantly signaled to the supervisor. The physiological data is
8

Blue Eye Technology

presented using a set of custom-built controls:
A pie-chart used to present a percentage of time the operator was actively acquiring the visual
information
A VU-meter showing the present value of a parameter time series displaying a history of selected
parameters' values
Data Security
Only registered mobile devices can connect to the system.
Bluetooth connection authentication and encryption.
Access rights restrictions.
Personal and physiological data encryption.
Devices that work on principle of blue eye technology
The principal of blue eyes technology has been utilized for inventing various devices. Few of the prototypes
have been discussed below.
Pong and Magic
A robot called Pong observes retinal activity and recognizes user presence and responds accordingly with,
a human-like smile or computer-based action like initializing a desktop PC. This "attentive" robot created by
the User group dramatizes computers emerging awareness by sensing peoples presence and reacting to
their comings and goings.
"Pong" utilizes a small video camera to provide attention to its user. This camera is part of another prototype
named Magic pointing, which allows the eyes to affect the movement of the cursor. This functionality can
only allow the cursor to arrive at the point where the user gazes by having him or her touch the pointing
device first. If the cursor were to move where the eyes go, the operation would be unproductive and
impractical, because this leaves the user exhausted and confused.

Emotion mouse
User is developing prototypes to recognize eye movement and also looking at an intelligent pointing device
that directly captures relevant data on the users emotional patterns. The computer can therefore either slow
down, when he determines the user is anxious or tired, or speed up, when he identifies that the user is in a
hurry.
9

Blue Eye Technology

The pointing device is called the emotion mouse, which can track a users mood or emotional state and make
the computer respond accordingly. The device can measure heart rate, temperature, galvanic skin response
and minute bodily movements and matches them with six emotional states: happiness, surprise, anger, fear,
sadness and disgust and includes a set of sensors, including infrared detectors and temperature-sensitive
chips. IR takes heart rate on the thumb,temperature is taken using a thermo-resistor chip. These values are
input into a series of discriminate function analyses and correlated to an emotional state. To be included in
the discriminate function analysis, the proportion of each signal's emotion-specific variance (that is not
accounted for by other non-excluded signals) to total variance must exceed a criterion proportion, which for
the mouse is 0.001. After signals are excluded from the analysis, remaining signals are analyzed
simultaneously to describe the baseline relationship by a number of discriminate functions equal to one less
than the number of emotions sought (N-1) or the number of physiological signals.
These prototypes, among others in its Blue Eyes, provide a sneak peak into the future of computing. As the
human-computer interaction continues to be enhanced especially with a targeted commercial roll-out in at
least five years another major stride in facing a new frontier in computer technology has been achieved
Blue Eyes Benefits
Prevention from dangerous incidents Minimization of ecological consequences financial loss a threat to a
human life Blue Eyes system provides technical means for monitoring and recording human-operator's
physiological condition. The key features of the system are:
Visual attention monitoring (eye motility analysis)
Physiological condition monitoring (pulse rate, blood oxygenation)
Operators position detection (standing lying)
Wireless data acquisition using Bluetooth technology
Real-time user-defined alarm triggering physiological data, operator's voice and overall view of the
control room recording
Recorded data playback

Conclusion
The wireless link between the sensors worn by the operator and the supervising system makes it possible to
improve overall reliability, safety and assures proper quality of system performance. These new possibilities
can cover areas such as industry, transportation, military command centers or operation theaters.
Researchers are attempting to add more capabilities to computers that will allow them to interact like
humans, recognize human presents, talk, listen, or even guess their feelings. It avoids potential threats
resulting
from
human
errors,
such
as
weariness,
oversight,
tiredness.

10