Escolar Documentos
Profissional Documentos
Cultura Documentos
Human
Operator
Monitoring
System
•Motivation
•What is
BlueEyes ?
BlueEyes
•Physiological Human-Operator Monitoring System
foundations
•System
design
•System Team members Paweł Kowalik
implementation
•Future
Piotr Kubiaczyk
improvements Krystian Nowak
•Project work Stanisław Osiński
summary
•System
Tomasz Pużak
demonstration
Team mentor Jan Kniat, Ph. D.
Agenda
Monitoring
System
•Motivation
•What is
BlueEyes ?
• Motivation
•Physiological
foundations • What is BlueEyes (and what is not) ?
• Physiological foundations
•System
design
•System
implementation • System design
•Future
improvements
•Project work
• System implementation
summary
•System
• Future improvements
demonstration
• Project work summary
• System demonstration
BlueEyes
Human
Operator
Motivation
Monitoring
System
•Motivation
•What is • Human error - a frequent reason of
BlueEyes ?
•Physiological catastrophes and ecological disasters
foundations
•System – temporal indisposition
design weariness
–
•System oversight
–
implementation Monitoring of conscious
•
brain involvement
•Future
improvements Capable sensors available
•
•Project work
summary
•System
demonstration
What is BlueEyes ?
Monitoring
System
•Motivation
•What is • The complex solution for human-operator
BlueEyes ?
•Physiological monitoring:
foundations
•System
– conscious brain involvement monitoring
design
•System
– triggering user-defined actions
implementation
•Future
– data recording
improvements
•Project work
summary
•System
demonstration
What is BlueEyes not?
• Doesn’t predict nor interfere with
operator’s thoughts
• Cannot force directly the operator to work
BlueEyes
Human
Operator
•Motivation
•What is • Generic control rooms
BlueEyes ?
•Physiological – Power station
foundations
•System – Captain bridge
design
•System – Flight control centers
implementation
•Future – Operating theatres – anesthesiologists
improvements
•Project work
summary
•System
• Common application
demonstration
– A simpler system version for drivers
BlueEyes
Human
Operator
BlueEyes - benefits
Monitoring
System
•Motivation
•What is • Prevention from dangerous incidents
BlueEyes ?
•Physiological • Minimization of
foundations
•System – ecological consequences
design
•System – financial loss
implementation
•Future – a threat to a human life
improvements
•Project work
summary • The reconstruction
•System
demonstration of the course of
operator’s work
BlueEyes
Human
Operator
Physiological foundations
Monitoring
System
•Motivation
•What is
BlueEyes ?
•Physiological Human brain
foundations
•System
design
•System Preconscious brain Conscious brain
implementation
•Future • alogical • manages the
improvements
•Project work • „automatic” knowledge
summary
•System
• logical reasoning
demonstration
• makes decisions
BlueEyes
Human
Operator
•Motivation
•What is
BlueEyes ?
•Physiological
foundations
•System
design
•System
implementation
•Future
improvements
•Project work
summary
•System
demonstration
Eye movements
Monitoring
System
•Motivation
•What is • Retina - central vision (macula)
BlueEyes ?
•Physiological • Central vision and
foundations
•System the conscious brain
design
•System
implementation Saccade - a rapid
•Future
improvements eye jump to a new
•Project work
summary
location assigned
•System by the conscious
demonstration
attention process.
•Motivation
•What is • Eye position measuring - direct infrared
BlueEyes ?
•Physiological oculography
foundations
•System
• Oxy- and deoxyhemoglobin measurment
design
•System
• Two axial
implementation
accelerometer
•Future
improvements • Ambient light
•Project work
summary sensor
•System
demonstration
Jazz Multisensor
BlueEyes
Human
Operator
•Motivation
•What is
BlueEyes ?
•Physiological
foundations A serious limitation of:
•System
design – mobility
•System
implementation – comfort
•Future
improvements – safety
•Project work
summary
•System
demonstration
BlueEyes
Human
Operator
•Motivation
•What is
BlueEyes ?
•Physiological
foundations
Solution:
•System
design
•System
A wireless Personal Area
implementation
•Future
Network
improvements
•Project work
summary
•System
demonstration
BlueEyes
Human
Operator
Design objectives
Monitoring
System
•Motivation
•What is • A mobile data acquisition device
BlueEyes ?
•Physiological • A reliable real-time buffering, processing
foundations
•System and recording
design
•Objectives • A clear visualization interface
•System
overview • Mechanisms for creating custom data
•DAU analyzers
•CSU
• To ensure a possibility of distributing the
•System
implementation processing among a number of computers
•Future
improvements
•Project work
summary
•System
demonstration
BlueEyes
Human
Operator
System overview
Monitoring
System
•Motivation
•What is Data Acquisition Unit Central System Unit
BlueEyes ?
•Physiological
foundations
Atmel 89C52 Connection Visualization
•System Bluetooth Bluetooth Manager Module
design microcontroller
•Objectives
Jazz Data
•System Data Analysis
Multisensor Logger
overview
•DAU
•CSU
DAU - features
Monitoring
System
•Motivation
•What is Data Acquisition Unit
BlueEyes ? • Lightweight
•Physiological
foundations • Runs on batteries - low
Atmel 89C52
•System
design microcontroller
Bluetooth power consumption
•Objectives
Jazz • Easy to use - does not
•System
overview Multisensor disturb the operator
•DAU
working
•CSU
• ID cards for operator
•System
implementation authorization
•Future
improvements • Voice transmission using
•Project work
summary
hardware PCM codec
•System
demonstration
BlueEyes
Human
Operator
CSU - features
Monitoring
System
•Motivation
•What is Central System Unit
BlueEyes ? • Connection
•Physiological
foundations
management
Connection Visualization
•System
design
• Data processing Bluetooth Manager Module
•Objectives
• Visualization Data Analysis
Data
•System Logger
overview
• Data recording
•DAU
•CSU
•System
implementation
• Access
•Future verification
improvements
•Project work • System maintenance
summary
•System
demonstration
BlueEyes
Human
Operator
•Motivation
•What is • Saccade detector and conscious brain
BlueEyes ?
•Physiological involvement level calculation
foundations
Eye movement velocity
•System
Raw eye 250 saccade
design 200
movement data 150
•Objectives 100
Velocity
50
•System 0
-50
overview -100
-150
-200
•DAU -250
•System
implementation
•Future
improvements Conscious
Visual
•Project work
brain
summary involvement
level
•System 60
demonstration
Lva)
50
BlueEyes
Human
Operator
•Motivation
•What is • Raw and processed data visualization
BlueEyes ?
•Physiological using:
foundations
•System – VU-meters
design
•Objectives – pie-charts
•System
overview – time series
•DAU
•CSU
•System
implementation • audio and video
•Future
improvements data playback
•Project work
summary
•System
demonstration
BlueEyes
Human
Operator
Data security
Monitoring
System
•Motivation
•What is • Only registered mobile devices
BlueEyes ?
•Physiological can connect to the system
foundations
•System
• Bluetooth connection authentication
design
•Objectives • Bluetooth connection encryption
•System
overview • Access rights restrictions
•DAU
• Personal and physiological data encryption
•CSU
•System
implementation
•Future
improvements
•Project work
summary
•System
demonstration
BlueEyes
Human
Operator
Implementation - DAU
Monitoring
System
•Motivation
•What is • Made from scratch by the team
BlueEyes ?
•Physiological • Bluetooth module inside
foundations
•System • Jazz sensor
design
•System
interface
implementation
•DAU
• Microphone and
•CSU earphone sockets
•Future
improvements • 5-key keyboard
•Project work
summary • Two lines, 8
•System characters LCD
demonstration
• Powered from
6 AA batteries
BlueEyes
Human
Operator
Implementation - DAU
Monitoring
System
•Motivation
•What is
BlueEyes ?
•Physiological
•DAU
•CSU
•Future
improvements Batteries and
•Project work
summary
voltageBluetooth module
level monitor
•System
demonstration
BlueEyes
Human
Operator
Implementation - CSU
Monitoring
System
•Motivation
•What is • Standard MDI user
BlueEyes ?
interface
•Physiological
foundations • Custom-built
•System
design
Bluetooth
•System communication
implementation
routines
•DAU
•CSU • Intel USB camera
•Future
improvements
• ID card programmer
•Project work
summary
• Developed using Visual C++ 6.0
•System • Deployment environment:
demonstration
Windows 2000,
MS SQL Server
BlueEyes
Human
Operator
Future improvements
Monitoring
System
•Motivation
•What is • DAU
BlueEyes ?
•Physiological – small CMOS camera to monitor the
foundations
•System
operator’s point of gaze
design
•System
– single PCB (SMD technlogy)
implementation
•System
– low voltage ICs - LiIO batteries power
demonstration
•Future
improvements
• CSU
•Project work
summary
– data mining algorithms
– advanced database encryption using e.g.
AES algorithm
BlueEyes
Human
Operator
•Motivation
•What is
BlueEyes ? • Bluetooth and PAN technology acquired
•Physiological
foundations • New skills on designing software and
•System
design
hardware systems gained
•System
implementation
• A mobile device implemented
•System • 31 KLOC of C++ code and
demonstration
•Future 4 KLOC of 8051 assembler code written
improvements
•Project work
• All parts of the Project Kit efficiently used
summary
• New approach to system safety proposed
and implemented
BlueEyes
Human
Operator
Monitoring
System
•Motivation
•What is
BlueEyes
BlueEyes ?
•Physiological
foundations
•System
design
Human-Operator Monitoring System
•System
implementation
•Future
improvements
•Project work
System demonstration
summary
•System
demonstration
BlueEyes
Human
Operator
Monitoring
System
•Motivation
•What is
BlueEyes ?
•Physiological
foundations
Thank you for your attention
•System
design
•System
implementation
•System
demonstration
•Future
improvements
BlueEyes
•Project work
summary
Human-Operator Monitoring System
DAU - components
Monitoring
System
microphone MC 145483
earphone PCM codec
UART
microcontroller
LCD display
UART MAX232
LED indicators
BlueEyes
Data Acquisition Unit
ID card Simple
interface keyboard
ID card
BlueEyes
Human
Operator
Operator
Data Initializes
••Data demultiplexer Bluetooth
Manager modules
•Data producer queues
connections
... Bluetooth ...
Connection • Supports operator
... Manager roaming
...
• Feeds Operator
Raw Data
Manager
Operator Buffer with
Manager
Bluetooth physiological
Alert buffer data
Alerts
BlueEyes
Human
Operator
“Yes”
No
... answer
... Decision tree
“Yes” coefficients
“No”
No
answer
BlueEyes
Human
Operator
Data
consumer
creates
Data
Data Data Data
Processing
consumer & Data
consumer
producer queue producer queue
Data
consumer
Data
Data Data
Processing
consumer &
consumer
producer queue
BlueEyes
Human
Operator
Connection
Manager
Visualization Data
Data Analysis
Module Logger
Processed data