BCI

Tetiana Aksenova AACÌMP 2008
1
AACÌMP, Kyiv 2008
Tatiana Aksenova
Brain Computer InterIacing
2
Center of Atomic Energy,
Grenoble
3
· LE2S : Laboratory of Microelectronics and
Systems for Health
÷ Embedded systems,
÷ Ìmplantable systems
÷ Methods and algorithms for the modeling and
data processing
÷ Hardware Realization 'on-chip'
· "Clinatec¨
÷ Brain computer interface
brain-computer interface (BCÌ)
direct neural interface
brain-machine interface
direct communication pathway between
brain and an external device.
Rehabilitation: stroke, autism, attention etc.
· Less disabled users
· Healthy users
÷ Gamers
5
BCÌ
·One-way BCÌs -
accept commands from the brain - readout
send signals to the brain (Neuroprosthetics) - write-in
visual, auditory, cognitive
·Two-way BCÌs - exchange information in both directions
proiect at Johns Hopkins Universitv
·Ìnvasive BCÌs
·Partially-invasive BCÌs
·Non-invasive
·Animal BCÌ research
·Human BCÌ research
6
BCÌ 'write÷in' and 'readout'
BCÌ 'readout'
BCÌ 'write-in'
7
· Electrical activity.
÷ EEG (electroencephalography)
÷ ECoG (electrocorticogram)
÷ Single cell neuronal activity
with intracortical electrodes
LFP
· Other technologies - MEG (magnetoencephalography), NIRS (near-infrared spectroscopy),
and fMRI (functional magnetic resonance imaging), PET (Positron Emission Tomography)
fMRÌ , NÌRS based on hemodynamic responses (dynamic regulation of the blood flow in the
brain).
BCÌ 'readout'- Signal Acquisition
MEG fMRÌ
- Non-invasive
- Partially-invasive BCÌs
- Ìnvasive BCÌs
3 cm
0.1cm
microns
8
Electrodes
noninvasive BCÌ
Ìnvasive BCÌ
·Scalp based EEG electrodes
·Electrodes placed on the subdural surface of cerebral cortex
·Ìmplanted intracortical microelectrodes
Cerebral cotrex
9
Cortex sensorimoteur
10
Signal characterization
· EEG (recording from the area of centimeters)
÷ Low spatial resolution
÷ Narrow band
÷ Artifacts
÷ Low signal-to noise ratio
÷ Limited specificity
· Single cell recordings (recording from the area of microns)
÷ The problem of stability for long-term (degradation due to formation the scar
tissue around the electrode tips)
· ECoG ÷ balance ?
11
Electrophysiological sources of control
·Changes in spiking pattern
·Response to the external stimuli
÷ Evoked potentials (20 msec)
· VEP (visual evoked potentials)
· Auditory evoked potentials
· Somatosensory evoked potentials ,
·Action related changes
÷ P300 (potential 300 msec),
÷ ERD/ERS
·Event related desynchronization ERD
Amplitude attenuation of mu and beta rhvthms over
contalateral hand area
·Event related synchronization ERS
An increased amplitude of mu and beta rhvthms over
ipsilateral hand area
Mu waves - of 8-13 Hz
High Beta Waves (19Hz+); Beta Waves (15-18Hz); Low Beta Waves (12-15Hz).
12
· Synchronous BCÌ - the onset of activity is externally-paced (with stimuli),
signal has to be analyzed in predefined time windows
Stimuli are demonstrated several times to average responses and to
improve signal to noise ratio
· Asynchronous BCÌ - continuous analyzing of the signal and single
response detection at unknown time moment
Operating Modes
Features extraction from raw signals depends on the technology of data
acquisition and operation mode (Synchronous or Asynchronous BCÌ)
13
Synchronous BCÌ
Stimuli-feedback
matrix speller
BCÌ synchronous - response of the brain to the stimuli.
14
Asynchronous BCÌ
· Continuous Signal processing
· Different tasks can be used for the BCÌ
÷ Ìmagery movement
÷ Mental tasks,
÷ Ìmagination of the mouvements of the objects,
÷ Ìmagery sounds and etc.
· Asynchronous BCÌ is based on the
Plasticity of the brain
15
· BCÌ includes 2 Stages :
÷ Learning
÷ Real time application
16
BCÌ ÷ learning stage
Model or classifier adjustment according to the user
(electrodes, frequency bands, coefficients of controller etc.)
User learning to control voluntary neuronal activity using
neuronal plasticity of the brain
The BCI and user form two adaptive controllers, successful operation is the result
of adequate adaptation of each svstem.
Effectiveness of control and the Time of learning
17
Feedback at close loop BCÌ
Learning to operate a BCÌ using a feedback
·Discrete feedback
÷ Binary - on/off indication of success
·Continuous feedbacks
visual
÷ a cursor on a screen
÷ vertical bar may change in height.
÷ picture of a hand as the subject imagines his own
hand executing movements
÷ Vertual reality
÷ Prosthetics
Combining BCÌ and virtual reality
(VR) technology
18
Feedback with electrical stimulation ?
by using electrical brain stimulation to deliver both 'virtual' tactile cues and
rewards to freely roaming rats, people have been able to instruct animals
remotely to navigate through complex mazes and natural environments .
(Talwar et al., 2002)
19
Brain plasticity
· BCÌ related study
(For ex. Jackson et all, 2006; Velliste et all, 2008)
Reprogramming of the brain
20
Jackson et all, 2006
21
Signal Preprocessing
to improuve signal-to-noise ratio
· Filtering (temporal and spatial)
· Artifact filtering
· PCA
· Methods of source analyses
·ÌCA
·Ìnverse methods
Mainly for non invasive EEG based BCÌ due to the quality of signal
22
classification or modeling
controller
Classification or regression techniques are used depending
on the quality of the signal
÷ Regression for invasive BCÌ
÷ Mainly Classification for nonivasive
Model or classifier is used to produce the control command
Features extraction from raw signals depends on
the technology of data acquisition (EEG, ECoG, action potential of
single neuron recording) and
operation mode (Synchronous or Asynchronous BCÌ)
23
Ìnvasive BCÌs
· Electrods are implanted directly into the the brain during neurosurgery
· Electrods are used for electrical stimulation or recording
· Ìnvasive devices produce the highest quality signals of BCÌ devices
local field potential , multiunit NA
· Scar-tissue build-up causing the signal to become weaker or even lost
Cerebral cotrex Ìntracortical electrods array ÷ visual cortex, motor cortex
Subcortical electrods
an implant in the lateral geniculate nucleus for visual
prosthesis, Harvard Medical Scool (John Pezaris and Clay
Reid)
(a) Utah/Cyberkinetics Array;
(b) Close-Up;
(c) Polyamide 'bioactive'Array;
(d) 256-shank (1024 electrode) Michigan Array
24
Ìnvasive BCÌs readout
artificial vision (animal research)
· Ìn 1999, decoding neuronal firings to reproduce images seen by cats. 177 brain cells
in the thalamus
Garrett Stanley, Harvard University
Garrett Stanley's recordings of cat vision using a BCÌ
implanted in the lateral geniculate nucleus
The lateral geniculate nucleus of the thalamus is a part of
the brain. which is the primarv processor of visual
information
(top row: original image; bottom row: reconstracted)
25
Schalk, BCÌ2008
26
Ìnvasive BCÌs (write-in)
artificial vision (human research)
Ìn 2002-05, series of 16 patients received artificial vision
implant at visual cortex (Dobelle)
Resolution of the implants in 2002 was 144
elerctrods (72 for each of the two brain
hemispheres).
Ìn 2003 the resolution was 484 electrodes, 242
electrodes for each hemisphere, matrix 15 by 16.
ArtiIicial Vision Ior the Blind by Connecting a Television Camera
to the Visual Cortex
Wm. H. Dobelle (ASAIO Journal 2000)
Electrodes are used for electrical
stimulation according to the signal
from camera after signal processing
procedure
27
Stimulation Parameters
· Stimulation delivered to each electrode to produce frame of the
image
÷ train of six pulses at 30 Hz.
÷ frame rates - 4 per second
· Each pulse is symmetric, biphasic (-/+) with a pulse width of 500 sec
per phase (1,000 sec total).
· Threshold amplitudes of 10-20 volts (zero-peak) (may vary +/-20%
from day to day)
· The system is calibrated each day
28
The complete artificial vision system showing the computer
and electronics package on the belt with output cable to the electrodes
on the brain. (2000)
Dobelle et all. ASAIO Journal 2000
29
Jens Naumann, 2002
Dobelle implant had permitted him to see rough
images of large objects
Ìn February of 2003, the implants started to fail, and
he was forced to have them removed .
Other patients use the system with different degree
of success.
Visual perception is still limited
30
artificial vision (human research)
· Learning Retinal Ìmplant System, containing a 50-electrode device
·2006 - 2 patients
an implant is surgically placed into the eye of
patients that provides input to the retinal nerve cells
·pair of spectacles containing an integrated mini-camera and transmitter components
for wireless signal and energy transmission
·the spectacles are connected via a cable to "The Pocket Processor" at the patient's
waist that provides information processing function
31
Retinal Ìmplant
Vince (Feb 2007)
32
Auditory implant
Donoghue, BCÌ2008
33
Ìnvasive BCÌs focusing on
motor neuroprosthetics (readout)
animal BCÌ
Signals recording from monkey and rat cerebral cortexes
· Rats, early 90s, the control of external manipulators
· Monkeys
÷ navigation computer cursors on screen (2D)
÷ commanding robotic arms: reaching and grasping
Lebedev et all., 2006
34
Ìnvasive BCÌs focusing on motor
neuroprosthetics
An example- Monkey reaching and grasping
5 Multielectrod arrays (16-64 electrods)
at M1 (motor cortex) , PMd (dorsal premotor cortex), SMA (suplementary motor aria)
S1 (primery somatosensory cortex), PP (posterior parietal cortex)
35
Signal processing
· Signal acquisition
Multichannel multiunit recording of neuronal activity
(action potentials) or LFP
Donoghue, BCÌ2008
36
Spike detection and sorting
Spike trains
Patterns of neuronal activity (firing rate)
Modeling ÷ MÌMO filters
Signal processing
) ( ) ( ) (
0
t t a a t

) ( ) ( ) (
0
t t a a t

Spike trains
37
Vector of velocity depend on vector of neuronal firing rates
(control based on the analyses of the activity of 50-200
neurons)
Time delay for the command ~ 100ms
Monkey reaching and grasping
) ( ) ( ) (
0
t t a a t J

Velliste et all, 2008
Lebedev et al, 2006
38
Spike rasters of 116 units used for control,
(red, x; green, y; blue, z; purple, gripper).
The x, y, and z components, of robot position
Velliste et all, 2008
Preferred Direction J. Donoghue, 2001
39
Ìnvasive BCÌs focusing on
motor neuroprosthetics
(human research)
2004
BrainGate, brain implant system, bio-tech company Ciberkinetics,
Department of Neuroscience, Brown University, Professor John
Dologhue
three patients have been implanted (patients with ALS or spinal cord
injury)
chip with 100 electrodes
1998 - first brain implant in human
Emory University, Atlanta, Philip Kennedy and Roy Bakay
Patient with locked-in syndrome' after brain-stem stroke.
control a computer cursor
40
· Matthew Nagle (1980-2007) paralyzed from the neck down
· implanted on 2004
· 96-electrods array «Utah array» placed over the region of motor cortex that
controlled his left hand and arm
· control a computer "mouse" cursor, using it to press buttons to control TV,
check e-mail, and so on.
· 4 patients
BrainGate
system
41
J. Donoghue, 2001
42
Non-invasive BCÌs
Use the phenomena of plasticity of the brain
· extensive training required
Electroencephalography (EEG)
Mid of 90th
Data acquisition system :
16 -128 electrodes
Sampling rate 64 Hz - 38.4 kHz.
Simulink ( Matlab )
43
Schalk, BCÌ2008
44
Preprocessing
Spatial filtering
Time frequency filtering
Noise normalization
PCA
Ìndependent component analysis ÌCA
Ìnverse methods for source analysis
Brainwaives with
Multichannel EEG
45
Features extraction
Features
· Single of multiple channel Band powers
· Power spectral density values
· Autoregressive parameters
· Wavelet transform parameters
· Ìnverse model based features (ex. Matrix of cortical current density)
· Row EEG and EEG time derivatives
Spatial information
Multiple channel descriptors
·channel interconnection
ex. graph of correlation
·spatial synchronization and desynchronization
·spatial complexity
Temporal information
·Concatenation of features from
different time segments into a feature
vector (high dimensionality)
·Combination of classifications at
different segments
·The use of dynamic classifier
46
Classification
· All type of classification techniques
Combination of classifiers
47
· Synchronous BCÌ ÷ according to classification
· Asynchronouse BCÌ - poorlv developed
÷ According to Classification
Ìf (t) Class 1 u(t) ÷0
Ìf (t) Class 2 u(t) ÷1
Problem of false positive (6-25 º)
÷ According to regression equation
u(t) ÷ w
i
feature
i
(t)
Adaptive controller
x(t) ÷ a
i
feature
i
(t)
v(t) ÷ b
i
feature
i
(t)
Control command
Wolpaw et all. 2004
48
Wolpaw and McFarland (2004)
49
EEG based BCÌ
Pfurtscheller et al (2003)
50
Games
Emotiv Systems
51
ECoG based BCÌ
·signal magnitude is five times larger (0.05 -
1.0 versus 0.01-0.2 mV for EEG),
·spatial resolution is much finer (mms versus
cms for EEG) (depending on
·electrodes)
·frequency bandwidth is higher (0-500 Hz
versus 0- 50 Hz for EEG).
·ECoG is more robust compared to EEG
·May provide stable recording for months
·Safety ÷ manageable risk
Schalk, BCÌ2008
52
Schalk, BCÌ2008
53
· BCÌ study in humans
Leuthardt(2004); Wilson(2006); Leuthardt(2006); Felton(2007); Schalk(2007);
Schalk(2008);
University of Wisconsin, Madison. BCI R&D Wadsworth center, Albany
÷ Of line study. Prediction of hand movement
÷ BCÌ 1D, 2D cursor movement.
· 1D 75-100% of success
Leuthardt(2004);
· 2D 53-73% of success
Schalk(2008);
· 1-2D accuracy 40-100 % 2-7 days, 45 min per day
Felton(2007);
÷ Features
· mu (8-12 Hz), beta (18÷26 Hz), and gamma (>30 Hz) oscillations
· Row signal
÷ BCÌ technique - the same as for EEG based BCÌ
brief training period in comparison to EEG based BCÌ
with brieI training period oI 3-24 min
· BCÌ related study in monkey
Mehring(2003); Andersen(2004); Rickert(2005)
Of line studv.
Prediction of hand movement
54
Partially invasive ECoG based BCÌ
Electrodes implanted at various parts of cortex
Schalk (2008);
55
Task related changes in ECoG
Leuthardt et al., J Neural Eng, 2004
Leuthardt et al, IEEE T Biomed Eng, 2006
Wilson et al., IEEE T Biomed Eng, 2006
Felton et al., J Neurosurg, 2007
Schalk, BCÌ2008
56
Two-DimensionaI BCI ControI
Schalk, BCÌ2008
57
Schalk et al., J Neural Eng, 2008
Two-DimensionaI BCI ControI
Subj D (actual movement) Subj E (imagined movement)
vertical control horizontal control vertical control horizontal control
(tongue) (hand) (tongue) (hand)
58
Movement joystick prediction
Schalk, BCÌ2008
59
Schalk, BCÌ2008
60
BCÌ Speech
Schalk, BCÌ2008
61
· projects Open Source.
÷ Biosig, platform (Matlab and Octave)
÷ BCÌ2000, plateform BCÌ
÷ BCÌ-info, international platform of the BCÌ
information
Competitions (ÌV-th BCÌ competition is open)
62
BCÌ competition ÌV
BCÌ Competition ÌV please visit
http://ida.first.fraunhofer.de/projects/bci/competition_iv
Ex. Problem ÌV
63
Cited direction of future development
· Light Reactive Imaging for BCÌ
laser implanted inside the skull, trained on a single
neuron
· Devices to measure neurotransmitters release (spatial
resolution 200 m, temporal resolution ÷ 1 sec)
· Fine wires placed in the brain vasculature
· Stimulation devices based on ultrasound or microwaves
· Neuronal axons stretched up to several sm
· Biocompatible polymers with penetrating nanotubes
· Neurotransmitter based stimulation
· Optical stimulation of targeted genetically modified cell
types
64
· Stony Brook University and Avery Biomedical Devices Ìnc. will reportedly continue development of the
Dobelle brain implant
· An approach similar to the Dobelle brain implant is pursued in a European research project
named CORTÌVÌS, which aims to demonstrate the feasibility of an electrode array implant to make an
interface with the visual cortex of blind people - to create a "Cortical Visual Neuroprosthesis".
· Related is also the Neural Prosthesis Program (NPP) at the National Ìnstitute of Neurological Disorders
and Stroke (NÌNDS, part of the NÌH),
· the Utah Electrode Array (UEA, a cortically based visual neuroprosthesis system) at the University of Utah
(Dick Normann),
· the cortical neuroprosthesis project at the University of NSW, Australia (Vivek Chowdhury, John Morley
and Minas Coroneo),
· the work on an intra-cortical visual prosthesis at the Ìllinois Ìnstitute of Technology (Phil Troyk),
· the work towards a cortical implant for artificial vision at the Ligon Research Center of Vision (Raymond
Ìezzi and Pat McAllister, Kresge Eye Ìnstitute, Wayne State University and Detroit Medical Center), and
· the development of an implant for artificial vision at NÌDEK in Japan.
Work towards a visual prosthesis through an implant in the lateral geniculate nucleus (LGN) is performed at
· Harvard Medical School (John Pezaris and Clay Reid; related to the above-mentioned earlier research by
Garrett Stanley).
Companies that develop electrode array based invasive brain interfaces i
· A brain implant similar to the Dobelle implant was developed by Mohamad Sawan and the PolyStim team of
the Ecole Polytechnique de Montreal, Canada, and it was called the Electronic Visual Cortex Stimulator
(EVCS).
· Cyberkinetics Neurotechnology Systems, Ìnc. (BrainGate interface) and
· Neural Signals, Ìnc. (Brain Communicator).
· Pnina Dan of OSM-DAN Ltd, a consultancy company in Rehovoth, Ìsrael, aims to apply TMS or rTMS
(repetitive Transcranial Magnetic Stimulation) as a visual prosthesis for stimulating the visual cortex. Her
work may require an increased spatial resolution of rTMS as well as a decrease in risks of developing
seizures during prolonged semi-continuous use.
· The approach is related to the non-invasive brain stimulation targetted by companies like Nexstim

BCI

Enviado por

Dados do documento

Descrição original:

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

BCI

Enviado por

Direitos autorais:

Formatos disponíveis

Tetiana Aksenova AACÌMP 2008

Você também pode gostar