Brain Computer Application Interface

IT10088074

The Brain Computer Interface Application Framework

H.G .Udayanga (IT10088074)
BSc. (Hons) in Information Technology Department of Information Technology Sri Lanka Institute of Information Technology Sri Lanka August 2013

Declaration
1

Brain Computer Application Interface

IT10088074

I declare that this is my own work and this report does not incorporate without acknowledgment any material previously submitted for a Degree or Diploma in any other University or institute of higher learning and to the best of my knowledge and belief it does not contain any material previously published overwritten by another person except where the acknowledgment is made in the text. Also, I hereby grant to Sri Lanka Institute of Information Technology the nonexclusive right to reproduce and distribute my dissertation, in whole or in part in print, electronic or other medium. I retain the right to use this content in whole or part in future works (such as articles or books).

Signature: Date: The above candidate has carried out research for the B.Sc Dissertation under my supervision.

Signature of the supervisor: Date:

Brain Computer Application Interface

IT10088074

Abstract
Brain computer interfaces are emerging research area. Brain computer interfaces has stepped from scientific fictions to consumer devices .Apart from research grade BCI devices, low cost consumer devices has come to the market. In this project we are going to build a Application framework for Neurosky Mind wave, a low cost consumer grade BCI device. Furthermore we are going to investigate the capability of device to control a third party applications such as a game or Assistive application. We are going to find the suitable applications for this kind of device and propose and implement software wise enhancement for this type of Applications.

Brain Computer Application Interface

IT10088074

Acknowledgment
Upon this page we humbly put down our thoughts of deepest gratitude for all those who guided us in our endeavors to create the Tourism Portal. Though it seems impossible to name all those who lent us a helping hand along our way, we would like to mention a few names in representation of all our mentors. We first convey our utmost gratitude to Mr.Jayantha Lal Amararachchi (Student Coordinator and Lecturer, Sri Lanka Institute of Information Technology) Drs. Rohana Thilakumara (Supervisor, Lecturer, Sri Lanka Institute of Information Technology), and Mr. Darshika Koggalahewa(Co-supervisor, Assistant Lecturer, Sri Lanka Institute of Information Technology), for providing us with proper guidance for the Comprehensive Design Analysis Project and guiding us along the correct path. We wish to thank all our fellow students who helped us in numerous ways to make this project a success. Finally with a wave of gratitude for all those Well-wishers and mentors whose names we failed to mention, we bring this Statement of acknowledgment to conclusion.

Brain Computer Application Interface

IT10088074

Contents
INTRODUCTION & RESEARCH ............................................................................................................. 6 Background Context .............................................................................................................................. 7 Research Problem ................................................................................................................................. 8 Research Gap ......................................................................................................................................... 8 Research Problem Questions .............................................................................................................. 9 Literature Survey .................................................................................................................................... 9 Methodology.............................................................................................................................................. 11 System Architecture ............................................................................................................................. 11 System Overview ................................................................................................................................. 12 BCI Enhanced Key Board ................................................................................................................... 12 BCI Enhanced LUDO Game............................................................................................................... 14 Methodology.......................................................................................................................................... 17 Overview of the implementation methodology ............................................................................. 18 RESULTS DISCUSSION ........................................................................................................................ 18 Technologies used ............................................................................................................................... 19 References ................................................................................................................................................ 20

List of Abbreviations

Brain Computer Application Interface

IT10088074

BCI

{ (Brain Computer Interface) - is a direct communication pathway between the

brain and an external device. Neurosky { is a manufacturer of BCI technologies for consumer product applications.

Mindwave {BCI - consumer grade BCI manufactured by Neurosky

EEG {(Electroencephalography ) - is the recording of electrical activity along the scalp

ThinkGear { -is a library for communication between Neurosky BCI and a Computer C#{ An object-oriented programming language

INTRODUCTION & RESEARCH

Brain Computer Application Interface

IT10088074

Background Context
Brain-Computer Interface (BCI) technology is a potentially powerful communication and control option in the interaction between Human and Computer systems. A Brain Computer Interface (BCI) is a direct communication pathway between the brain and an external device. This technology enables signals that are generated in the brain to control an external activity such as control of a keyboard of a computer or control of a wheelchair. Over the past few years, variety of applications have been developed that are exclusively for use with a EEG headset by companies like Neurosky and Emotiv. In this proposal we are proposing novel a brain computer application interface that can be facilitated in developing brain controlled applications. By proposing this frame work we try break down barrier between brain computer interface researchers and software developers community. Our proposed framework will facilitate to professional developers as well as independent enthusiast to dive in to the depth of brain controlled application world without bothering about brain wave analyzing, feature extraction and classification. Furthermore they wouldn't want to brother about devices without considering devices they would be able to develop their creative gaming ideas. The objective of the proposed project is to bring expert knowledge of Neurophysicists toward the programmers and motivate them to use brain waves as a novel way of controlling computers.

Brain Computer Application Interface

IT10088074

Research Problem

Brain computer interfaces are immerging research area in these days. Many manufactures have been able to develop consumer grade BCI such as neurosky and Emotive. There are two main problems arising in developing brain computer Applications . They are the capability of the devices and required knowledge of developing brain computer applications. Most o f consumer level brain computer interfaces which are available in the market define some parameters using their proprietary algorithms which are kept in secret. They have marketed their device by saying anyone can build BCI application using these parameter values. In this project we have use neurosky mind wave a consumer grade BCI.It has three parameters .They are attention level ,meditation level and blink strength .He have research how these parameters are effective in developing Brain computer applications. Furthermore we have to investigate what types of applications are suitable for these types of devices .Most of these devices are marketed as gaming devices. So we have to find what types of games can be played using these devices.

Research Gap
In this project Neurosky mind wave BCI is used. Mind wave has only two sensors. One is used to measure the EEG activities of our eye and other one is a reference sensors. It can measure EEG of motor functions. Therefore we forced to defend their defined parameters

Brain Computer Application Interface

IT10088074

Research Problem Questions

1. Can one static sensor on the forehead (Instead grid of sensors) use to BCI applications 2. What are the advantages and limitations of the NeuroSky mindset in creating BCI applications ? 3. what are the best parameters that can be used to build BCI applications? 4. How is the accuracy of collected brain waves tested?

Literature Survey
Literature Review provides the basis for a research project in order to identify the domain with respect to tools and techniques, performance of the existing system; it's merits and demerits new trends etc. Jacques Vidal first introduced the terminology of Brain-Computer Interaction (BCI) in 1973 (Vidal, 1973). In a visionary article he posed the fundamental question that we are still trying to answer nowadays: Can these observable electrical brain signals be put to work as carriers of information in man-computer communication or for the purpose of controlling such external apparatus as prosthetic devices or spaceships?" (Vidal, 1973)

Vidal illustrates the laboratory setup used to investigate such a possibility and experiments to approach the solution of such problem. A little less than 30 years after a
9

Brain Computer Application Interface

IT10088074

typical medical laboratory has more or less the same apparatus, disregarding the obvious difference in computer power and digital memory involved in the laboratory. The EEG recording is obtained by placing electrodes on the scalp with a conductive gel. Most systems use caps or nets into which electrodes are embedded; this is particularly common when high-density arrays of electrodes are needed. Each electrode is connected to the input of a differential amplifier (usually one amplifier per pair of electrodes). A common system reference electrode is connected to the other input of each differential amplifier. The voltage amplification between the active electrode and the reference is typically 1,000-100,000 times, reaching 60-100 dB of voltage gain. Most EEG systems these days, however, are digital, and the amplified signal is digitized via an analog-to-digital converter, after being passed through an anti-aliasing filter.
In what research has been done so far, the field information technology & BCI technology has develop a wealth of computer technology to support the various aspect of BCI technology, software applications and Hardware applications. Primarily there are two categories of BCI systems invasive and noninvasive. Invasive systems interact with the brain directly via electrodes / sensors that are implanted into the brain or its surface. While noninvasive systems interact with the brain indirectly via electrodes / sensors placed on the surface of the head that detect brain signal emissions (e.g. Electro-Encephalography (EEG), functional Magnetic Resonance Imaging (fMRI), and Magnetic Sensor Systems).

10

Brain Computer Application Interface

IT10088074

Methodology System Architecture

EEG values are captured from Neurosky Mind wave device using device driver and record them. For that purpose We use Think gear API .Then calculate the attention ,meditation and blink strength values.

11

Brain Computer Application Interface

IT10088074

System Overview
Here We have developed two main Application. 1) BCI Enhanced Key Board for disabled person 2)BCI Enhanced LUDO Game

BCI Enhanced Key Board

Interface of Blink Key Board

This application has been design for disabled users to communicate with the world ithout any communication barriers. The Neurosky device detect the strength of eye blink. These eye blink is used to control button of the Blink Key Board. First user has to select the relevant row by blinking user eye when its highlighted. (Wen the Key Board i s
12

Brain Computer Application Interface

IT10088074

open button row of it are automatically highlighted from bottom to top) After that the button of the selected row highlighted from the left right Blinking eyes user has to select the correct button when it is highlighted. After typing all the word of the text SAY IT button should be selected then what you have type is pronounced.

BCI Enhanced LUDO game

13

Brain Computer Application Interface

IT10088074

Interface of LUDO game

we have developed a Ludo game. The value of the Die of this game is generated according to the Attention level of the user. If the user is having a higher attention level, the Die will get higher values. If the attention level of the user is low, the Die will get lower values. The Attention level values of the user is given to the application by the BCI device that is worn by the user. Then Running Average of those values are automatically generated and those Running Average values decide the value that the Die should get .

Research Findings
Mainly Neurosky Mind wave give us three parameters 1)Attention Level 2)Meditation level 3)Blink Strength
14

Brain Computer Application Interface

IT10088074

These parameters are defined by their proprietary algorithms which are not published. There for we had to check above parameters and usability for consumer level software applications We mainly did three tests using randomly chosen students of our institution. 1)Measure both attention and meditation levels of students while they were reading a paragraph of a book. 2) Measure both attention and meditation levels of students while they were listion to a music 3) Measure both attention and meditation levels of students while they were playing a car race game

Attention Level Values

120 100 80 60 40 20 0 1 28 55 82 109 136 163 190 217 244 271 298 325 352 379 406 433 460 487 514 541 568 595 622 649 676 703 730 757 784 811 838 865 892 919 946

Attention level -Student A

15

Brain Computer Application Interface

IT10088074

Attention Level Values

90 80 70 60 50 40 30 20 10 0 1 10 19 28 37 46 55 64 73 82 91 100 109 118 127 136 145 154 163 172 181 190 199 208 217 226 235 244 253 262 271 280 289 298 307 316 325

Attention level -Student B

Given above are some values that we had taken during our Experiments. We can see clearly that these data are rapidly varying e were able to see same kind of situation in Meditation data also. We can use these types of signal for controlling an application. Therefore we calculated running average of collected data

Running Average of Attention Level Values

30 20 10 0 1 27 53 79 105 131 157 183 209 235 261 287 313 339 365 391 417 443 469 495 521 547 573 599 625 651 677 703 729 755 781 807 833 859 885 911

Student A running Average

16

Brain Computer Application Interface

IT10088074

Running average of Attention Level Values

25 20 15 10 5 0 1 9 17 25 33 41 49 57 65 73 81 89 97 105 113 121 129 137 145 153 161 169 177 185 193 201 209 217 225 233 241 249 257 265 273 281 289

Student B Running Average values

We can clearly observe, even in running average values there is some amount of variation .So this type of signals are not suitable for controlling software application. So we have to select blink strength as our controlling parameter.

Methodology
17

Brain Computer Application Interface Overview of the implementation methodology

IT10088074

As for the development methodology we use the agile scrum method. Whereas the whole system is broken break in modules a and individually modules are developed. Then individual modules are integrated and build the system .Then test. If there need to add enhancements or correct error build the relevant module and integrate until a efficient and effective system comes out.

RESULTS DISCUSSION
We initially developed a key board for disable persons and a ludo game using Think gear Neurosky device can not go beyond that due to the limitation the device. But We have add enhancements for our applications for user convenience. For example ,The calculator take considerable amount of time and effort to operate it. We hope to add some methods to reduce these parameters. We hope to add some word suggestion methods for that. 18

Brain Computer Application Interface

IT10088074

Technologies used
1. Neurosky mindwave BCI Device 2. Microsoft Visual Studio 2010 3. Net Framework 3.5 4. C# programming Language 5. Microsoft Excel 2007

Conclusion
In this project we have used a neurosky mind wave device which has only single sensor. To control advanced application such as 3d games capabilities of the used BCI devices should be in a high standard level with the fully functional options. Also there should be many sensors which are capable of capturing brain signal within vast areas of the brain.

19

Brain Computer Application Interface

IT10088074

one sensor is not enough of developing advance applications. .Neurosky basically defines three parameters .But out of them only blink strength can use for serious real time applications .Attention and meditation values are not much useful in developing real-time applications. Those values cannot be considered as much reliable or accurate. Therefore the team decided to implement applications depending on the capabilities of the using BCI mind wave headset. So the implemented applications are Mind controlling Calculator and a Keyboard which is working based on the users eye blink. Also a Ludo Game controlling based on the eye blink strength . As a conclusion we can say to create serious applications using a BCI device ,Device should have a accessed to different regions of the brain such as motor cortex. Accessing motor cortex is much useful in developing BCI enhanced games.

References
1. "Brain-Computer Interfaces: Beyond Medical Applications," 2013. [Online]. Available: http://lifesciences.ieee.org/articles/114-brain-computer-interfaces-beyond-medicalapplications. 2. E. Kader, B. P. B. P. Emilie Belley, J.-A. Filion, A. Nutter, M. Parent-Vachon, M. Saulnier, B. P. Stephanie Shedleur, B. P. Tsz Ting Wan, B. B. Elissa Sitcoff and P. O. Nicol Korner-Bitensky, "MOTOR IMAGERY - Information for Patients and Families," 20

Brain Computer Application Interface 2010. [Online]. Available: http://strokengine.ca/intervention/admin/patient/Motor%20ImageryFamily%20InformationDec2010.pdf. 3. "P300 (neuroscience)," 2012. [Online]. Available: http://en.wikipedia.org/wiki/P300_(neuroscience). [Accessed 2013].

IT10088074

4. K. Nakayama and M. Mackeben, "Steady State Visual Evoked Potentials In the Alert Primate," 1982. [Online]. Available: http://visionlab.harvard.edu/members/ken/Ken%20papers%20for%20web%20page/027 VisionRes82.pdf.

5. "Event-related Potential: An overview," 2011. [Online]. Available: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3016705/. [Accessed 2013].

6. "Electroencephalogram (EEG)," WebMD, LLC., 2010. [Online]. Available: http://www.webmd.com/epilepsy/electroencephalogram-eeg-21508. 7. "What Is A Framework?," 2003. [Online]. Available: http://www.codeproject.com/Articles/5381/What-Is-A-Framework. 8. "Neurosky MindWave Mobile," Neurosky, 2012. [Online]. Available: http://neurosky.com/Products/MindWaveMobile.aspx. 9. Microsoft, "Microsoft Visual Studio," Microsoft, 2013. [Online]. Available: http://www.microsoft.com/visualstudio/eng/products/visual-studio-ultimate-2012. 10. S. Du and M. Vuskovic, "Temporal vs. Spectral Approach to Feature Extraction," [Online]. Available: http://medusa.sdsu.edu/Robotics/Neuromuscular/Our_Publications/FE_Sijiang_press.pd f. 11. J. B. Ochoa, "EEG Signal Classification for Brain," 2002.

21