Brain-Computer Interfacing for Assistive Robotics: Electroencephalograms, Recurrent Quantum Neural Networks, and User-Centric Graphical Interfaces

Brain–Computer Interfacing for Assistive Robotics

Electroencephalograms, Recurrent Quantum Neural Networks and User-Centric Graphical Interfaces

Vaibhav Gandhi, B.Eng, M.Eng, Ph.D

School of Science and Technology, Middlesex University, London, UK

Table of Contents

Cover image

Title page

Copyright

List of Figures

List of Tables

Preface

Acknowledgments

List of Acronyms

Chapter 1. Introduction

1.1 Introduction

1.2 Rationale

1.3 Objectives

Chapter 2. Interfacing Brain and Machine

2.1 Introduction

2.2 The Brain and Electrode Placement

2.3 Operational Techniques in BCI

2.4 Data Acquisition

2.5 Preprocessing: A Signal Enhancement Requirement along with Noise Reduction

2.6 Feature Extraction

2.7 Classification

2.8 Post-processing

2.9 Validation and Optimization Techniques

2.10 Graphical User Interface [GUI]

2.11 Strategies in BCI Applications

2.12 Performance Measures of a BCI System

2.13 Conclusion

Chapter 3. Fundamentals of Recurrent Quantum Neural Networks

3.1 Introduction

3.2 Postulates of Quantum Mechanics

3.3 Quantum Mechanics and the Schrodinger Wave Equation

3.4 Theoretical Concept of the RQNN Model

3.5 Traditional RQNN-Based Signal Enhancement

3.6 Revised RQNN-Based Signal Enhancement

3.7 Discussion

3.8 Conclusion

Chapter 4. The Proposed Graphical User Interface (GUI)

4.1 Introduction

4.2 Overview of the Proposed GUI Within the BCI Framework

4.3 Interfacing MATLAB and Visual Basic

4.4 Conclusion

Chapter 5. Recurrent Quantum Neural Network (RQNN)-Based EEG Enhancement

5.1 Introduction

5.2 Traditional RQNN Model for EEG Enhancement

5.3 Revised RQNN Model for EEG Signal Enhancement

5.4 Towards Subject-Specific RQNN Parameters

5.5 Discussion

5.6 Conclusion

Chapter 6. Graphical User Interface (GUI) and Robot Operation

6.1 Introduction

6.2 The EEG Acquisition Process

6.3 RQNN-Based EEG Signal Enhancement

6.4 Autonomous and Supervised GUI Operation

6.5 Maneuvering the Simulated Mobile Robot Using Only MI EEG

6.6 Maneuvering the Physical Mobile Robot Using Only MI EEG

6.7 Conclusion

Chapter 7. Conclusion

7.1 Contributions of the Book

7.2 Future Research Directions

7.3 Conclusion

Appendix A. Understanding Evaluation Quantifiers for the Proposed Interface

Bibliography

Index

Copyright

Academic Press is an imprint of Elsevier

32 Jamestown Road, London NW1 7BY, UK

525 B Street, Suite 1800, San Diego, CA 92101-4495, USA

225 Wyman Street, Waltham, MA 02451, USA

The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, UK

Copyright © 2015 Elsevier Inc. All rights reserved

No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions.

This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted herein).

Notices

Knowledge and best practice in this field are constantly changing. As new research and experience broaden our understanding, changes in research methods, professional practices, or medical treatment may become necessary.

Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information, methods, compounds, or experiments described herein. In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility.

To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein.

ISBN: 978-0-12-801543-8

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library

Library of Congress Cataloging-in-Publication Data

A catalog record for this book is available from the Library of Congress

For information on all Academic Press publications visit our website at http://store.elsevier.com/

Typeset by MPS Limited, Chennai, India www.adi-mps.com

Printed and bound in the United States of America

List of Figures

List of Tables

Preface

Vaibhav Gandhi

Brain–computer interface (BCI) technology provides a means of communication that allows individuals with severely impaired movement to communicate with assistive devices using the electroencephalogram (EEG) or other brain signals. The practicality of a BCI has been made possible by advances in multi-disciplinary areas of research related to cognitive neuroscience, brain-imaging techniques and human–computer interfaces. The end goal of a BCI is to enable monitoring of the underlying brain processes and subsequent utilization of this information for communicating and controlling devices solely through the brain without depending on the normal output pathways of peripheral nerves and muscles.

This book is a result of four years of doctoral research, focusing on an important aspect of BCI for real-time assistive robotic application. This book will benefit both mature and early-stage researchers, including postgraduate and doctoral students. Researchers from other fields interested in BCI should also find this book interesting and informative, including its detailed discussion of prevailing research trends. This book should also be helpful in courses related to quantum mechanics, neural networks, signal processing, robotics and human–computer interaction, since it covers the fundamental postulates of quantum mechanics, their application for filtering noisy EEG signals, signal processing fundamentals and adaptive graphical user interfaces (GUIs) for robot control application in considerable detail.

Dealing with the unknown embedded noise within the raw EEG and the inherent lower bandwidth of BCI are still two of the major challenges in making BCI practical for day-to-day use. The raw EEG signal recorded non-invasively during motor imagery (MI) is intrinsically embedded with non-Gaussian noise, and the actual, noise-free EEG has so far not been obtained. Therefore, a novel filtering approach utilizing the concepts from quantum mechanics is discussed in this book. This filtering approach is constructed using a layer of neurons within the neural network framework and is referred to as the Recurrent Quantum Neural Network (RQNN).

Another major challenge in two-class BCI systems is the inherently low bandwidth of the communication channel, which may lead to a sluggish response when controlling a mobile robotic device. An intelligent and adaptive user interface, which plays a very important role as a front-end display for the BCI user, is discussed in this book. The framework of the proposed intelligent Adaptive User Interface (iAUI), also referred to as a brain–robot interface, is consistent for use with a range of applications, e.g., for controlling either a mobile robot or a robotic arm. Both the RQNN filtering technique and the iAUI have been investigated for real-time application and are discussed in detail in this book.

The book contains seven chapters. Chapter 1 introduces the rationale, the aims and objectives of the book. Chapter 2 presents in detail a general EEG-based BCI system. The chapter begins with an introduction of the biological fundamentals of the brain and a description of EEG rhythms. This is followed by a description of the operational techniques in BCI (synchronous and asynchronous communication) and an in-depth explanation of the various signal processing related building blocks of a typical BCI system including the data acquisition process, preprocessing, feature extraction, classification and post-processing approaches. Cross-validation and a few popular optimization techniques such as particle swarm optimization (PSO) and genetic algorithms (GA) which are utilized for tuning/selecting the RQNN model parameters are discussed. The chapter then describes the necessity and importance of a GUI, the different interface designs proposed by various research groups and the challenges and issues for practical implementation. Major BCI strategies and breakthrough techniques employed to enhance the usability of BCI signal processing outcome for applications in the form of shared control and adaptive shared control techniques are also discussed along with the performance evaluation measures for an interface. Chapter 3 begins with a brief overview of the fundamentals of QM and SWE. This chapter then describes the theoretical concept of the RQNN models and their application to filtering simple signals in the form of DC, stepwise DC and sinusoidal waveforms.

The proposed MI-based GUI design, i.e., the adaptive interfaces for a wheelchair/robot control application, are discussed in Chapter 4. This chapter begins by discussing the role of the proposed brain–robot in the complete BCI framework, and then covers the mobile control and arm control applications. Chapter 5 builds on the foundations laid in the form of the proposed quantum mechanics-based RQNN models discussed in Chapter 3. The traditional and the revised RQNN models are investigated for different BCI competition datasets, as are subject-specific RQNN model parameters using a two-step inner-outer five-fold cross-validation technique. Chapter 6 presents the brain–robot interface for a practical robot control task. The EEG acquisition process within the BCI laboratory and the procedure for applying the RQNN technique in real time are also discussed. The proposed autonomous and supervised interfaces for real-time maneuvering of the mobile robot to reach different locations within the simulated and physical robotic arena are then discussed and compared with existing designs in the field of BCI. Chapter 7 concludes the book, including potental future directions and research ideas to potentially further enhance the performance of the BCI system. References and appendices are given at the end of the book.

Acknowledgments

This book is based on Vaibhav Gandhi’s Ph.D. thesis, which was submitted to the University of Ulster, UK. The research work was carried out at the Intelligent Systems Research Center, University of Ulster, UK and the Department of Electrical Engineering, IIT Kanpur, India. The author is thankful to his family and friends whose personal support has been instrumental in making this book possible.

List of Acronyms

1-D 1-dimensional

ADC Analog to Digital Converter

ALS Amyotrophic Lateral Sclerosis

AM Adaptation Module

ANN Artificial Neural Network

ANOVA2 Two-way Analysis of Variance

AR Autoregressive

ARM Autonomous Arm

BCI Brain–Computer Interface

BCW Brain-Controlled Wheelchair

BP Band Power

CA Classification Accuracy

CAR Common Average Reference

CM Communication Module

CMRR Common-Mode Rejection Ratio

CSCI Cervical Spinal Cord Injuries

CSP Common Spatial Patterns

CSSP Common Spatio-Spectral Pattern

CSSSP Common Sparse Spectral Spatial Pattern

DFT Discrete Fourier Transform

ECG Electrocardiogram

EEG Electroencephalogram

EKF Extended Kalman Filter

EMD Emperical Mode Decomposition

EMG Electromyogram

EOG Electrooculogram

EP Evoked Potential

ERD Event-Related Desynchronization

ERP Event-Related Potential

ErrP Error Potential

ERS Event-Related Synchronization

FBCSP Filter-Bank CSP

FE Feature Extraction

FEP Feature Extraction Process

FLDA Fisher’s LDA

FT Fourier Transform

GA Genetic Algorithm

gbest global best

GUI Graphical User Interface

hBCI hybrid BCI

HHT Hilbert–Huang Transform

iAUI Intelligent Adaptive User Interface

ICA Independent Component Analysis

ID Index of Difficulty

IMF Intrinsic Mode Functions

IP Index of Performance

IRLS Iteratively Reweighted Least-Squares

IRM Information Refresh Module

ITR Information Transfer Rate

KF Kalman Filter

lbest local best

LDA Linear Discriminant Analysis

MI Motor Imagery

MLE Maximum Likelihood Estimation

MM Monitor Module

MND Motor Neurone Disease

MT Movement Time

NC No-Control

NN Neural Network

NTSPP Neural–Time-Series-Prediction

OVR One Versus Rest

PCA Principal Component Analysis

pdfs Probability Density Functions

PMs Prediction Models

PSD Power Spectral Density

PSO Particle Swarm Optimization

QM Quantum Mechanics

QNN Quantum Neural Network

RBF Radial Basis Function

RCSP Regularized CSP

RMSE Root Mean Square Error

RQNN Recurrent Quantum Neural Network

SARM Supervised Arm

SBCSP Sub Band CSP

SCI Spinal Cord Injury

SCP Slow Cortical Potentials

SG Savitzky–Golay

SMA Supplementary Motor Areas

SMR Sensorimotor Rhythm

SNR Signal-to-Noise

SSVEP Steady State Visually Evoked Potential

SVM Support Vector Machines

SWLDA Stepwise LDA

t-f Time-Frequency

TSD Time-Varying Signed Distance

UDP User Datagram Protocol

UKIERI UK-India Education & Research Initiative

UMDA Univariate Marginal Distribution Algorithm

VB Visual Basic

VEPs Visually Evoked Potentials

Chapter 1

Introduction

Verbal or non-verbal information exchange is the basis of human communication. However, some people lose this fundamental ability of communication through accidents or inherited neuromuscular disorders. In the absence of methods for repairing or restoring function due to disease or damage, various alternatives in the form of assistive devices to enable individuals to communicate with and control their environment have been developed. The brain–computer interface (BCI), i.e., electroencephalography (EEG)-based communication, is a new way of controlling devices that does not require eye movement or muscle activity. This chapter introduces the various components of a typical BCI system and explains each component’s importance and function within the complete BCI system.

Keywords

Brain–computer interface; electroencephalography; graphical user interface; motor imagery; signal filtering

1.1 Introduction

Verbal or non-verbal information exchange is the basis of human communication. However, some people lose this fundamental ability of communication because of accidents or inherited neuromuscular disorders. The purpose of the work presented in this book is to contribute to the development of novel methods to allow people to regain freedom of movement/communication by way of controlling devices directly with their brain, bypassing the normal communication channels.

The human brain is estimated to contain about 100 billion neurons [1–4]. The spinal cord acts as an intermediate cable that carries information to and from our brain to control various body parts and their movements. People with an injury to the spinal cord are still able to generate the output signals from the brain, but these signals do not reach the specific body parts because the intermediate spinal cable is damaged. Several technologies using a joystick, head movement, eye gazing and many more may help a physically challenged person to control a robotic device or a wheelchair [5–9]. However, these techniques require the use of partial movement control through the hand, head or eyes etc., and therefore make the control issue less complicated. The issue becomes more challenging when people with complete loss of control over their voluntary muscles are involved, a condition generally known as locked-in syndrome [10,11], in which people are unable to speak and move but are conscious and can think and reason. A number of neurological diseases such as stroke¹, severe cerebral palsy², motor neuron disease (MND)³, amyotrophic lateral sclerosis (ALS), and encephalitis⁴ can result in such severe motor paralysis [12]. Many of these diseases can lead to restrictions in communication capacity. A brain–computer interface (BCI) can enable such physically challenged people to achieve greater independence by making technology accessible. BCI technology provides an alternative communication channel between the human brain (that does not depend on the brain’s normal output channels of the peripheral nerves and muscles) and a computer [13–21]. The three most commonly discussed diseases/injuries cited in the BCI literature as being a case of locked-in syndrome are ALS, high spinal cord injury and brain stem stroke [16,22–24].

• Patients suffering from ALS can undergo severe physical impairment due to the degeneration of nerve cells that control the voluntary muscles. In the later stages of ALS, the most basic human actions are affected, including speech, swallowing and breathing [25].

• Spinal cord injury (SCI) can result in damage to myelinated fiber tracts or the nerve roots that carry the signals to and from the brain [25]. In complete SCI, most of the motor functions and sensation below the neurological level