Visual Neuroprosthetics

ABSTRACT
A recent technological breakthrough in materials and micro fabrication technologies afford neuro surgeons, ophthalmologists and engineers the opportunity to reconsider the prospect of providing a useful visual sense to profoundly blind. This will be accomplished by electrically stimulating their visual systems via an array of implanted electrodes, which can be safely implanted into the visual parts of the brain with little significant long term consequences. A survey of the present state of research on a visual neuroprosthetic, interfaced with the occipital cortex, as a means through which a limited but useful visual sense could be restored to profoundly blind people is given. We review the most important physiological principles regarding this neuroprosthetic approach and emphasize the role of neural plasticity to achieve the desired behavior of the system. While the full restoration of vision is unlikely in the near future, the discrimination of shape and location of objects could allow blind subjects to navigate in a familiar environment and to read enlarged text, resulting in a substantial improvement in the quality of life of blind and visually impaired persons. Moreover, if we can understand more about the fundamental mechanisms of neuronal coding, and to safely and selectively stimulate nervous system, there will real potential to apply this knowledge clinically.

Dept of CSE

MIT, Mysore

Visual Neuroprosthetics

CHAPTER 1

INTRODUCTION
In medicine, prosthesis, prosthetic, or prosthetic limb is an artificial device extension that replaces a missing body part. Neuroprosthetics (also called neural prosthetics) is a discipline related to neuroscience and biomedical engineering concerned with developing neural prosthetic. Neural prosthetic are a series of devices that can substitute a motor, sensory or cognitive modality that might have been damaged as a result of an injury or a disease. Cochlear implants provide an example of such devices. The first known cochlear implant was created in 1957. These devices substitute the functions performed by the ear drum and Stapes, while simulating the frequency analysis performed in the cochlea. A microphone on an external unit gathers the sound and processes it; the processed signal is then transferred to an implanted unit that stimulates the auditory nerves through a microelectrode array. Through the replacement or augmentation of damaged senses, these devices intend to improve the quality of life for those with disabilities. Neuroprosthetics are devices that interact with and control the nervous system. They are designed to reproduce or substitute for neurological and physiological function that has been lost to injury or disease. Wearable electrical stimulation systems deliver impulses to peripheral nerves. They induce a variety of beneficial effects, including muscle building, relaxation of spastic muscle, and improvement of blood circulation, reduction of joint contractures and alleviation of pain. Neuro-Prosthetics improves function in a way that is physiological. Patients with central nervous system dysfunction lose function when muscles become paralyzed. However, muscles can be stimulated with enough force to induce purposeful movement. Loss of vision poses extraordinary challenges to individuals in our society which relies heavily on sight. Although in recent years the techniques of molecular genetics have led to a rapid identification of a great number of genes involved in visual diseases, the nervous system once damaged is capable of little functional regeneration and currently there is no effective treatment for many patients who are visually handicapped as a result of degeneration or damage to: 1) the retina, 2) the optic nerve, or 3) the brain. While

pharmacological interventions provide therapeutic solutions to many physiological problems,

Dept of CSE 2 MIT, Mysore

Visual Neuroprosthetics a pharmacological approach to the mechanisms of blindness has not been discovered. A new, human engineered approach to this problem has generated new hope in those suffering from profound vision loss. This approach, a vision neuroprosthesis, is based on the observation that electrical stimulation of neurons at almost any location along the visual path will evoke visual perceptions called phosphenes.

1.1

NEUROPROSTHETICS

1.1.1 VISUAL PROSTHETICS

A visual prosthesis can create a sense of image by electrically stimulating neurons in the visual system. A camera would wirelessly transmit to an implant; the implant would map the image across an array of electrodes. The array of electrodes has to effectively stimulate 600-1000 locations, stimulating these optic neurons in the retina thus will create an image. The stimulation can also be done anywhere along the optic signal's path way. The optical nerve can be stimulated in order to create an image, or the visual cortex can be stimulated, although clinical tests have proven most successful for retinal implants. Hence we can refer the visual impairment due to cortex problem also as cortical visual neuroprothetics.

1.1.2 COGNITIVE PROSTHETIC

Cognitive prosthetic seek to restore cognitive function to individuals with brain tissue loss due to injury, disease, or stroke by performing the function of the damaged tissue with integrated circuits. The theory of localization states that brain functions are localized to a specific portion of the brain. However, recent studies on brain plasticity suggest that the brain is capable of rewiring itself so that an area of the brain traditionally associated with a particular function (i.e. auditory cortex) can perform functions associated with another portion of the brain (I.e. auditory cortex processing visual information). Implants could take advantage of brain plasticity to restore cognitive function even if the native tissue has been destroyed.

Dept of CSE

MIT, Mysore

Visual Neuroprosthetics

1.1.3 MOTOR PROSTHETICS

Devices which support the function of autonomous nervous system include the implant for bladder control. In the somatic nervous system attempts to aid conscious control of movement include Functional Electrical Simulation and the lumbar anterior root stimulator.

1.2

EYE
Eyes are organs that detect light and convert it into electro-chemical impulses

in neurons. The simplest photoreceptors in conscious vision connect light to movement. In higher organisms the eye is a complex optical system which collects light from the surrounding environment, regulates its intensity through a diaphragm, focuses it through an adjustable assembly of lenses to form an image, converts this image into a set of electrical signals, and transmits these signals to the brain through complex neural pathways that connect the eye via the optic nerve to the visual cortex and other areas of the brain as show in fig 1.1.

Fig 1.1 Human Eyes The human eye is an organ which reacts to light for several purposes. As a conscious sense organ, the eye allows vision. Rod and cone cells in the retina allow conscious light perception and vision including color differentiation and the perception of depth. The human eye can distinguish about 10 million colors. In common with the eyes of other mammals, the human eye's non-image-forming photosensitive ganglion cells in the retina receive the light signals which affect adjustment of the size of the pupil, regulation and

Dept of CSE

MIT, Mysore

Visual Neuroprosthetics suppression of the hormone melatonin and entrainment of the body clock.Structure of an eye illustrated in fig 1.2

Fig 1.2 Structure of an Eye Anterior Chamber: The space between the cornea and iris filled with Aqueous Humor. Posterior Chamber: The space between the iris and the front of the lens filled with Aqueous Humor. Cornea: The cornea is a clear, dome-shaped surface that covers the front of the eye. It is the first and most powerful lens in the eye's optical system. To keep it transparent the cornea contains no blood vessels. Tears that flow over it and aqueous humor in the chamber behind it keep it nourished. When you hear of eye banks and eye transplants, it is the cornea that is being replaced. The cornea can be damaged from: accidents, infections, and genetic defects. Iris: This is the colored part of the eye: brown, green, blue, etc. It is a ring of muscle fibers located behind the cornea and in front of the lens. It contracts and expands, opening and closing the pupil, in response to the brightness of surrounding light. Just as the aperture in a camera protects the film from over exposure, the iris of the eye helps protect the sensitive retina. Ciliary Body: This is where the Aqueous Humor is produced. Optic Nerve: Each optic nerve has about 1.2 million nerve fibers. This is the cable connecting the eye to the brain. Optic Tract: The nerves that connect the optic chiasm to the LGN are called the optic tract. If one of these tracts is damaged, vision will be lost in one side of each eye.
Dept of CSE 5 MIT, Mysore

Visual Neuroprosthetics Pupil: The pupil is the hole in the center of the iris that light passes through. The iris muscles control its size. Retina: The retina is the film of the eye. It converts light rays into electrical signals and sends them to the brain through the optic nerve. The sides of the retina are responsible for our peripheral vision. The center area, called the macula, is used for our fine central vision and color vision. The retina is where most the problems leading to vision loss occur. Three of the leading causes of blindness, from retina damage, are Retinitis Pigmentosa, Macular Degeneration and Diabetic Retinopathy. Sclera: The sclera is the white, tough wall of the eye. It along with internal fluid pressure keeps the eye shape and protects its delicate internal parts. Choroid: The choroid is a layer of blood vessels between the retina and sclera; it supplies blood to the retina. In the disease called Macular Degeneration, abnormal blood vessels grow into the space between the retina and choroid damaging the macula. Vitreous Humor: The vitreous humor is a jelly like liquid that fills most of the eye (from the lens back). As we age it changes from a gel to a liquid and gradually shrinks separating from the retina. This is when people start seeing floaters, dark specs in their vision. This is a normal sign of aging, but in a few cases the retina can become detached as the vitreous separates. Photoreceptor Cells: The retina is composed of two types of photoreceptor cells. When light falls on one of these cells, it causes a chemical reaction that sends an electrical signal to the brain. Cone cells give us our detailed color daytime vision. There are 6 million of them in each human eye. Most of them are located in the central retina - macula fovea area. There are three types of cone cells: one sensitive to red light, another to green light, and the third sensitive to blue light.

Rod cells are about 500 times more sensitive to light then cone cells; they give us our dim

Dept of CSE

MIT, Mysore

Visual Neuroprosthetics light or night vision. They are also more sensitive to motion then cone cells. There are 120 million rod cells in the human eye. Most rod cells are located in our peripheral or side vision. Macula - (yellow spot): This part of the retina is the most sensitive. Its diameter is only 7 mm or about 1/4 inch. It is responsible for our central or reading vision. This part of the retina gives us 20/20 vision. Without the macula, you would be blind - Legally blind that is. People with eye diseases like Macular Degeneration have vision from 20/200 to 20/800.

1.2.1 GENERAL PROPERTIES OF EYE

The eye is not properly a sphere; rather it is a fused two-piece unit. The smaller frontal unit, more curved, called the cornea is linked to the larger unit called the sclera. The corneal segment is typically about 8 mm (0.3 in) in radius. The sclerotic chamber constitutes the remaining five-sixths; its radius is typically about 12 mm. The cornea and sclera are connected by a ring called the limbus. The iris the color of the eye and its black center, the pupil, are seen instead of the cornea due to the cornea's transparency. To see inside the eye, an ophthalmoscope is needed, since light is not reflected out. The fundus (area opposite the pupil) shows the characteristic pale optic disk (papilla), where vessels entering the eye pass across and optic nerve fibers depart the globe. The vertical measure, generally less than the horizontal distance, is about 24 mm among adults, at birth about 1617 mm. The retina has a static contrast ratio of around 100:1 (about 6.5 f-stops). As soon as the eye moves it readjusts its exposure both chemically and geometrically by adjusting the iris which regulates the size of the pupil. The approximate field of view of a human eye is 95 out, 75 down, 60 in, 60 up. About 1215 temporal and 1.5 below the horizontal is the optic nerve or blind spot which is roughly 7.5 high and 5.5 wide.

1.3

BRAIN
The human brain has the same general structure as the brains of other mammals, but

is larger than expected on the basis of body size among other primates. Estimates for the number of neurons (nerve cells) in the human brain range from 80 to 120 billion. Most of the expansion comes from the cerebral, especially the frontal lobes, which are associated with executive functions such as self-control, planning, reasoning, and abstract thought. The

Dept of CSE

MIT, Mysore

Visual Neuroprosthetics portion of the cerebral cortex devoted to vision is also greatly enlarged in human beings, and several cortical areas play specific roles in language, a skill that is unique to humans. The cerebral cortex is nearly symmetrical, with left and right hemispheres that are approximate mirror images of each other. Anatomists conventionally divide each hemisphere into four "lobes", the frontal lobe, parietal lobe, occipital lobe, and temporal lobe. This division into lobes does not actually arise from the structure of the cortex itself, though: the lobes are named after the bones of the skull that overlie them, the frontal bone, parietal bone, temporal bone, and occipital bone as show in fig 1.3. The borders between lobes are placed beneath the sutures that link the skull bones together. Because of the arbitrary way most of the borders between lobes are demarcated, they have little functional significance. With the exception of the occipital lobe, a small area that is entirely dedicated to vision, each of the lobes contains a variety of brain areas that have minimal functional relationship. The parietal lobe, for example, contains areas involved in somato sensation, hearing, language, attention, and spatial cognition. In spite of this heterogeneity, the division into lobes is convenient for reference.

Fig 1.3 Brains Significant functional aspects of the occipital lobe are that it contains the primary visual cortex and is the part of the brain where dreams come from. The primary function of the occipital lobe is controlling vision and visual processing. The occipital lobe helps us see and identify different things that we look at. It also helps us

Dept of CSE

MIT, Mysore

Visual Neuroprosthetics differentiate and understand different colors. As soon as a baby is born, occipital lobe function begins. The visual stimuli received by the infants eye is sent over to the occipital lobe that helps the baby interpret the vision. As the infants grow, his/ her vision becomes more and more acute and the ability to understand various images by the occipital lobe also grows. The function of occipital lobe is not just limited to visual recognition. The brain occipital lobe function also includes the ability to understand and differentiate between different shapes. If the brain did not process the different shapes we see, we will not be able understand the geometry of shapes. This would also make it impossible for one to understand various letters as one would not be able to interpret or differentiate between the different shapes.

Dept of CSE

MIT, Mysore

Visual Neuroprosthetics

CHAPTER 2

VISUAL NEURO PROSTHETICS

Loss of vision poses extraordinary challenges to individuals in our society which relies heavily on sight. Although in recent years the techniques of molecular genetics have led to a rapid identification of a great number of genes involved in visual diseases, the nervous system once damaged is capable of little functional regeneration and currently there is no effective treatment for many patients who are visually handicapped as a result of degeneration or damage to: 1) the retina, 2) the optic nerve, or 3) the brain. While

pharmacological interventions provide therapeutic solutions to many physiological problems, a pharmacological approach to the mechanisms of blindness has not been discovered. A new human engineered approach to this problem has generated new hope in those suffering from profound vision loss. This approach, a vision neuroprosthesis, is based on the observation that electrical stimulation of neurons at almost any location along the visual path will evokes visual perceptions called phosphenes. The concept of artificially producing a visual sense in blind individuals is founded on our present understanding of the structure of the mammalian visual system, its processing elements, and the relationship between electrical stimulation of any part of the visual pathways and the resulting visual sensations. A number of studies have shown that electrical stimulation of the retina, optic nerve and visual cortex evokes the perception points of light (called phosphenes) at specific regions in space. However, although these retinal or optic nerveprosthetic may prove to be useful for restoration of sight lost in diseases such as retinitispigmentosa and age related macular degeneration, the retina, and the output neurons of the eye (ganglion cell neurons, which in turn give rise to the optic nerve axons) often degenerate in many retinal blindness, so that these approaches may not be always helpful. Stimulation of the primary visual cortex because the neurons in higher visual regions of the brain (visual cortex) often escape from this degeneration. If these higher visual centers can be stimulated with visual information in a format somewhat similar to the way they were stimulated prior to the development of total blindness, a blind individual may be able to use this stimulation to extract information about the physical world around him/her.

Dept of CSE

10

MIT, Mysore

Visual Neuroprosthetics

2.2 PHYSIOLOGICAL FOUNDATIONS OF A CORTICAL VISUAL NEUROPROSTHETIC

Before looking at the specifics of a visual neuroprosthesis, it will be helpful to review some of the physiological foundations for this neuroprosthetic approach. 1. There is abundant and positive clinical experience with many neural prosthetic interfaces. 2. Most forms of blindness are of retinal origin and leave the higher visual centers unaffected. 3. The visual pathways and primary visual cortex are organized in a relatively rational scheme. 4. Electrical stimulation of neurons in the visual pathway evokes the perception of point of light 5. The plasticity of the brain will foster significant functional reorganization.

Dept of CSE

11

MIT, Mysore

Visual Neuroprosthetics

CHAPTER 3

ENGINEERING A VISUAL NEUROPROSTHETIC

The most fundamental requirements of any neurological prosthesis are well understood. In order for a device to effectively emulate a neurological system, it has to do three things: 1. It must collect the same kind of information that the system normally collects. 2. It has to process that information. 3. It must communicate the processed information, in appropriate way withother parts of the nervous system. Fig 3.1 illustrates the basic components of visual cortically based approach. The system will use a bioinspired retina able to perform some of the image pre-processing functions of the retina. This bioinspired device will transform the visual world in front of a blind individual into electrical signals that can be used to excite neurons at the occipital cortex. These signals will be delivered to intracortical microelectrodes that will excite visual cortex neurons in an appropriate way. Since signals reaching the cortex from the retina and Lateral Geniculate Nucleus (LGN) arrive not at the surface of the cortex (layer 1) but a depth of 1-2 mm (layer 4) as in Fig 3.2, we need intracortical penetrating electrodes with exposed tips located in layer 4 and with tip sizes of the same order of magnitude as the neurons that are intended to be stimulated. For this reason we are using the Utah Electrode Array (UEA), which has 100 microelectrodes, each 1.5 mm long, arranged in a square grid contained in a package 4.2 mm by 4.2 mm (Fig. 3.3). This array of penetrating electrodes has been designed to compromise as little cortical volume as possible. Thus, each needle has been made as slender as possible yet retains sufficient strength to withstand the implantation procedure.

Dept of CSE

12

MIT, Mysore

Visual Neuroprosthetics

Fig 3.1 Basic components of Visual Neuroprosthetics Further, consistent with concept of blunt dissection used by neurosurgeons, these penetrating structures displace the tissues they are inserted into rather than cut their way through them. Finally an integrated telemetry system will transmit power and data (electric impulses) to the electrode array inserted into the visual cortex. The whole visual neuroprosthetic device is expected to recreate a limited, but functionally useful visual sense in blind individuals allowing them to navigate in familiar environments and to read enlarged text. One strategy we are employing is not to simply record an image with high resolution, but to transmit visual information in a meaningful way to the appropriate site/s in the brain. In order to achieve this, we have to take into account the coding features of the biological visual. system and design constraints related with the number and distribution of the set of working electrodes the visual scene is mapped to.

Dept of CSE

13

MIT, Mysore

Visual Neuroprosthetics

Fig 3.2 Layer of Cortex The UEA consists of one hundred 1.5 mm-long microneedles that were designed to be inserted into the cerebral cortex to a depth of 1.5 mm, the level of normal neural input to the cerebral cortex. The electrodes of the UEA and USEA are built on a square grid with 400 m spacing. One hundred gold bond pads are deposited on the back surface of these arrays, and one hundred 1.25 mil insulated gold wires are bonded to these pads and to a percutaneous connector for connection to external electronics. The tip of each microneedle is metalized with iridium oxide to facilitate electronic to ionic transduction, and the entire array, with the exception of the tip of each microneedle, is insulated with a biocompatible polymer. The UEA also consists of up to 100 microneedles, but their lengths are graded from 0.5 mm to 1.5 mm along the length of the array. The graded lengths of the USEA ensure that when it is inserted into a peripheral nerve, the electrode tips uniformly populate the nerve, with most nerve fibers being no more than 200 m away from an active electrode tip.

Dept of CSE

14

MIT, Mysore

Visual Neuroprosthetics

Fig 3.3 Utah Electrode Array (UEA)

3.1

DEVELOPMENT VISUAL

OF

RECONFIGURABLE FRONT-END

BIOINSPIRED

PROCESSING

(ARTIFICIAL RETINA):
One of the major challenges of our approach is the design and development of a bio inspired platform able to transform the visual world in front of a blind individual into a set of electrical signals, that will be used to stimulate, in real time, the neurons at his/her visual cortex. The block diagram representation of this is show as in the below figure 3.4

Fig 3.4 block Diagram of cortical visual neuro prosthetics

Dept of CSE 15 MIT, Mysore

Visual Neuroprosthetics According to our design constraints, these signals should be as similar as possible to the output signals of the real retina. The question of how the information about the external world is compressed in the retina, and how this compressed representation is encoded in spike trainsis therefore of seminal importance. The recent decades have revealed many details about the ways the retina is organized to serve several sub-functions. Sampling across the retina is not uniform and therefore retino topic gradients and magnification factors have to be introduced to match image representation with cortical topography. It is further clear that several streams of information are processed in parallel from any retinal point by several dozens of inter-neuronal subtypes before contrast; brightness; orientation movement and colour are finally coded as modulation of ganglion cell action potential series. While chromatic information is not of utmost priority a differential characterization will nevertheless be required when designing achromatic processing modules for basic representation of image components. Similarly the high sensitivity pathways originating from rod photoreceptors may be silenced by mimicking daylight intensity (photopic) conditions using adequate pre-amplification of dimmer signals. Far from a simple transducer of light into electrical neural impulses, the retina performs a locally-computed spatio-temporal contrast enhancement function, and a very efficient compression of visual information. These tasks are essential to provide a high adaptation capability to very different lighting conditions, a high noise immunity, and to efficiently communicate the visual information by means of a limited number of optic nerve fibers. Thus, our entire experience of the external visual world derives from the concerted activity of a limited number of ganglion cells in the retinal output layer which have to represent all the features of objects in the visual world, namely their color, intensity, shape, movement, and the change of these features in time. This representation has to be unequivocal and fast in order to ensure object recognition for any single stimulus presentation within a few hundreds of milliseconds. Figure 3.5 shows the architecture of the bioinspired retinal model we are currently using. It is based on electrophysiological recordings from populations of retinal ganglion cells. Through a set of parameterized filters and functions, we obtain a portable model that can be easily translated into a hardware description for automatic synthesis using the appropriate tools. The input images are captured by a photosensor array (preferably a logarithmic response camera)and are processed by a set
Dept of CSE 16 MIT, Mysore

Visual Neuroprosthetics of separate spatial and temporal filters that enhance specific features of the captured visual information. The model can take into account the irregular distribution of photoreceptors within the human retina: a high density of pixels and smaller receptive field sizes in central areas; lower density and bigger receptive fields in peripheral areas. A gain factor can be specified for every individual channel as well as a global gain. The output of these filters is combined to produce an output map we call the information figure. The next stage reduces the information figure array to the resolution of the electrode matrix, with the option of defining specific receptive field shapes and sizes. We call activity matrix to this low resolution representation (see Fig. 3.5). Finally, a mapping and neuromorphic coding (into output pulses that can be sent to each electrode) is carried out and feeds the radio frequency link that goes to the microelectrode array. The model implemented in the present version of the software is a simplified version of an integrate-and-fire spiking neuron. Each neuron accumulates input values coming from its receptive field until it reaches a programmable threshold. Then it fires and discharges the accumulated value. The whole system is presently able to work properly up to 40 MHz. This means that 40,000 electrodes could be stimulated with an inter-spike temporal resolution equal to or lower than 1msec. The use of reconfigurable circuitry (FPGA) let us to adjust or even change the spiking model easily.

Dept of CSE

17

MIT, Mysore

Visual Neuroprosthetics

Fig 3.5 Architecture of the bioinspired retinal model

Dept of CSE

18

MIT, Mysore

Visual Neuroprosthetics

CHAPTER 4

IMPLEMENTATION
A Retiner is a tool developed during a project CORTIVIS for modeling, simulating, testing and comparing the vision models developed. A retiner is a program primarily written in MATLAB.A screen shot of Retiner tool is as shown in fig 4.1.

Fig 4.1 Retiner tool Retiner is the name of the hardware/software environment for testing retina models. Its software front-end has been implemented using MATLAB .The main purpose of Retiner is to describe and simulate the retina processing. Through a set of parameterized filters and characteristics (which are very close to the hardware constraints for a digital implementation) we obtain a retina portable model that can be translated into a hardware description for automatic synthesis. According to the block diagram depicted in Fig. 3.5, Retiner includes:

Dept of CSE

19

MIT, Mysore

Visual Neuroprosthetics A) Photoreceptors: modeled after a camera sensor (preferably a logarithmic response camera), take the input from the image acquisition device and extract the three color channels (red, green and blue). A gain factor can be specified for every individual channel as well as a global gain. At the moment, the software allows acquisition of still images in most digital formats, webcams compatible outputs with Video for Windows and video streams in AVI format. B) Ganglion modules (filters): perform filtering operations over the input channels to extract or enhance relevant features of the scene. In the current implementation, three filters have been considered: a. MLvs S: Yellow vs Blue, performed by a Difference of Gaussians Filter (DoG) as the shown in Fig. 4.1. b. LSvs M: Red vs Green, another DoG. c. LMS: an achromatic filter, implemented as a laplacian filter. The ML vs S filter contribution is a DoG between the average of the green (M) and red channel (L), and the blue channel (S), where M, L and S stand for medium, large and short wavelength of light.Filter b is analogous to the first one, but using different color channels. C) Weighting module: that produces a user defined weighted sum of the three filter outputs. Any intermediate result (filter outputs and information figure) can be observed and saved to an image or video file. D) Electrode output configuration: allows selecting the number of electrodes in the stimulator matrix. The weighted filtered output will result in an assignation or mapping to a limited number of electrodes. The value assigned to an electrode has been computed by averaging the value in the pixels in its receptive field. At the moment the receptive field area is fixed, but it will be configurable for every electrode. An approximation to the cortical integration or the induced image the stimulation of electrodes would invoke in a implanted individual can also be displayed as an output (see example in Section V). This integration consists in a custom blurring of the microelectrodes output. The mapping will be made to be re-configurable, so that every output can be redirected to the convenient electrode. This
Dept of CSE 20 MIT, Mysore

Visual Neuroprosthetics flexibility will be useful in the case of damaged electrodes or adaptation to specific individual features, avoiding surgery in any case.

Fig 4.2 Example of mask used to implement a 2D DoG filter If conventional CCD and CMOS camera sensors are used, which have a linear response to light intensity? This feature can be a problem in visual scenes in which a high contrast exists. If an image presents areas of darkness and very brilliant zones, essential information at the entry point of our system will be lost, as most exposed pixels will get saturated. We would be missing important features of the scene. To overcome this disadvantage, a logarithmic response camera is being used. The response of this camera is distributed so that it is never saturared. With this camera, we are able to acquire scenes in

Dept of CSE

21

MIT, Mysore

Visual Neuroprosthetics which a brilliant source of light (even the sun) appears together with simple non-brilliant objects (see Fig. 4.2 for a comparison).

Fig 4.3 A view of a window in a shiny day CCD camera (left) gets saturated, losing details that can be observed with a logarithmic camera (right)

4.1

NEUROMORPHIC CODING
The output of the activity matrix, highlighting relevant information in the input

image, must be coded into neural-like pulses. The neuromorphic coding module is in charge of this task. The output of this stage will feed the radio-frequency link that goes to the microelectrode array. The model implemented is a simplified version of an integrate-and-fire spiking neuron. The neuron accumulates input values coming from its receptive field until it reaches a threshold. Then it fires and discharges the accumulated value. We also include a leakage term, to make the accumulated value diminish for low or null input values. Fig. 5 shows the spike or pulsed output obtained by the coding module in response to input activity pulses of different amplitude and duration. We can observe the effects of the leakage factor and the thresholding in the middle trace of the same figure. The more intense is the input value and the longest it is sustained, we will obtain longer spike trains. This part of the model has been developed with a XILINX blockset for SIMULINK, that allows easily building and testing hardware models and, after testing, generating automatically VHDL code for synthesis.
Dept of CSE 22 MIT, Mysore

Visual Neuroprosthetics

Fig 4.4 Successive values of a microelectrode activity (top). Integration values (middle). Pulse-coded output (bottom).

4.2

OBSTACLES IN IMPLEMENTATION
A neuroprosthetic system must be implanted into the nervous system and remain fully

functional for periods that will eventually extend too many decades. Therefore these devices must be highly biocompatible and be able to resist the attack of biological fluids, proteases, macrophages or any substances of the metabolism. Furthermore it is necessary to take into account the possible damage of neural tissues by permanent charge injection using multielectrode arrays and the most effective means of stimulating the cerebral cortex. These considerations place unique constraints on the architecture, material, and surgical techniques
Dept of CSE 23 MIT, Mysore

Visual Neuroprosthetics used in the implementation of neural interfaces. Once a particular type of electrode is selected, the next step is to design a surgical procedure for electrode implantation. Even though the individual microelectrodes of the UEA are extremely sharp, early attempts to implant an array of 10 x 10 electrodes into the visual cortex in different animal models only deformed the cortical surface and resulted in incomplete implantation. A system that rapidly inserts the UEA into the cortex has been developed allowing implantation in a manner that minimizes dimpling and compression of the subjacent structures. The implantation is so rapid that the cortex experience only slight mechanical dimpling and the insertion is generally complete. The most typical findings in acute experiments are occasional microhemorrhages emanating from the electrode tracks probably due to the high probability of electrode tips encountering one or more blood vessels during implantation. This typically resolves itself and aside from a few mechanically-distorted and somewhat hyperchromic neurons, the neurons near most tracks appear normal (Fig 4.4).

Fig 4.5Microhaemorrhages of Neurons

4.2.1 Mathematical Modeling

Accurate characterization of the nonlinear input/output (I/O) parameters of the normally functioning tissue to be replaced is paramount to designing a prosthetic that mimics normal biologic synaptic signals. Mathematical modeling of these signals is a complex task "because of the nonlinear dynamics inherent in the cellular/molecular mechanisms comprising neurons and their synaptic connections."The output of nearly all brain neurons is dependent on which post-synaptic inputs are active and in what order the inputs are received.

Dept of CSE

24

MIT, Mysore

Visual Neuroprosthetics Once the I/O parameters are modeled mathematically, integrated circuits are designed to mimic the normal biologic signals. For the prosthetic to perform like normal tissue, it must process the input signals, a process known as transformation, in the same way as normal tissue.

4.2.2 Size
Implantable devices must be very small to be implanted directly in the brain, roughly the size of a quarter. One of the example of micro implantable electrode array is the Utah array. Wireless Controlling Devices can be mounted outside of the skull and should be smaller than a pager.

4.2.3 Power Consumption

Power consumption drives battery size. Optimization of the implanted circuits reduces power needs. Implanted devices currently need on-board power sources. Once the battery runs out, surgery is needed to replace the unit. Longer battery life correlates to fewer surgeries needed to replace batteries. One option that could be used in the medical field to recharge implant batteries without surgery or wires is being used in powered toothbrushes. These devices make of inductive coupling to recharge batteries. Another strategy is to convert electromagnetic energy into electrical energy, as in radio frequency identification tags.

4.2.4 Bio Compatibility

Cognitive prosthetic are implanted directly in the brain, so biocompatibility is very important obstacle to overcome. Materials used in the housing of the device, the electrode material (such as iridium oxide), and electrode insulation must be chosen for long term implantation. Subject to Standards: ISO 14708-3 2008-11-15.Crossing the Blood Brain Barrier can introduce pathogens or other materials that may cause an immune response. The brain has its own immune system that acts differently than the immune system of the rest of the body.

4.2.5 Data Transmission

Wireless Transmission is being developed to allow continuous recording of neuronal signals of individuals in their daily life. This allows physicians and clinicians to capture more
Dept of CSE 25 MIT, Mysore

Visual Neuroprosthetics data, ensuring that short term events like epileptic seizures can be recorded, allowing better treatment and characterization of neural disease. Methods of data transmission must be robust and secure. Neurosecurity is a new issue. Makers of cognitive implants must prevent unwanted downloading of information or thoughts from and uploading of detrimental data to the device that may interrupt function.

4.2.6 Correct Implantation

Implantation of the device presents many problems. First, the correct presynaptic inputs must be wired to the correct postsynaptic inputs on the device. Secondly, the outputs from the device must be targeted correctly on the desired tissue. Thirdly, the brain must learn how to use the implant. Various studies in brain plasticity suggest that this may be possible through exercises designed with proper motivation.

4.3

CASE STUDY OF VISUAL NEURO PROSTHETICS

A major prerequisite for the possible clinical application of this neuroprosthetic

approach, aside from safety considerations, is proper, non-invasive patient selection criteria. The visual cortex of potential candidates for such neuroprosthetic devices has to be capable of processing visual information, but there is evidence that the occipital parts of the brain utilized by sighted subjects to process visual information are transformed in some blind subjects and utilized to process tactile and auditory stimuli. Figure 4.5 shows an example. It displays the areas of the brain that are activated when a sighted person reads Braille characters (Fig 4.5A). The activated areas are displayed in red and represent the activation of the somatosensory and motor cortex on the left side of the brain that process tactile information from the right hand being used for tactile decoding of the symbols. The above figure contrast with Fig 4.5B, which shows the areas activated when an early blind subject read Braille characters. The areas of activation are not restricted to the somatosensory and motor cortex contra laterally to the right hand used for reading. Rather, there is marked activation of the occipital cortex, the part of the brain that we use to process visual information, the primary visual cortex. This cross-modal plasticity, by which the visual cortex is recruited for processing of tactile information, is associated with a significant

Dept of CSE

26

MIT, Mysore

Visual Neuroprosthetics improvement in the tactile reading skill and is supported by a variety of additional, converging data.

Dept of CSE

27

MIT, Mysore

Visual Neuroprosthetics

CHAPTER 5

TECHNOLOGY CONSIDERED IN MARKET

Visual prosthetics are being developed as a potentially valuable aid for individuals with visual degradation. The visual prosthetic in humans remains investigational.Let us consider few technologies in market.

5.1 Visual Cortical Implant

Fig 5.1 Visual cortical Implant

Dr. Mohamad Sawan, Professor and Researcher at Polystim neuro technologies Laboratory at the Ecole Polytechnique de Montreal, has been working on a visual prosthesis to be implanted into the visual cortex. The basic principle of Dr. Sawans technology consists of stimulating the visual cortex by implanting a silicon microchip on a network of electrodes, made of biocompatible materials, wherein each electrode injects a stimulating electrical current in order to provoke a series of luminous points to appear (an array of pixels) in the field of vision of the blind person. This system is composed of two distinct parts: the implant and an external controller. The implant is lodged in the visual cortex and wirelessly receives data and energy from the external controller. It contains all the circuits necessary to generate the electrical stimuli and to monitor the changing microelectrode/biological tissue interface. The battery-operated outer controller consists of a micro-camera, which captures images, as well as a processor and a command generator, which process the imaging data to translate the captured images and generate and manage the electrical stimulation process. The external

Dept of CSE

28

MIT, Mysore

Visual Neuroprosthetics controller and the implant exchange data in both directions by a transcutaneous radio frequency (RF) link, which also powers the implant.

5.2 Intracortical Visual Prosthesis

The Laboratory of Neural Prosthesis at Illinois Institute of Technology (IIT), Chicago, is developing a visual prosthetic using Intracortical Iridium Oxide (AIROF) electrodes arrays. These arrays will be implanted on the occipital lobe. External hardware will capture images, process them and generate instructions which will then be transmitted to implanted circuitry via a telemetry link. The circuitry will decode the instructions and stimulate the electrodes, in turn stimulating the visual cortex. The group is developing a wearable external image capture and processing system. Studies on animals and psychophysical studies on humans are being conducted to test the feasibility of a human volunteer implant

5.3 Harvard/MIT Retinal Implant

Joseph Rizzo and John Wyatt at the Massachusetts Eye and Ear Infirmary and MIT began researching the feasibility of a retinal prosthesis in 1989, and performed a number of proof-of-concept epiretinal stimulation trials on blind volunteers between 1998 and 2000. They have since developed a subretinal stimulator, an array of electrodes that is placed beneath the retina in the subretinal space and receives image signals beamed from a camera mounted on a pair of glasses. The stimulator chip decodes the picture information beamed from the camera and stimulates retinal ganglion cells accordingly. Their second generation prosthesis collects data and sends it to the implant through RF fields from transmitter coils that are mounted on the glasses. A secondary receiver coil is sutured around the iris.

5.4 Artificial Silicon Retina (ASR)

The brothers Alan Chow and Vincent Chow have developed a microchip containing 3500 photo diodes, which detect light and convert it into electrical impulses, which stimulate healthy retinal ganglion cells. The ASR requires no externally-worn devices.[5]

Dept of CSE

29

MIT, Mysore

Visual Neuroprosthetics The original Optobionics Corp. stopped operations, but Dr. Chow acquired the Optobionics name, the ASR implants and will be reorganizing a new company under the same name. The ASR microchip is a 2mm in diameter silicon chip (same concept as computer chips) containing ~5,000 microscopic solar cells called microphotodiodes that each have their own stimulating electrode.

5.5 Optoelectronic Retinal Prosthesis

Daniel Palanker and his group at Stanford University have developed an optoelectronic system for visual prosthesis that includes a subretinal photodiode array and an infrared image projection system mounted on video goggles. Information from the video camera is processed in a pocket PC and displayed on pulsed near-infrared (IR, 850-900 nm) video goggles. IR image is projected onto the retina via natural eye optics, and activates photodiodes in the subretinal implant that convert light into pulsed bi-phasic electric current in each pixel. Charge injection can be further increased using a common bias voltage provided by a radiofrequency-driven implantable power supply [13] Proximity between electrodes and neural cells necessary for high resolution stimulation can be achieved utilizing the effect of retinal migration.

5.6 Dobelle Eye

Similar in function to the Harvard/MIT device, except the stimulator chip sits in the primary visual cortex, rather than on the retina. Many subjects have been implanted with a high success rate and limited negative effects. Still in the developmental phase, upon the death of Dr. Dobelle, selling the eye for profit was ruled against in favor of donating it to a publicly funded research team

Dept of CSE

30

MIT, Mysore

Visual Neuroprosthetics

CHAPTER 6

APPLICATIONS
6.1 Traumatic Brain Injury
More than 1.7 million people in the United States suffer traumatic brain injury.Orthosis for TBI patients to control limb movement via devices that read neurons in brain, calculate limb trajectory, and stimulate needed motor pools to make movement. (Anderson Paper, Cole at NIH - specifically "Computer software as an orthosis for Brain Injury",)Parkinson's disease Nearly 1 million people in the United States are affected by Parkinson's disease. Deep Brain Stimulation relieves symptoms of Parkinson's Disease for numerous patients.[30] Parkinson's Disease patients could benefit from a cortical device that mimics the natural signals needed to promote dopamine production. Another possible avenue for mitigation of PD is a device that supplements dopamine when given specific neuronal inputs which would let the body regulate dopamine levels with its intrinsic sensors.

6.2 Speech Deficits

Approximately 7.5 million people in the United States have trouble speaking.[31] Many of these can be attributed to aphasias. The success of cochlear implants suggests that cortical implants to the speech areas of the brain can be developed to improve speech in such patients.

6.3 Paralysis
According to the Christopher and Dana Reeve Foundation's Paralysis Resource Center, approximately 6 million people are living with paralysis in the United States. Paralysis results from many sources, stroke, traumatic brain injury, neurodegenerative diseases like multiple sclerosis and Lou Gehrig's Disease, and congenital sources. Many patients would benefit from a prosthetic device that controls limb movement via devices that read neurons in brain, calculate limb trajectory, and stimulate the needed motor pools to make movement. This technology is being developed at the Andersen Lab, located at the California Institute of Technology. The goal is to develop a device to enable locked in patients, those without the ability to move or speak, to communicate with others.
Dept of CSE 31 MIT, Mysore

Visual Neuroprosthetics

6.4 Spinal Cord Injuries

Neuroprosthetics have been shown to be an effective and safe method in restoring hand movement in adults following spinal cord injuries. This neuroprosthesis consists of an implanted receiver-stimulator, an external shoulder position sensor and a terminal electrode. The terminal electrode is placed on the motor point of a muscle, this enables a low electrical threshold to be utilized. The external sensor measures voluntary movements that occur in the counter lateral (opposite) shoulder and bases motor output commands on this information. A radiofrequency signal is then transmitted to the implanted receiver stimulator and is later converted to electrical stimuli that depolarize the peripheral nerve. Evaluations of the neuroprosthetic are preformed based on clinical outcome which measure the improvement of hand function on scales of impairment and performance of daily living. [32]

6.5 Societal Impact/Market Information

Nearly disease. 1
[29]

million

people

in

the

United

States

are

affected

by

Parkinson's

Alzheimer's Disease is projected to affect more than 107 million people worldwide

by the year 2050.[26]Just these two diseases indicate that there is already a large market for cognitive neural prosthetics, with more potential markest space revealed in traumatic brain injury and speech problems (particularly damage to Broca's or Wernicke's areas). More than 1.4 million people in the United States suffer traumatic brain injury. Approximately 7.5 million people in the United States have trouble speaking.[31] Many of these can be attributed to aphasias. More than 6.5 million people in the United States have suffered stroke.

Dept of CSE

32

MIT, Mysore

Visual Neuroprosthetics

CONCLUSIONS
Clinical applications such as artificial vision require extraordinary diverse, lengthy and intimate collaborations among basic scientists, engineers and clinicians. On-going research on the anatomy and physiology of the visual pathways will yield a better understanding of the parallel processing capabilities of the central nervous system and the role of neural plasticity in the interpretation of visual information. The strategy of using a combination of experimentation and modeling to understand the mechanisms of visual coding will allow the design and development of fast and flexible bioinspired systems able to process signals from external devices before they are fed into a machine-brain interface to safely stimulate the nervous system. However there are many questions regarding

biocompatibility, safety and even nano technological issues that need to be addressed before a cortical neuroprosthesis can be considered a viable clinical therapy. The implant is to be inserted directly into the brain, where it will remain without intervention for decades. Further, the system will not be a simple passive device, but it will contain active circuitry for multiplexing and telemetry. While some progress has been made in many of these fields, it is clear that more animal experiments are needed. Although the full restoration of vision seems to be impossible, the discrimination of shape and location of objects could allow blind subjects to navigate in a familiar environment and to read enlarged text, resulting in a substantial improvement in the quality of life of blind and visual impaired persons. However it is necessary to go step by step, not creating false expectations that could negatively affect this emerging approach. We are still a long way away from a highly functional visual prosthesis, but the success of the cochlear implant encourages the pursuit of this neuro technological application. The plastic changes in the brain of blind subjects allows them to extract greater information from touch and hearing, thus improving their quality of life and enhancing their integration in the social and working environment of a sighted society. Neuroprosthetics have already enabled the immobile to operate computers by thought alone, the partially paralyzed to walk and groom themselves, the deaf to hear, and the blind to see. Its wide spread developments had created aspirations in the minds of even the
Dept of CSE 33 MIT, Mysore

Visual Neuroprosthetics paralyzed that they have a chance to live. Scientists are still working very hard in this field. As seen from the previous pages this is of immense importance in this developing era. This in future, if developed widely, may lead to a world with much reduced physical defects. Fulfilling the dream of reducing the distance between the mind and knowledge is thus not far away.

Dept of CSE

34

MIT, Mysore

Visual Neuroprosthetics

BIBLIOGRAPHY
[1] CORTICAL VISUAL NEUROPROSTHETIC FOR THE BLIND published by Instituto de Bioingeniera

[2] IEEE paper Cortical Visual Neuro-Prosthesis for the Blind: Retina-Like Software/Hardware Preprocessor published by E. Ros, E. FernndezDept. of Computer Architecture and Technology, University of Granada, Spain

[3] http://www.99main.com/~charlief/Blindness.htm

[4] http://en.wikipedia.org/wiki/Visual_prosthesis

Dept of CSE

35

MIT, Mysore