Advances in Invasive Brain–Computer Interface Technology and Decoding Methods for Restoring Movement and Future Applications

Abstract

Many advances have been made in the field of invasive brain–computer interface (BCI) technology in recent years, including improved implantable microelectrodes, better-performing neural decoding algorithms, and the development of invasive BCI systems that have helped restore functional movement in paralyzed clinical study participants. This chapter describes historical developments that have paved the way for recent achievements in the invasive BCI field. Critical technology components of an invasive BCI system are described, and future applications and directions for the field are discussed.

Keywords

Brain–computer interface, Decoding, Implantable electrodes, Machine learning, Microelectrodes, Movement restoration, Neural interface, Paralysis, Sensory restoration, Signal processing, Spinal cord injury, Stroke, Traumatic brain injury

Outline
Introduction 415
Historical Perspective 415
Restoring Movement in Quadriplegia 417
Designing and Developing Effective Brain–Computer Interface Systems 418
- Neural/Brain Interface 418
- Amplifying and Digitizing Neural Activity 420
- Robust Neural Features for Long-Term Decoding 421
- Neural Decoding Algorithms 423
Conclusion 423
References 423

Introduction

There have been many new developments in the invasive brain–computer interface (BCI) field with respect to computational and processing methods that have deepened our understanding of how the brain works and allowed neural signals to be deciphered and harnessed for a wide array of applications. These applications include thought-controlled computer interfaces for disabled users, and most recently technologies specifically aimed at restoring movement in people living with paralysis. With continued advancements in the areas of implantable microelectrodes, advanced signal processing, and neural decoding methods, the list of applications for BCI continues to expand. In the future we can expect to see increased research and development efforts related to decoding neural patterns and technologies that interface with not only the brain but also nerves throughout the body. These technologies will allow us to tap into and decipher the vast amount of information present to help treat and potentially diagnose a wide variety of conditions. With new brain–computer and neural interfaces and methods to decode neural signals, we will at some point in the future be able to understand the language of the human nervous system, leading to new treatment options for the millions of patients worldwide living with disease and debilitating injuries.

Historical Perspective

Examples of early research efforts that paved the way for BCIs and advances in neural decoding are studies that recorded electrical activity in the brain to understand better how sensorimotor information is encoded ( ). In subsequent work differing neuronal firing patterns were observed during passive and active movements ( ). Furthermore, neurons were identified that had modulation patterns correlating to force ( ). Important discoveries such as “directional tuning” were made, where neuronal firing rate changes in the motor cortex were associated with different arm trajectories and certain neurons were found to have a “preferred” direction ( ). Later, with the increased availability of more sophisticated data collection and computer systems, more advanced analyses were performed to deepen understanding of the patterns in neuronal populations ( ). Since each neuron in the brain is connected to a large number of other neurons, understanding pattern changes (modulation) of large neuronal populations is critical to further our knowledge of how the brain represents and processes information.

Initially electrodes used to probe and collect data in the brain were made exclusively by hand in the lab (typically limited to a handful of recording sites), making it difficult to study large populations of neurons and network behavior. However, implantable electrode technology evolved as microfabrication techniques were developed to produce more precisely manufactured electrode arrays with more recording sites. One such design, developed at the University of Utah, is fabricated by etching silicon to form “spikes” with recording sites at their tips (see Fig. 29.1 ). Another design, developed at the University of Michigan, is made using a thin-film process to create flexible thin shanks that can be inserted into the brain to record signals. Yet another type of electrode array used in BCIs is an electrocorticography (ECoG) array or grid. ECoG arrays do not penetrate into the cortex and can be placed on top of or below the dura to record neural signals that are useful for neural decoding. Unlike the first two types of electrodes mentioned, the ECoG array cannot record single-unit action potentials and the electrode spacing is typically larger. As discussed later, these differences are important because they affect the specificity of the signal recorded, and need to be considered when developing BCI systems and neural decoding methods for various applications.

In the 1990s an area of research called machine learning, branching from the field of artificial intelligence, began to grow rapidly ( ). Machine-learning algorithms were developed and could be run on personal computers to handle increasingly large datasets, making them well suited for deciphering brain activity data. Machine-learning methods could be used to find and learn patterns in complex datasets for later automatic recognition when these patterns reoccurred. Thoughts in the brain are reflected in the electrical activity patterns generated by neurons “firing.” If this electrical activity is detected and recorded, neural decoding algorithms can be trained to recognize brain activity associated with certain thought patterns, such as thoughts about movement. This fueled research activity in neural decoding, opening the possibility of interpreting brain activity patterns so scientists and engineers could, in a limited way, recognize thought patterns and “read the mind.”

With the advancements of neural interface technology and neural decoding methods, the possibility of linking brain signals to computers and devices was beginning to become a reality. Researchers began to think about the possibility of allowing a disabled person to have thought-control of a computer or assistive device to restore lost functions such as speech, and at some time potentially movement. A new area of invasive BCI technology and new applications had been launched.

Continued research in nonhuman primates demonstrated that it was possible to decode movement-related thoughts and allow the animals to control robotic arms with their thoughts ( ). Researchers began to study the behavior of neuronal populations by using multielectrode arrays and developing more sophisticated neural decoding algorithms to correlate neural activity with movement in three-dimensional space ( ). Neural decoding was later combined with neuromuscular stimulation to allow volitional control of temporarily paralyzed arm muscles in nonhuman primates ( ). These initial research findings provided the foundation for subsequent development of invasive BCI technology and neural decoding methods for human applications.

In the late 1990s researchers began to develop invasive BCI technology and neural decoding methods with clinical applications in mind. One of the first studies involved the use of neural decoding for restoration of speech ( ). Other studies were planned and conducted for participants living with paralysis due to spinal cord injury (SCI), stroke, and amyotrophic lateral sclerosis to allow direct thought-control over cursor movement in personal computers ( ). As the patient imagined or attempted various arm and hand movements, neural decoding algorithms were trained to recognize the neural modulation patterns associated with each movement. Once a decoding algorithm was trained and in place, the patients could use their thoughts to control not only a cursor on a computer but also robotic devices ( ).

Restoring Movement in Quadriplegia

With previous successes in restoring movement in temporarily paralyzed muscles of a nonhuman primate, other groups began to work toward restoring movement in persons living with paralysis. This involved using an electronic “neural bypass” to reroute signals effectively from the brain to paralyzed muscles in an effort to restore functional movement. In 2016 results from a first in-human study were reported, documenting the use of a cortical implant to restore movement in a paralyzed human ( ). The study participant was a 24-year-old male with stable, nonspastic, C5/C6 quadriplegia from cervical SCI sustained in an accident 4 years earlier. The BCI system decoded intracortically recorded signals and linked them to a custom neuromuscular stimulation system to restore volitional control over hand movement. As shown in Fig. 29.2 , a tiny electrode array (4 mm × 4 mm across) with 96 electrodes (1.5 mm long) was implanted in the motor cortex and a custom neuromuscular electrical stimulation sleeve was used for muscle activation.

The participant was able to regain continuous volitional wrist and dexterous finger movements through his own thoughts. He was also able to achieve functional movements relevant to daily life, as shown in Fig. 29.3 . This was a large step forward in BCIs and helped pave the way for future medical devices for treating SCI, stroke, traumatic brain injury, severe nerve injuries or degeneration, and other conditions where movement is severely impaired.

One of the challenges in using an electronic neural bypass to reroute signals around a damaged spinal cord is that the central pattern-generator neural networks used in rhythmic movement are bypassed completely. In recent work artificial central pattern generators were developed in an attempt to replace this lost function. The study participant was able to activate these artificial pattern generators through thoughts ( ), and could think about static and dynamic/rhythmic movements, such as flexing and wiggling the fingers or wrist (as in movements used during teeth brushing or scratching), and switch between the two volitionally, as shown in Fig. 29.4 .

Designing and Developing Effective Brain–Computer Interface Systems

To do any job effectively you need planning, the right raw materials, and the appropriate tools. To decode neural activity accurately, the tools used to measure the neural activity need to give quality signals with sufficient information content for the task the user is being asked to accomplish. Providing quality signals for decoding requires the right neural/brain interface (e.g., electrodes), amplifier circuitry, an analog-to-digital converter (ADC), and feature extraction and decoding algorithms to allow thought-control over a device. This type of BCI system architecture is shown in Fig. 29.5 , and the development of such a system must be carefully planned with several objectives in mind.

A BCI system must acquire signals with the appropriate amount of information content for the targeted application. In movement restoration applications, for example, the task may involve volitional control of multiple body joints. The signals being recorded therefore need to be informative for all of the degrees of freedom involved in the movement desired. Decoding multiple degrees of freedom requires recording signals from multiple neurons or signals that represent multiple neurons (such as local field potential and multiunit activity). Signals are informative when one (or more) of the signals changes in a detectable and repeatable way when a user imagines different movements he/she would like to make. Thus the quality of the signals (how their modulation compares to the noise present) must be sufficient and the information content (the amount of the repeatable change present for different imagined movements) must be suitable as well.

Neural/Brain Interface

The neural/brain interface is what provides the “raw material” for the BCI system, and is key to acquiring quality signals containing useful information. The neural/brain interface must have a sufficient number of recording sites (e.g., electrodes) to measure neural activity—activity that is preferably unique from any neighboring site. Another approach is to decompose a signal recorded from a single site/electrode into various components known as features for input into the decoding algorithm. This is often referred to as feature decomposition or feature extraction process, and is discussed in the next section. Researchers use one or both of these methods, depending on the decoding application and practical limitations. When using both, a large set of features can be obtained and feature selection can become important, so that only useful features are used for decoding purposes.

Selecting or developing the right neural/brain interface in a BCI system can be an arduous task. Consideration of what is appropriate or practical for the user in the application at hand and the signal the interface will provide for neural decoding is extremely important. Neural/brain interfaces can be noninvasive or invasive. Focusing on the use of electrodes as the interface type, the signal quality and signal content vary depending on which electrode type is selected. These various types include electroencephalography (EEG), ECoG, and penetrating electrodes. Each provides a signal with differing information content and needs a different procedure to place it, which requires careful consideration.

EEG electrodes can be used in a noninvasive BCI system and placed on the scalp to collected brainwaves produced by large populations of neurons firing in the brain ( ). EEG is advantageous in that it is noninvasive and does not required surgical intervention to place. However, EEG signals can be contaminated by motion artifact (scalp/electrode movement) and are separated further from the neurons, the source of the neural information desired, compared to ECoG and penetrating electrodes. This distance and the presence of other tissue (dura, bone, muscle, skin, and hair) reduce the quality of the signal and its information content, making it more challenging to use for applications with high degrees of freedom. Nonetheless, many decoding algorithms for EEG signals have been developed and used in a variety of applications. In one study participants used EEG-based BCI technology to control a robotic arm for reaching, grasping, and moving an object in three-dimensional physical space ( ). It has also been demonstrated that an EEG-based BCI system can allow human subjects to control a quadcopter in three-dimensional space ( ).

Unlike EEG electrodes, ECoG electrodes are invasive and are placed under the bone to allow them to be closer to the brain and provide higher spatial resolution ( ). They are intended to rest on the surface of the brain, so placement requires surgery either above or below the dura. ECoG electrode arrays are composed of a thin, flexible substrate material with multiple recording sites. Polyimide or silicon is spin-coated on to a wafer and metalized (or foil is used) and then etched or laser cut to leave traces and recording sites ( ). Another layer of the insulating material is placed over the traces and the insulation is removed via an ultraviolet laser at the recording sites. ECoG electrode arrays have been successfully used for invasive BCI applications such as one-dimensional cursor control ( ), and also in epilepsy applications for detection of seizure onset activity ( ). This type of electrode has been used chronically as well, and may offer stable signals over extended periods (years) due to a reduced inflammatory response in the brain when compared to some existing penetrating electrodes ( ).

Penetrating electrodes have been shown to provide quality signals with sufficient information content for applications with multiple degrees of freedom in nonhuman primates and humans ( ). Different styles of implantable electrodes have been developed through the years. The two designs described above, the Utah array and the Michigan-style electrode, have been used in multiple labs around the world. The Utah array is fabricated by beginning with a piece of silicon and etching it down until relatively sharp shanks are left (in an equally spaced pattern), which are then metalized and coated with parylene-C to insulate the shanks ( ). The insulation at each tip is removed by etching or laser. Some failures in these types of arrays have been observed in the past, depending on how long they had been used in the body ( ). The Utah array (see Fig. 29.1 ) was commercialized by Blackrock Microsystems Inc. and obtained United States Food and Drug Administration (FDA) clearance for 30-day monitoring of cortical activity. This array has been used under Investigational Device Exemption by the FDA in multiple long-term clinical studies involving movement and sensory research ( ). The Michigan-style electrodes are fabricated in a thin-film process where a sheet of silicon is used to form the implant. Photolithography techniques are used to form the metalized patterns (traces and recording sites) on the silicon ( ). The metalization is then insulated by coating the structure with parylene-C and the recording sites are exposed. These penetrating electrodes have the advantage of having multiple recording sites along their shanks (depth direction in the brain), but they too can suffer from failures during chronic implantation ( ).

Amplifying and Digitizing Neural Activity

The type and location of the amplifiers in a BCI system are extremely important. Due to the tiny voltages neuronal activity generates when measured via an electrode, the amplifiers must have high input impedance and extremely low noise characteristics. Advances in ultralow-power and low-noise amplifiers for multielectrode BCI systems have been made in recent years ( ). The location of the amplifier is also important, and placing it as close to the electrodes as possible will reduce the effects of stray interference. Finally, high-frequency electromagnetic interference that resides in the frequency range above the range of interest for neural signals can be filtered out. However, 60 Hz noise from nearby equipment coupled into the BCI system through parasitic capacitance can be difficult to filter, as it falls directly within the frequency range of interest. In this case, notch filters can be used to remove the 60 Hz noise and its harmonics. Performing a spectral analysis of the signals before and after applying the filter is recommended to evaluate the effectiveness of the filtering used.

After amplification, digitization of the neural signals for storage or real-time decoding purposes is often required in BCI systems. A variety of ADCs can be used, given that their dynamic range, sampling frequency, and number of channels are sufficient. Scanning or multiplexing ADCs can be used to handle a large number of channels (electrodes) in the BCI system, but the settling time of the multiplexing circuitry needs to be short enough so as not to impact on the accuracy of the measurement. In one example a high-speed analog multiplexer was combined with successive-approximation, charge-redistribution ADC architecture to interface with 100 electrodes while achieving low power consumption ( ).

Robust Neural Features for Long-Term Decoding

At the heart of every BCI system are algorithms that extract information from the neural signals recorded. In the field of machine learning and neural decoding, a feature is a measurable property of the signal of interest. In clinical applications features should be robust and remain stable over the entire clinical use period, which may be years for chronic implant scenarios. Feature-extraction algorithms must accurately recognize patterns in acquired signals, even in the presence of noise. Analogous to how we learn language early in life, the first stage is to learn how to recognize basic features (phonemes) before learning more complex combinations of those sounds (words and sentences). In this way useful information, such as motor cortex signals representing the desire to close the hand on an object, can be recognized and isolated or “extracted” in a continuous stream of data. Subsequently more complex sequences can be recognized, such as the intent to close the hand and pick up an object.

Useful features contained in recorded brain signals include single-unit activity (SUA), multiunit activity (MUA), and local field potential (LFP). SUA represents axonal output of neurons in close proximity to the recording electrode ( ), and is often recorded from penetrating electrodes which have recording sites at their tips and can be placed in close proximity to neurons of interest. These penetrating electrodes can, however, cause an inflammatory tissue response, which often forms an insulating glial sheath around the electrode ( ). More recently researchers have found that local neurodegeneration may actually be the primary cause of SUA loss over time in penetrating electrodes ( ). Fig. 29.6 shows the decrease in SUA over time in a chronically implanted penetrating electrode array in nonhuman primates ( ). Researchers have also explored thinner, more flexible electrodes made of carbon fiber that have been shown to reduce the inflammatory response ( ).